onnxruntime-extensions: Heap OOB Write in LogMelSpectrum

1. Background

I was reading through the audio preprocessing code in microsoft/onnxruntime-extensions, looking at how Whisper ONNX models feed audio into the LogMelSpectrum custom op. LogMel::Compute() in shared/api/speech_features.hpp allocates an output tensor and fills it via a row-by-row copy from the mel filterbank output. The output buffer is sized once at model initialization using expected_time, derived from n_samples_ and hop_length_. The copy loop reads mag_time elements per row, computed at runtime from the STFT output tensor shape. The two values are used together with no check between them.

2. The vulnerable code

Inside LogMel::Compute(), the output buffer is allocated for mel_freq × expected_time floats:

/* speech_features.hpp:264 */
const int64_t expected_time = n_samples_ / hop_length_;

/* speech_features.hpp:266 */
buff = logmel.Allocate({mel_freq, expected_time});

mag_time comes directly from the STFT output tensor shape with no validation against expected_time:

const int64_t stft_time = stft_norm.Shape()[2];
const int64_t mag_time = stft_time - 1;

The copy loop then runs mag_time elements per row into a buffer sized for expected_time elements per row:

/* speech_features.hpp:273-279 */
for (int m = 0; m < mel_freq; ++m) {
    std::copy(
        log_spec.begin() + m * mag_time,
        log_spec.begin() + m * mag_time + mag_time,
        buff + m * expected_time    /* row slot holds expected_time floats only */
    );
}

When mag_time > expected_time, every iteration copies more data than the row slot holds. The only guard in the function is assert(stft_norm.Shape().size() == 3), which checks the tensor rank, not the time dimension, and is stripped in release builds.

3. How far it overflows

For a standard Whisper model: mel_freq = 80, hop_length = 160, n_samples_ = 480000 (30 seconds at 16kHz), giving expected_time = 3000. The output buffer is 80 × 3000 × 4 = 960000 bytes.

For audio of T seconds at 16kHz:

stft_time ≈ T × 16000 / hop_length
mag_time  = stft_time - 1 ≈ T × 100

At T = 31 seconds: mag_time ≈ 3100. Each row of the copy loop reads 3100 floats into a 3000-float slot. The overflow per row is 100 floats = 400 bytes, and it compounds across all 80 rows:

overflow per row:   100 floats × 4 bytes = 400 bytes
total overflow:      80 rows  × 400 bytes = 32,000 bytes

Each additional second of audio past the chunk boundary adds another ~100 floats per row, scaling the overflow by 32KB per second. The overflow size is directly controlled by the audio input length.

4. Reachability

The overflow requires mag_time > expected_time at the point the copy loop runs. Whether that condition is met depends on the pipeline configuration upstream of LogMel::Compute().

The bundled default Whisper configuration is not vulnerable. AudioDecoder runs with max_samples_ = 480000 by default, which truncates input to exactly 30 seconds. Combined with chunk_size = 30 in the LogMelSpectrum config, this keeps mag_time ≤ expected_time regardless of how long the audio input is.

Three deployment patterns are vulnerable:

• AudioDecoder with max_samples = 0 — disables truncation entirely. Any service processing long-form audio (meetings, interviews, podcasts) commonly sets this. Audio of duration T produces mag_time ≈ T × 100, so the overflow grows without a hard ceiling.
• Mismatched chunk_size and max_samples_ — for example, chunk_size = 15 with the default max_samples_ = 480000. Audio is truncated to 30 seconds but expected_time = 1500, so any input above 15 seconds produces mag_time ≈ 3000 > 1500 — an overflow of 1500 floats per row × 80 rows × 4 bytes = 480KB even on a 30-second clip.
• Custom pipeline bypassing AudioDecoder — any deployment wiring LogMelSpectrum directly after a custom STFT or from a tensor input loses the truncation gate entirely.

In the first two cases an unauthenticated client submitting audio to a transcription endpoint can trigger the overflow with a single request. The overflow size is proportional to input audio length, so the attacker controls how much is written.

5. ASAN confirmation

I wrote a standalone C++ reproducer (poc_ortx016.cpp) that models the exact control flow from speech_features.hpp:263–279: allocate a buffer for mel_freq × expected_time floats, then run the copy loop with mag_time = 3100 and expected_time = 3000. Compiled with -fsanitize=address:

ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7f99bd3fee00

WRITE of size 12400 at 0x7f99bd3fee00 thread T0
    #0 memmove (/usr/lib/libasan.so.8)
    #1 trigger_bug() poc_ortx016.cpp:20
           std::copy(log_spec.begin() + m * mag_time,
                     log_spec.begin() + m * mag_time + mag_time,
                     buff + m * expected_time);
    #2 main poc_ortx016.cpp:28

0x7f99bd3fee00 is located 0 bytes after 960000-byte region
allocated in trigger_bug() poc_ortx016.cpp:11

buffer allocated: 960000 bytes  (mel_freq=80 × expected_time=3000 × 4 bytes)
overflow amount:   32000 bytes  (mel_freq=80 × 100 floats/row × 4 bytes)
overflow per row:    400 bytes  (mag_time=3100 − expected_time=3000 = 100 floats)

ASAN catches the write at the last row (m = 79): buff + 79 × 3000 = buff + 237000, copying 3100 floats writes the final 100 past the end of the 960KB allocation. The reported write size of 12400 bytes is the full std::copy operation for that row (3100 × 4 bytes), of which the trailing 400 bytes are out of bounds.

6. The fix

The copy loop needs to know the actual number of time frames to copy per row before touching the output buffer. The simplest correct approach is to clip mag_time to expected_time before the loop, matching the behavior the default pipeline achieves through truncation upstream:

+const int64_t copy_time = std::min(mag_time, expected_time);
+
 for (int m = 0; m < mel_freq; ++m) {
     std::copy(
         log_spec.begin() + m * mag_time,
-        log_spec.begin() + m * mag_time + mag_time,
+        log_spec.begin() + m * mag_time + copy_time,
         buff + m * expected_time
     );
 }

An alternative is to return an error when mag_time > expected_time, which makes the mismatch visible to the caller rather than silently dropping the excess frames. Either approach fixes the OOB write. The clip is the lower-risk change for deployed code since it preserves the output shape contract.

7. Disclosure

Reported to Microsoft Security Response Center on 2026-05-04 (MSRC case 115486, VULN-185992). MSRC assessed the issue as moderate severity and confirmed reachability in configurations where AudioDecoder.max_samples = 0 or where chunk_size is smaller than the effective audio cap, but concluded that the default bundled configuration is not affected and rated the case below the threshold for immediate servicing. No CVE was issued. The report was shared with the engineering team for awareness.

The fix landed in the main branch on 2026-07-01 (commit 8e7b615, PR #1087). The coordinated disclosure window requested by MSRC elapsed on 2026-07-09. This post is published after that date.

Other recent work: an out-of-bounds array access in ieee80211_ml_epcs() in the mac80211 WiFi stack (separate post), a heap buffer overflow in the iSCSI target CHAP authentication code (separate post), and a series of heap overflow and OOB read fixes in the rtl8723bs staging WiFi driver (separate post).