file/libmagic — OOB Read in ELF Core Note Parser

A missing bounds check in file's ELF core note parser. When processing a FreeBSD-style NT_PRPSINFO note, the code reads the process name field without first verifying that the field's offset falls within the note's data buffer. A crafted ELF core file with a small NT_PRPSINFO descriptor triggers an out-of-bounds read.

1. Context

file(1) is the standard Unix utility for identifying file types by inspecting content rather than extension — it reads a file's first bytes, matches them against a database of "magic" signatures, and returns a human-readable description: "ELF 64-bit LSB executable", "JPEG image data", "PDF document", and so on. The underlying library is libmagic, which is used directly by antivirus engines, intrusion detection systems, web server upload validators, and any security tool that needs to classify files before processing them. The critical property of file and libmagic is that they are specifically designed to inspect untrusted input — that makes a memory safety bug in their parsers more serious than one in, say, an image editor where the user controls what files they open.

I found this bug while auditing src/readelf.c, the code responsible for parsing ELF-format files. ELF is not just for executables and shared libraries — it is also the format used for core dumps. When a program crashes (for example, from a segfault or an unhandled signal), the kernel writes a core file containing the process's memory snapshot, register state, and structured metadata stored in ELF note sections. Core files are a standard forensic artifact that developers and analysts feed to tools like gdb, crash, and file to understand what happened. The relevant code path in readelf.c is do_core_note(), which extracts the process name, PID, and signal from the note sections of any ELF core file it is given.

2. NT_PRPSINFO and the FreeBSD layout

ELF core files store their metadata in a PT_NOTE segment, which is a sequence of structured notes. Each note has a name (e.g., "CORE" on Linux, "FreeBSD" on FreeBSD), a type code, a size, and a variable-length descriptor. The type code determines how the descriptor should be interpreted. NT_PRPSINFO (type 3) carries a prpsinfo_t struct — process info recorded at crash time: the command name, the full command line, the PID, the parent PID, the UID/GID, and a few status flags. This is the note that file(1) reads to produce output like from 'crash_demo' when it identifies a core file.

The in-memory layout of prpsinfo_t differs between OS families — FreeBSD's struct has different field sizes and a different ordering than Linux's. In the FreeBSD variant, the process name (a null-terminated string, up to 80 characters) sits at a fixed offset from the start of the note descriptor, with the offset depending on whether the core is 32-bit or 64-bit:

if (clazz == ELFCLASS32)
    argoff = 4 + 4 + 17;       /* = 25 */
else
    argoff = 4 + 4 + 8 + 17;   /* = 33 */

The code then reads the process name directly:

if (elf_printf(ms, ", from '%.80s'", nbuf + doff + argoff) == -1)
    return -1;

3. The missing check

The read at nbuf + doff + argoff happens before any check that doff + argoff + 81 (the maximum number of bytes the %.80s format could read, plus the null terminator scan) is within the bounds of the note's data buffer. The note descriptor is attacker-controlled: a crafted ELF core file can set descsz to a small value while still providing a valid-looking NT_PRPSINFO note type, making doff + argoff point past the end of the allocated buffer.

There is a size check one line later for the PID field:

pidoff = argoff + 81 + 2;
if (doff + pidoff + 4 <= size) {
    /* read PID */
}

So the PID access is guarded, but the process name access immediately before it is not.

4. Triggering it

A minimal ELF core file with a PT_NOTE segment containing a FreeBSD-named NT_PRPSINFO note and a descsz smaller than argoff + 81 is enough. Setting descsz = 1 (a one-byte descriptor) while declaring the note type as NT_PRPSINFO and the OS style as FreeBSD sends do_core_note() straight into the unguarded read. The script below constructs the minimal triggering file:

import struct

def elf_header(e_phoff, e_phnum):
    # ELF64 header: little-endian, FreeBSD OSABI (0x09)
    ident = b'\x7fELF' + b'\x02' + b'\x01' + b'\x01' + b'\x09' + b'\x00' * 8
    return (ident
        + struct.pack('
python3 make_trigger.py
file crafted.core   # triggers the OOB read in do_core_note()

When file runs, do_core_note() enters the OS_STYLE_FREEBSD branch, computes argoff = 33 (64-bit path), and calls elf_printf(ms, ", from '%.80s'", nbuf + doff + 33) on a buffer that is only 1 byte long. Depending on what follows in memory, this reads whatever happens to be there — adjacent heap data or mapped pages — until a null byte is found. The output varies by platform and run; under ASAN the process immediately aborts with a heap-buffer-overflow report.

5. Fix — commit 6bb1b445

Christos Zoulas committed the fix in commit 6bb1b445 with the message "Add missing bounds check (Alexandru Hossu)". The change adds the missing guard before the process name read:

+if (doff + argoff + 81 <= size) {
     if (elf_printf(ms, ", from '%.80s'", nbuf + doff + argoff) == -1)
         return -1;
+}
The fix mirrors the pattern already used for the PID field a few lines below — the same kind of bounds check was already present for the adjacent access, just not for this one.