rtl8723bs — WiFi Heap Overflow in the Linux Kernel

1. Background

The rtl8723bs driver handles the Realtek RTL8723BS SDIO WiFi/Bluetooth combo chip, which ships in a wide range of embedded Linux devices — tablets, single-board computers, industrial controllers. It was added to the kernel's staging tree in 2017 (commit 554c0a3abf21) and has been there ever since.

I found the primary bug — a heap overflow in OnAuthClient() — while reading through the driver's 802.11 management frame handling code. From there, a reviewer pointed out that the same function was also missing basic frame length guards, which led to a second patch. The same IE parsing pattern repeated across the driver, so I kept going. Three patch series in total. The first two cover the OnAuthClient() overflow, the client-side and AP-side frame length checks, and the IE handler OOBs in the association path. The third adds fixes for five more functions: update_beacon_info(), bwmode_update_check(), issue_assocreq(), join_cmd_hdl(), rtw_get_wps_ie(), and a separate one-byte heap overflow in rtw_cfg80211_set_wpa_ie() reachable via nl80211.

2. OnAuthClient heap overflow

The function OnAuthClient() in core/rtw_mlme_ext.c processes 802.11 Authentication frames received by the station. When the AP sends an Auth sequence 2 frame using Shared Key authentication, the driver looks for a Challenge Text IE (element ID 0x10) and copies its payload into a fixed 128-byte buffer:

/* rtw_mlme_ext.c ~line 891 */
p = rtw_get_ie(pframe + WLAN_HDR_A3_LEN + _AUTH_IE_OFFSET_,
               WLAN_EID_CHALLENGE, (int *)&len,
               pkt_len - WLAN_HDR_A3_LEN - _AUTH_IE_OFFSET_);

if (!p)
    goto authclnt_fail;

memcpy(pmlmeinfo->chg_txt, p + 2, len);  /* len = raw IE length byte, 0-255 */

rtw_get_ie() validates that the IE fits within the received frame buffer — so the copy source is legitimate. What it does not check is whether len fits in the 128-byte destination chg_txt[]. The raw IE length field is a single byte and can be anywhere from 0 to 255. If an AP sends a Challenge Text IE with length 200, the driver writes 72 bytes past the end of chg_txt.

IEEE 802.11-2020 §12.3.3 is unambiguous: the Challenge Text element carries exactly 128 bytes of challenge data. The fix is a one-line addition — reject the frame if the IE length is anything other than sizeof(pmlmeinfo->chg_txt):

-if (!p)
+if (!p || len != sizeof(pmlmeinfo->chg_txt))

The overflow target is struct mlme_ext_info, with chg_txt at offset 40. Immediately following it are aid, bcn_interval, capability, and a handful of protocol state flags. With a crafted length of 200, 72 bytes past the end of chg_txt are overwritten.

3. Auth algorithm toggle — any device is affected

The overflow is in the Shared Key path, which at first looks like it only affects devices explicitly configured for WEP Shared Key authentication. It does not. There's a toggle in the same function:

if (status == WLAN_STATUS_NOT_SUPPORTED_AUTH_ALG) {
    pmlmeinfo->auth_algo ^= 1;   /* toggle Open ↔ Shared */
    pmlmeinfo->reauth_count = 0;
    set_link_timer(pmlmeext, 1);
    return _SUCCESS;
}

The attack sequence is two frames. First, send an Auth Response (seq=2, status=13 — WLAN_STATUS_NOT_SUPPORTED_AUTH_ALG). The driver toggles from Open System to Shared Key and sends a new Auth Request. Then immediately send a second Auth Response (seq=2, status=0) with a crafted Challenge Text IE. The driver is now in the Shared Key path and executes the overflow unconditionally.

This works against any device running this driver that is attempting to connect to a network, regardless of how the device is configured. The attacker just needs to be within radio range.

4. Missing frame length checks — OnAuthClient

After I submitted the first patch, Dan Carpenter pointed out that OnAuthClient() was also missing basic frame length validation. Two problems:

First, get_da(pframe) inspects the ToDs/FrDs bits in Frame Control (bytes 0–1) and returns either Addr1 (bytes 4–9) or Addr3 (bytes 16–21). GetPrivacy(pframe) also reads the Frame Control field. Both are called before any check that pkt_len is at least WLAN_HDR_A3_LEN (24 bytes), so a short frame causes an out-of-bounds read.

Second, the seq and status fields are read at pframe + WLAN_HDR_A3_LEN + offset + 2 and +4 with no check that those offsets are within the frame. The fix adds two guards: one before the first pframe access, and a second after computing offset (0 for open-system, 4 for WEP) to cover the seq/status reads:

+if (pkt_len < WLAN_HDR_A3_LEN)
+    goto authclnt_fail;

 /* check A1 matches or not */
 if (memcmp(myid(&(padapter->eeprompriv)), get_da(pframe), ETH_ALEN))
     return _SUCCESS;
 ...
 offset = (GetPrivacy(pframe)) ? 4 : 0;

+if (pkt_len < WLAN_HDR_A3_LEN + offset + 6)
+    goto authclnt_fail;

5. Missing frame length checks — OnAuth (AP mode)

The same class of missing frame length guards turned up on the AP side. OnAuth() handles 802.11 Authentication frames received by the driver when it is operating as a soft-AP. Three OOB reads:

The first operation after the AP-state guard is GetAddr2Ptr(pframe), which reads 6 bytes at offset 10–15 (the SA field). This happens with no prior check that the frame is at least WLAN_HDR_A3_LEN (24) bytes long. A station could send a truncated auth frame and cause an OOB read before the driver has done anything else with the packet.

When the Privacy bit is set in Frame Control, the code sets iv = pframe + WLAN_HDR_A3_LEN and immediately reads iv[3] to extract the key index. There is no check that the frame is long enough to contain four bytes after the header. A frame that is between 24 and 27 bytes long passes the first guard but triggers an OOB read here.

After computing offset (0 for open-system, 4 for WEP), the code reads algorithm at pframe + WLAN_HDR_A3_LEN + offset and seq at +offset + 2. No check confirms that those reads are within the frame.

The fixes use return _FAIL for the first two guards rather than goto auth_fail, because the auth_fail block calls memcpy(pstat->hwaddr, sa, ETH_ALEN) to build a rejection frame — but sa has not been set at those points. Jumping there would copy stack garbage into the destination address. The third guard, after sa is valid, sets a defined status code and uses goto auth_fail:

+if (len < WLAN_HDR_A3_LEN)
+    return _FAIL;

 sa = GetAddr2Ptr(pframe);
 ...
+if (len < WLAN_HDR_A3_LEN + 4)     /* inside GetPrivacy() branch */
+    return _FAIL;

 iv = pframe + prxattrib->hdrlen;
 prxattrib->key_index = ((iv[3] >> 6) & 0x3);
 ...
+if (len < WLAN_HDR_A3_LEN + offset + 4) {
+    status = WLAN_STATUS_UNSPECIFIED_FAILURE;
+    goto auth_fail;
+}

 algorithm = le16_to_cpu(*(__le16 *)((SIZE_PTR)pframe + WLAN_HDR_A3_LEN + offset));
 seq       = le16_to_cpu(*(__le16 *)((SIZE_PTR)pframe + WLAN_HDR_A3_LEN + offset + 2));

6. HT_caps_handler OOB write

While in the same file, I found a similar issue in HT_caps_handler() in core/rtw_wlan_util.c. This function processes the HT Capabilities IE from an Association Response. It iterates pIE->length bytes and writes them into HT_caps.u.HT_cap[], which is a fixed 26-byte array:

for (i = 0; i < (pIE->length); i++) {
    HT_caps.u.HT_cap[i] = pIE->data[i];
    ...
}

Since pIE->length is a raw byte from the received frame (0–255), a malicious AP can cause up to 229 bytes of out-of-bounds writes into adjacent fields of struct mlme_ext_info. The fix truncates the iteration count with umin() instead of rejecting the IE outright — APs that send an oversized HT Caps element are handled gracefully rather than causing a connection failure:

-for (i = 0; i < (pIE->length); i++) {
+for (i = 0; i < umin(pIE->length,
+                      sizeof(pmlmeinfo->HT_caps.u.HT_cap)); i++) {

7. OnAssocRsp IE loop OOB read

The IE parsing loop in OnAssocRsp() iterates through all information elements in the Association Response frame. The loop condition is i < pkt_len, and at the top of each iteration it casts pframe + i to struct ndis_80211_var_ie * — which has a two-byte header (element ID + length). If the last IE in the frame starts at pkt_len - 1, reading pIE->length touches pframe[pkt_len], one byte past the allocation. Even when the header bytes are in bounds, the IE's declared pIE->length can extend past pkt_len, passing a truncated IE to the handler functions.

The fix adds two guards at the top of the loop body:

 for (i = (6 + WLAN_HDR_A3_LEN); i < pkt_len;) {
+    if (i + sizeof(*pIE) > pkt_len)
+        break;
     pIE = (struct ndis_80211_var_ie *)(pframe + i);
+    if (i + sizeof(*pIE) + pIE->length > pkt_len)
+        break;
     switch (pIE->element_id) {

8. More IE loop OOB reads — beacon, join, and WPS paths

The same missing two-guard pattern turned up in several more places across the driver. The subsequent review rounds also surfaced additional bugs in the individual IE handlers.

update_beacon_info() in core/rtw_wlan_util.c processes incoming Beacon frames. Three problems here. First, the len computation — pkt_len - (_BEACON_IE_OFFSET_ + WLAN_HDR_A3_LEN) — uses an unsigned result with no prior check that pkt_len is large enough. A frame shorter than 36 bytes wraps len to a very large value and sends the loop over memory far past the receive buffer. Second, the IE loop had the same missing two-guard problem as the others. Third, the WMM OUI comparison called memcmp(pIE->data, WMM_PARA_OUI, 6) before checking pIE->length == WLAN_WMM_LEN — an IE with fewer than 6 bytes of payload caused the memcmp to read into adjacent frame data. The conditions are now swapped so the length check comes first.

bwmode_update_check(), called from the same loop, rejects IEs that are longer than sizeof(struct HT_info_element) but accepts any shorter length including zero. After the check it casts pIE->data to struct HT_info_element * and reads infos[0] at offset 1. A zero-length IE passes the guard and causes an OOB read. The guard was changed from > to != to require an exact match.

issue_assocreq() and join_cmd_hdl() in core/rtw_mlme_ext.c walk stored network IE data filled earlier from the AP's Beacon and Probe Response frames. The loop bounds problems were the same as the others, but there were also several bugs inside the individual handlers. The vendor-specific OUI comparisons in issue_assocreq() called memcmp on a 4-byte OUI without checking that pIE->length is at least 4. In the WPS truncation path, when wifi_spec is 0, the code sets vs_ie_length = 14 and passes pIE->data to rtw_set_ie() regardless of the actual IE length — an IE between 4 and 13 bytes triggers an OOB read of up to 10 bytes. The fix skips the IE when pIE->length < 14. The HT Capability IE handler copied sizeof(struct HT_caps_element) bytes from the IE payload without checking the IE was that long, and then passed the raw pIE->length to rtw_set_ie() — if the IE was longer than the struct, that caused an OOB read from pmlmeinfo->HT_caps. In join_cmd_hdl(), the WMM check used pIE->length >= 4 before calling WMM_param_handler(), but the handler reads pIE->data + 6 and copies sizeof(struct WMM_para_element) (18 bytes) for a total of 24 bytes — the guard was strengthened to pIE->length >= WLAN_WMM_LEN. The HT Operation IE handler cast pIE->data to struct HT_info_element * and read infos[0] at offset 1 with no minimum length check.

rtw_get_wps_ie() in core/rtw_ieee80211.c searches IE data for a WPS IE. The loop only checked cnt < in_len before reading in_ie[cnt + 1] — if the buffer ends on an element_id byte with no length byte following, this reads one byte past the end. There was also no check that the declared payload fits within in_len, and the 4-byte WPS OUI comparison ran without confirming the IE payload was at least 4 bytes long.

9. rtw_cfg80211_set_wpa_ie() — WPA IE one-byte overflow (local)

This one is different from the others: it is a local attack, not over-the-air. rtw_cfg80211_set_wpa_ie() in os_dep/ioctl_cfg80211.c handles WPA and WPA2 IE data supplied via NL80211_CMD_CONNECT. It copies the IE into supplicant_ie, a 256-byte array in struct security_priv:

memcpy(padapter->securitypriv.supplicant_ie, &pwpa[0], wpa_ielen + 2);

wpa_ielen is the raw IE length field — a u8, so it ranges from 0 to 255. When a local user issues a connect with a crafted WPA IE of length 255, wpa_ielen + 2 = 257 and the memcpy writes one byte past the end of the 256-byte buffer into the adjacent last_mic_err_time field.

The existing length check in rtw_parse_wpa_ie() does not prevent this. It compares *(wpa_ie+1) against (u8)(wpa_ie_len - 2). With wpa_ie_len = 257, the cast produces (u8)(255) = 255, which matches — so the check passes silently.

The fix adds an explicit bounds check before both the WPA and WPA2 copy paths:

+if (wpa_ielen + 2 > sizeof(padapter->securitypriv.supplicant_ie)) {
+    ret = -EINVAL;
+    goto exit;
+}

10. Patches

Three series went to the staging mailing list. Series 1 (v8) covered the OnAuthClient() heap overflow, the client-side frame length checks, and the AP-side frame length guards in OnAuth() — by v8, the earlier patches had already landed in Greg's tree and only the OnAuth() fix remained. Series 2 (v5) fixed the HT_caps_handler() OOB write, the OnAssocRsp() IE loop OOB read, a truncated frame guard before the fixed-field reads, and the WMM length guard. Series 3 (v5) fixed the remaining IE parsing issues in update_beacon_info(), bwmode_update_check(), issue_assocreq(), join_cmd_hdl(), and rtw_get_wps_ie(), plus the one-byte WPA IE overflow in rtw_cfg80211_set_wpa_ie(), with Reviewed-by from Luka Gejak.

All changes are in drivers/staging/rtl8723bs/:

• core/rtw_mlme_ext.c — OnAuthClient(): Challenge Text length check, frame length guards
• core/rtw_mlme_ext.c — OnAuth(): three frame length guards (header, IV, algorithm/seq)
• core/rtw_wlan_util.c — HT_caps_handler(): OOB write fix, LDPC/STBC macro OOB read fix
• core/rtw_mlme_ext.c — OnAssocRsp(): IE loop bounds checks, truncated frame guard, WMM length guard
• core/rtw_wlan_util.c — update_beacon_info(): pkt_len underflow guard, IE loop bounds checks, WMM condition ordering
• core/rtw_wlan_util.c — bwmode_update_check(): minimum IE length guard
• core/rtw_mlme_ext.c — issue_assocreq(): OUI length guard, WPS truncation guard, HT caps length check
• core/rtw_mlme_ext.c — join_cmd_hdl(): WMM length guard, HT info length check
• core/rtw_ieee80211.c — rtw_get_wps_ie(): header bounds, payload bounds, OUI length guard
• os_dep/ioctl_cfg80211.c — rtw_cfg80211_set_wpa_ie(): WPA/WPA2 IE size check