Nozomi Networks Discovers Unpatched DNS Bug in Popular C Standard Library Putting IoT at Risk

Nozomi Networks Discovers Unpatched DNS Bug in Popular C Standard Library Putting IoT at Risk

This article was updated on June 6, 2022 to note that uClibC-ng version 1.0.41 is fixed as of May 20, 2022. The fixed library can be downloaded here.

Nozomi Networks Labs discovered a vulnerability (tracked under CVE-2022-30295, ICS-VU-638779, VU#473698) affecting the Domain Name System (DNS) implementation of all versions of uClibc and uClibc-ng, a popular C standard library in IoT products. The flaw is caused by the predictability of transaction IDs included in the DNS requests generated by the library, which may allow attackers to perform DNS poisoning attacks against the target device.

According to their respective official websites, uClibc is known to be/have been used by major vendors such as Linksys, Netgear, and Axis (only until 2010, newer Axis products include other libraries), as well as Linux distributions such as Embedded Gentoo. uClibc-ng is a fork specifically designed for OpenWRT, a common OS for routers possibly deployed throughout various critical infrastructure sectors.

Because the library maintainer was unable to develop a fix, this vulnerability remains unpatched. For this reason, we are not disclosing the details of the devices on which we were able to reproduce the vulnerability.

In this blog, we start with a background of C Standard Libraries used in IoT devices, present a technical analysis of the issue and detail its exploitability, and then provide a comprehensive timeline of the disclosure process.

Background: C Standard Libraries and uClibc

In computing, a library is a collection of resources (functions, configuration data, specifications, etc.) that is shared among software. The purpose of libraries is to provide standard primitives that software can use to perform common operations (such as to access a file on a disk or perform a network operation), without the necessity of reimplementing them all the time.

A C standard library is no exception, in the sense that it is a library for the C programming language itself. A real-world example can be found in the well-known C “Hello, World!” program. When we type the pre-processor command “#include <stdio.h>” and invoke an output function (for instance, “printf”) to print “Hello, World!”, we are actually executing the code of the chosen output function provided by the selected C standard library for the implementation of the header file “stdio.h”.

Of note, uClibc is one of the possible C standard libraries available, and specifically focuses on embedded systems. Other famous C standard libraries are glibc (GNU C library), the musl libc, or the Microsoft C run-time library (CRT).

It’s important to note that a vulnerability affecting a C standard library can be a bit complex. Not only would there be hundreds or thousands of calls to the vulnerable function in multiple points of a single program, but the vulnerability would affect an indefinite number of other programs from multiple vendors configured to use that library.

Domain Name System (DNS)

As we know, in modern computer networks the DNS is a hierarchical database that serves the primary, and crucial, purpose of translating a domain name into its related IP address, allowing users to access resources by more easily by memorizing human-friendly names instead of complex sequences of numbers.

Under the hood, DNS requests are mostly transmitted through UDP, which is a connectionless protocol. Thus, in order to distinguish the responses of different DNS requests, besides the usual 5-tuple {source IP, source port, destination IP, destination port, protocol} and of course the query, each DNS request includes a parameter called “transaction ID”. This is a unique number per request that is generated by the client, added in each request sent, and that must be included in a DNS response to be accepted by the client as the valid one for a corresponding request. If this condition is false, the response is discarded; if true (and other checks are satisfied as well), the response is accepted and the program that asked for a translation of a certain domain name can initiate the connection to the IP address provided.

Example DNS equest
Figure 1. Example DNS request. The 5-tuple, txID, and query are highlighted in the red.

Similar to other C standard libraries, uClibc provides an extensive DNS client interface that allows programs to readily perform lookups and other DNS-related requests.

Poisoning Attacks to DNS

Because of its relevance, DNS can be a valuable target for attackers. In a DNS poisoning attack, an attacker is able to deceive a DNS client into accepting a forged response, thus inducing a certain program into performing network communications with an arbitrarily defined endpoint, and not the legitimate one. A DNS poisoning attack enables a subsequent Man-in-the-Middle attacks  because the attacker, by poisoning DNS records, is capable of rerouting network communications to a server under their control. The attacker could then steal and/or manipulate information transmitted by users, and perform other attacks against those devices to completely compromise them. The main issue here is how DNS poisoning attacks can force an authenticated response.

To have a DNS response accepted for a certain DNS request, the aforementioned 5-tuple, the query, and the transaction ID must be correctly set. Given that 1. the protocol is DNS, 2. the destination port is 53/udp, 3. the query is the target that an attacker wants to compromise, 4. the source IP address is the target machine, and 5. the destination IP address is public information (it is the IP address of the DNS server in use in a certain network), the only unknowns remain the source port and the transaction ID. It is vital that these two parameters are as unpredictable as possible, because if they are not, a poisoning attack could be possible.

Vulnerability Details

While we were reviewing the trace of DNS requests performed by an IoT device under test in our lab, we noticed the pattern of DNS requests performed from the output of Wireshark. If you look at the below screenshot (Figure 2) you can see that the transaction ID is first incremental, then resets to the value 0x2, then is incremental again.

Trace of DNS requests
Figure 2. Trace of DNS requests performed by an IoT device under test

While debugging the related executable, trying to understand the root cause, we eventually noticed that the code responsible for performing the DNS requests was not part of the instructions of the executable itself, but was part of the C standard library in use, namely uClibc 0.9.33.2 (“libuClibc-0.9.33.2.so”).

A source code review revealed that the uClibc library implements DNS requests by calling the internal “__dns_lookup” function, located in the source file “/libc/inet/resolv.c”.

DNS lookup function
Figure 3. DNS lookup function of uClibc.
In reference to line #1240, the function declares a static variable “last_id”. This variable contains the value of the transaction ID used in the last DNS request, and is initialized with the value 1 at the first invocation of “__dns_lookup”.
In reference to line #1240, the function declares a static variable “last_id”. This variable contains the value of the transaction ID used in the last DNS request, and is initialized with the value 1 at the first invocation of “__dns_lookup”.

Additionally, at line #1260, the function declares a variable “local_id”. This variable contains the value of the transaction ID that will be used in the upcoming DNS request, and is left uninitialized.

Afterwards, the following lines of code are executed.

DNS lookup function of uClibc
Figure 4. DNS lookup function of uClibc (2).

In reference to line #1309, at the first DNS request, the “local_id” variable is initialized with the value of the transaction ID of the last DNS request (“last_id”). Line #1320 is the actual core of the vulnerability: “local_id” is updated by incrementing its old value by 1. Given that the value is initialized to 1 at the first invocation, this is the reason that the transaction ID was reset to 0x2 the previous Wireshark screenshot (Figure 2).

After doing a bitwise AND at line #1321, this value is also stored inside “last_id” variable at line #1323. Finally, at line #1335, the value of “local_id” is copied inside the struct resolv_header “h” variable, which represents the actual content of the header of the DNS request. Similar revisions of this code were found in all versions available of uClibc and uClibc-ng.

Vulnerability Exploitability

Given that the transaction ID is now predictable, to exploit the vulnerability an attacker would need to craft a DNS response that contains the correct source port, as well as win the race against the legitimate DNS response incoming from the DNS server. Exploitability of the issue depends exactly on these factors. As the function does not apply any explicit source port randomization, it is likely that the issue can easily be exploited in a reliable way if the operating system is configured to use a fixed or predictable source port.

If the operating system applies randomization of source port (which is done by all OSs nowadays—for instance, modern Linux kernels use range 32768–60999), exploitability depends on the capacity of the attacker to bruteforce the 16 bit source port value by sending multiple DNS responses, while simultaneously winning the race against the legitimate DNS response. In this situation, exploitability depends on factors such as the bandwidth at disposal of the attacker, or on the response time of the DNS query. Additionally, it is more likely that an attacker will succeed at least once in performing a DNS poisoning attack if the target device performs a great number of identical queries in a given time frame, compared to a target that performs sporadic queries.

Mitigations

Note: Please see updates at the end of this blog.

This vulnerability remains unpatched, however we are working with the maintainer of the library and the broader community in support of finding a solution.

Because this vulnerability remains unpatched, for the safety of the community we cannot disclose the specific devices we tested on. We can, however, disclose that they were a range of well-known IoT devices running the latest firmware versions with a high chance of them being deployed throughout all critical infrastructure.

We recommend everyone increase their network visibility and security in both IT and OT environments.

Disclosure Process and Timeline

The timeline of events is reported below:

  • 2021/09/13: Disclosure to ICS-CERT
  • 2021/10/05: Asked for updates from ICS-CERT
  • 2021/10/05: Reservation of ticket numbers ICS-VU-638779, VU#473698
  • 2021/10/26: Asked for updates from ICS-CERT
  • 2021/11/19: Second attempt to ask for updates from ICS-CERT
  • 2021/11/29: Received invitation to participate in CERT/CC VINCE case
  • 2021/12/08: Request from CERT/CC to validate vulnerability on latest 1.0.39 version, and to provide insights regarding status of disclosure
  • 2021/12/09: Replied to CERT/CC, vulnerability confirmed on 1.0.39, informed CERT/CC that for now disclosure has been submitted to ICS-CERT only
  • 2021/12/09 (and later): Attempts by CERT/CC to invite uClibc-ng maintainer to the VINCE case
  • 2022/01/24: Multiple vendors invited to the VINCE case by CERT/CC, vulnerability disclosed to them
  • 2022/02/24: Asked for updates to CERT/CC
  • 2022/03/11: Attempt to contact uClibc-ng maintainer
  • 2022/03/14: Maintainer confirmed he was contacted by CERT/CC, explained he was unable to fix the vulnerability by himself, explicitly asked to publicly disclose it, hoping for help from the community
  • 2022/03/15-16: Discussion with CERT/CC on next steps. Planned to request a second confirmation from the maintainer, then proceed to notification of vendors of a disclosure in 30 days
  • 2022/03/18: Asked for second confirmation from maintainer
  • 2022/03/21: CERT/CC provided a status update in the VINCE case to vendors
  • 2022/03/21-22: Discussion with CERT/CC on next steps
  • 2022/03/23: Resent email to maintainer
  • 2022/03/29: Proposed to CERT/CC to proceed with the 30-day notification to vendors, given lack of responses (or counter-orders) from maintainer
  • 2022/04/01: CERT/CC approved the disclosure schedule proposal
  • 2022/04/01: Provided a status update in the VINCE case to vendors, informed them that the vulnerability would be disclosed on May 2nd
  • 2022/05/02: Public disclosure
  • 2022/05/20: Maintainer released a fix for this vulnerability
  • 2022/06/03: Nozomi Networks notified CERT/CC and vendors of fix
  • TBD: Vendors to implement fix

Mitigations Update

UPDATE: As of May 20, 2022 uClibC-ng version 1.0.41 is fixed. The fixed library can be downloaded here. Although the maintainer of the library has released this fix, asset owners need to wait for vendors to patch firmware. Here are some additional DNS security recommendations:

  • Deploy another device in the trusted local network that serves as a DNS proxy. Configure the vulnerable device to send DNS queries to the DNS proxy, and make sure that the DNS proxy only is allowed to send DNS queries to external networks (i.e. any network not controlled by asset owners)
  • Enable the Domain Name System Security Extensions (DNSSEC) on the device
  • Configure the device to send DNS queries to a trusted DNS server through an end-to-end encrypted communication (e.g., by using DNS over TLS, DNS over HTTPS, VPN, etc.)