Nozomi Networks Labs: Discovering and Reporting Vulnerabilities to Increase Security

Nozomi Networks Labs: Discovering and Reporting Vulnerabilities to Increase Security

As cybersecurity practitioners struggle to keep pace with continuous changes to the cyber threat landscape, threat actors continue to refine their Tactics, Techniques, and Procedures (TTPs) when carrying out cyberattacks. This is why the role of vulnerability research is fundamental in staying ahead of the curve by notifying vendors and asset owners of security vulnerabilities that threat actors could potentially exploit to launch an attack. This information can be used to identify new threats, provide protection to systems and applications, as well as improve existing security measures.

The vulnerability research to reporting pipeline is a framework for collecting and sharing information about vulnerabilities and cyber threats. The process involves identifying potential risks and weaknesses in software and systems, then creating solutions for those vulnerabilities. These vulnerabilities are assigned a Common Vulnerabilities and Exposures (CVE) number and (most times) a Common Vulnerability Scoring System (CVSS) score in a vulnerability database by a CVE Numbering Authority (CNA). CVE Numbering Authorities (CNA) are organizations that have been approved by the CVE Program to report on vulnerabilities. As a CNA authority, Nozomi Networks collaborates with vendors and the government to report and resolve vulnerabilities discovered by our security researchers, and to ensure sure those systems are safe from malicious attacks. In this blog, we will take you through the steps our to select, procure, investigate, discover, report, resolve, curate, and share vulnerabilities to help to increase security.

Who Are the Stakeholders?

Before getting into the pipeline, we must identify the stakeholders; this will help to understand who reports to whom during this process. There are five key stakeholders involved in the vulnerability reporting pipeline:

  • CVE Program: Collaboration of subject matter experts from public and private organizations who govern the reporting and recording of CVEs.
  • Security Researchers: Security Researchers procure and conduct investigative research on technology, in their industry of interest, to discover and report on vulnerabilities.
  • Vendors: Vendors develop technology solutions for the industry. Not only are vendors responsible for developing code to support new product features in their technology, they are also responsible for developing patches for discovered vulnerabilities by security researchers.
  • Government: The Government has a stake in vulnerabilities because it is a matter of national security! The Cybersecurity and Infrastructure Security Agency (CISA) is the sole government sponsor of the CVE Program; although it is U.S. sponsored, this is a global program and run by a collection of private organizations.
  • End User: The end user is responsible for implementing patches in a timely manner.  

The Vulnerability Reporting Pipeline  

In this section, we’ll cover the vulnerability reporting pipeline from a researcher’s perspective. Based on our process, the pipeline has a total of 8 steps, as seen in Figure 1: Select, procure, investigate, discover, report, resolve, curate, and share.

Vulnerability reporting pipeline
Figure 1. Vulnerability Reporting Pipeline

Select

Vulnerability Reporting Pipeline - Select
Figure 2. Vulnerability Reporting Pipeline – Select

Before vulnerability research can be conducted, the security researcher must select the device they wish to conduct research on. This decision can first be made based on industry alignment. For example, Nozomi Networks provides cybersecurity solutions for companies operating in the critical infrastructure space. This can include aerospace, oil and gas, manufacturing, building automation, electricity, water, and more. Companies within these industries use very similar Internet of Things (IoT) and Operational Technology (OT) devices, which we help provide monitoring and security for critical assets. All in all, focusing on a specific industry and popular technology used is one way to narrow down which devices to research.

Another point of consideration is emerging technology. Companies often want the newest security solution on the market. This could indicate to security researchers to stay ahead by researching technology before it becomes too popular. The benefits is that working with the vendor may be able to address key vulnerabilities before the technology scales and causes potential widespread impact. It is also important to look at devices that are known to have multiple vulnerabilities or are highly targeted by threat actors; threat intelligence insights could help with the narrowing down of this selection.

Procure

Vulnerability Reporting Pipeline – Procure
Figure 3. Vulnerability Reporting Pipeline - Procure

After the researchers select the device, they must then develop a strategy for procuring it. Procuring devices for vulnerability research can be a time-consuming and sometimes a difficult task. Because these devices can be pricey, they’re not so easy to obtain. There are three main ways a security researcher can procure a selected device:

  1. Vendor Relationship – Through various partner programs, the security researcher can obtain the IoT/OT device directly from the vendor,
  1. End User – The end user, or customer, may provide the researchers with the device, and
  1. Market – The researcher will turn to online marketplaces in hopes of purchasing the desired device.

Depending on the device, and the region it is being shipped from, it could take months for the device to be delivered.  

Investigate

Vulnerability Reporting Pipeline - Investigate
Figure 4. Vulnerability Reporting Pipeline – Investigate

Once the device is purchased and delivered to the research laboratory, the researchers can now begin to investigate different aspects such as firmware, software, language libraries, or the protocols being used. Researchers use the following techniques during this investigation stage:

  • Reconnaissance – As a first task, the researcher tries to gather as much information as possible from the target. “How many external services are exposed by the target?”, “Is the firmware obfuscated?”, “Are there any hardware debugging ports?”, or “How is each functionality intended to be used?” are just some of the questions that the assessor tries to answer, in order to have a clear idea of all attack surfaces and how to analyze them.
  • Reverse Engineering – Performing a reverse engineering activity is a systematic process that involves examining each functionality offered by the device to find potential vulnerabilities. There are two main types of reviews:
  1. Static analysis – this type of review is done by analyzing the code (disassembled, decompiled, or source when available) of the functionality, without executing it.
  1. Dynamic analysis – this type of review requires execution of the code of the functionality and it can only be done in specific situations (e.g., if the device allows any debugging access).

Both types of reviews are important for finding vulnerabilities in software, but dynamic reviews are more efficient because they require less time and effort from researchers while they provide a more complete picture of possible vulnerabilities in a given system or application.

  • Attacks Simulations – Attack simulations allow security researchers to test devices against plausible cyberattack scenarios. This is done by simulating an attack and then examining the results of the simulation to determine what happened, why it happened, and what could be done better in the future.  

There are several other techniques, not mentioned here, that security researchers apply during this investigation phase.

Discover

Vulnerability Reporting Pipeline - Discover
Figure 5. Vulnerability Reporting Pipeline – Discover

After extensive research is conducted on the device, the researcher then determines if any vulnerabilities discovered are exploitable. This is key because not all vulnerabilities are exploitable; here’s how you can tell the difference:

  • Exploitable vulnerability: The security flaw allows the researcher to access, manipulate, shut down the device, or to extract sensitive data.
  • Non-exploitable vulnerability: The security flaw does not allow the security researcher to take control of the device, but can still be impactful in the OT environment.

Sometimes researchers don’t have enough information from the vendor to determine if a vulnerability is exploitable or not, and may decide to submit it to the vendor for further analysis on exploitability.

Report

Vulnerability Reporting Pipeline – Report
Figure 6. Vulnerability Reporting Pipeline – Report

As a CNA of the CVE Program, Nozomi Networks must first report the discovered vulnerability to the vendor before reporting to ROOT, CNA-LR authorities or in other specialized public forums (e.g., CERT/CC VINCE). This allows time for the vendor to create a fix with the support of our security researchers, and hold off sharing the vulnerability until a patch is made available to end users.

Nozomi Networks has established a vulnerability disclosure policy which provides insights on how we protect users in a timely manner and how we ensure that the recommended fix is comprehensive and effective for all end users.

Vulnerability disclosure policy
Figure 7. Nozomi Networks Vulnerability Disclosure Policy

Our disclosure timeline consists of the following three phases:

  1. Vendor Notification: In the Vendor Notification phase, our researchers contact the vendor, providing them a technical analysis of the vulnerability, along with possible remediations. The vendor will then have 30 days to acknowledge receipt of the notification. In the event the researcher cannot get in contact with the vendor to report the vulnerability, we will take another avenue and notify a CNA Last Resort (CNA-LR) such as Critical Infrastructure Security Agency (CISA), to reach the vendor. In May of 2022, CISA was placed in charge of the management of CVEs for ICS and medical devices.
  2. Vendor Engagement: Once the vendor has been notified, we require that they produce, within 60 days, a technical explanation of how the issue shall be addressed and a timeline for public disclosure. Our researchers will remain available in case the vendor asks for any support. This is an iterative process where both Nozomi Networks researchers and the vendor try to validate and find a solution to the discovered vulnerability.
  3. Public Disclosure: During the public disclosure phase, the vendor publicly resolves the issue in 120 days from the initial reporting of the disclosure and it being published. This means that vendor has developed an action that users can apply to solve the issue, and that the CVE ID requested by the CNA was approved and published in the CVE Record.

This timeline is not strict. Nozomi Networks Labs is aware that some vulnerabilities can have profound ramifications on the affected systems. For this reason, and on a case-by-case basis, Nozomi Networks Labs is open to discuss a timeline extension if the vendor satisfactorily illustrates to Nozomi Networks Labs the technical rationale behind the request.

More information on our Public Disclosure Policy can be found on our website.

Resolve

Vulnerability Reporting Pipeline - Resolve
Figure 8. Vulnerability Reporting Pipeline – Resolve

In this step, the vendor begins the process of creating a patch for the discovered vulnerability. While each vendor has a different timeline based on available resources, our public disclosure policy states that a patch should be made available in a 1-4 month timeframe (timeline extensions can be granted on a case-by-case basis).

Once again, we stress that our researchers will not publicly share the discovered vulnerability until the vendor has completed the development of a remediation within the four-month timeframe. If a zero day is publicly shared prior to there being a fix, threat actors can immediately start targeting the vulnerability, thus leading to massive exploits with no remediation. We require a remediation in a reasonable timeframe because hiding a vulnerability would only help threat groups—who might have already discovered the vulnerability in independent research—compromise unaware asset owners.

Curate

Vulnerability Reporting Pipeline – Curate
Figure 9. Vulnerability Reporting Pipeline – Curate

As a CNA, Nozomi Networks has the authority to curate CVEs so that they can officially be accepted and used as reference in various CVE databases. When no other authorities responsible for the scope of the vulnerabilities found are identified and/or involved in the process, Nozomi Networks follows the steps to curate CVEs, as outlined by the CVE program. The process is as follows:

  1. Assign a CVE-ID: A CVE-ID is assigned to a disclosed vulnerability; this ID is unique to a specific vulnerability and is used in tracking/categorizing vulnerabilities in various databases.
  2. Put the CVE in the proper formats (e.g., flat file, JSON, or CSV). Regardless of format, the following fields must be filled out:
    CVEID:
    PRODUCT:
    VERSION:
    PROBLEMTYPE:
    REFERENCES:
    DESCRIPTION:
    ASSIGNINGCNA:
  3. Provide enough detail to make the vulnerability unique to the CVE-ID.

Additionally, here are some key points of consideration during this curation process:

  • When a vulnerability is revealed, there might not be a patch always readily available (e.g., if the software or device has reached the end of support).
  • The vulnerable version range might have a low limit but not an upper limit (e.g all versions are vulnerable or all version greater than X are vulnerable).
  • At some point (at a later date) the vulnerability might be covered by a new firmware version.

While the CVE program does not make Common Vulnerability Scoring System (CVSS) scores mandatory, other CVE databases encourage the use of this score, as it helps end users prioritize patching based on severity of the CVE. There are also additional scoring frameworks on the rise, so while CVSS is a start, it is not the only severity framework available.

Share

Vulnerability Reporting Pipeline - Share
Figure 10. Vulnerability Reporting Pipeline – Share

Once the vulnerability has been resolved it is time to share the remediation with the community.  Here are ways Nozomi Networks spreads the word to end users:

  • CVE Databases: We officially report the CVE and fix to ICS-CERT, NVD, the CVE Program, etc.
  • Blogs: For critical vulnerabilities, we publish a blog detailing the technical research behind the discovery along with the fix on our website.
  • Media: We work with various mainstream and cybersecurity news outlets to inform stakeholders of this vulnerability, in order to encourage immediate patch implementation.
  • Conferences: We share our discovered exploits at top conferences (Black Hat, Sa4, RSA, etc.) and for specialized audiences and organizations, to continue to spread the word to relevant stakeholders.

How We Protect Our Customers

The Security Researchers at Nozomi Networks ensure that our customers are protected by the latest vulnerabilities. Our threat intelligence package provides real-time detections against vulnerabilities discovered by our researchers, as well as critical vulnerabilities published in CVE databases. You can find out more information on our threat intelligence package here.

Summary

Security researchers play a crucial role in helping organizations identify new vulnerabilities and improve existing security measures. The goal is simple: find security vulnerabilities in software systems before the attackers exploit them. As cyber-threats continue to evolve, it is important for security practitioners to stay ahead of the curve. Researching and reporting security vulnerabilities is a difficult, time-consuming process that can take months or years of effort. Technology vendors should continue work with security researchers to create a strong partnership that benefits the industry.

If you want to learn more about our security research capabilities or want to collaborate on our vulnerability research efforts, please contact us.