The level of confidence that software functions as intended and is free of vulnerabilities, either intentionally or unintentionally designed or inserted as part of the software throughout the lifecycle.[1]
We concluded that the 3 main challenges to software assurance are incorporated code, code volume, and tool effectiveness. But what about a process that meets those challenges and puts the security in “DevSecOps”?
This article shares about the effective SwA tools we recommend, our tiered approach to software analysis, and the importance of constant research conducted by Subject Matter Experts (SMEs).
Effective Software Analysis Tools
A robust SwA capability must be built upon an infrastructure that provides the ability to analyze both source code and binary code. It must analyze it statically and dynamically, using multiple automated analysis tools and manual review processes.
Software analysis tools fall into four major categories. The use of multiple tools in each of these categories is critical to ensure identification of exploitable software vulnerabilities to the maximum extent possible.
4 Major Categories of Software Analysis Tools
Static Source Code Analysis Tools
Used when source code is available.
Mostly requires buildable source code, but a few will analyze non-buildable code.
Identifies software weaknesses in source code, representing potential exploitable vulnerabilities.
Language specific, and each tool has strengths and weaknesses in identifying specific types of weaknesses/vulnerabilities.
Static Binary Analysis Tools
Analyzes software in its binary executable form (e.g., Windows, or Linux/Unix compiled executables) without executing the application.
Used when source code is NOT available.
Identifies risks associated with the use of third party and commercial software libraries and applications.
Typically disassembles the binary to detect flaws and vulnerabilities.
Tools that fall into this category are less mature and are available through a limited number of commercial vendors and open-source projects.
Dynamic Source Code Analysis Tools
Used to analyze code written in interpreted software languages such as Perl, JavaScript, Python, PHP, PowerShell, Ruby, and MATLAB.
Analyzed dynamically while being executed through an interpreter to detect software vulnerabilities and malicious behavior.
Tools that fall into this category are available through a limited number of commercial vendors.
Dynamic Binary Analysis Tools
Used to identify software vulnerabilities and malicious behavior in applications during and after execution.
Platform and architecture specific.
Typically instruments the operational environment to allow monitoring of system changes and network activity during program execution.
Designed to detect malicious behavior while the application is executing. For this reason, the execution environment must be recreated or virtualized.
Tools that fall into this category are less mature and are available through a limited number of commercial vendors.
Software Analysis Process
Most modern software analysis tools produce a “mountain” of information that must be reviewed, vetted, and categorized as a part of risk categorization and mitigation activities. This process can be very time consuming and requires an experienced SwA analyst to conduct these activities.
SwA tools can be configured and tuned to be less sensitive, thus reducing the number of issues identified by a tool; however, this introduces greater risk that “true” issues will go undetected. Optimally, a robust SwA implementation will maximize the sensitivity of SwA tools while using automated and streamlined review processes to efficiently vet and categorize identified issues.
The goal of a robust software analysis process is to maximize “precision” (the ability to correctly distinguish “true positives”) and “recall” (the ability to identify all possible “true positives”). It is recognized, however, that this goal is often constrained by limitations in funding, time, or expertise.
To balance the goal of SwA with these competing factors, a tiered approach to implementing SwA can be used. This tiered approach takes into consideration that all software is NOT created equal.
Systems that are deemed to have higher CIA requirements may require higher levels of SwA rigor. The following tiered approach to implementing SwA into the DevSecOps pipeline takes this into consideration.
Our Robust, Scalable, & Tiered Approach to Software Assurance
COLSA’s approach to implementing a robust and scalable SwA solution is to integrate a tiered SwA capability into DevSecOps environments based upon the assurance requirements of the software being developed. These tiers are defined as follows:
Four Tiers of Assurance
1. Low Assurance Tier – Unvetted, Automated Scans
SwA analysis is conducted using highly automated scanning tools.
Scans can be configured to automatically initiate upon code commits.
Processes are focused largely on static analysis.
Unvetted tool results are automatically “pushed” back to the pipeline issue tracking system.
Analysis is typically completed relatively quickly (e.g., hours).
2. Medium Assurance Tier – Evaluated Scans
SwA analysis is conducted using highly automated scanning tools.
Scans can be configured to automatically initiate upon code commits.
Processes are focused largely on static analysis.
Tool results are vetted by a SwA analyst to eliminate “false positives” and to assign risk scores using industry-standard risk scoring processes.
Vetted tool results are “pushed” back to the pipeline issue tracking system.
Analysis is typically completed in a moderate amount of time (e.g., days).
3. High Assurance Tier – Dynamic Analysis, Fuzzing, Pen-testing
SwA analysis is conducted using semi-automated software analysis tools and processes.
Analysis processes are initiated upon code commits.
Focus is on dynamic analysis in an instrumented operationally simulated environment.
Results are vetted by a SwA analyst to validate findings and to assign risk scores using industry-standard risk scoring processes.
Vetted results are automatically “pushed” back to the pipeline issue tracking system.
Analysis is typically completed in a moderate amount of time (e.g., days).
4. Malware Tier – Reverse Engineering and Forensic Analysis
SwA analysis is conducted using manual reverse engineering and forensic analysis tools and processes.
Analysis processes are initiated upon code commits.
Analysis is conducted by a SwA analyst to identify security vulnerabilities and malicious content in critical software components.
Analysis results can be automatically “pushed” back to the pipeline issue tracking system but are typically provided to the requester in the form of a detailed report and briefing.
Analysis typically takes much longer to complete (e.g., weeks).
The Importance of SwA Research and Subject Matter Expertise
To maintain a successful SwA practice, research must be conducted to maintain visibility into the latest software exploitation techniques used by advanced adversaries. SwA tools and processes are constantly evolving to keep pace with the ever-changing SwA threat.
COLSA has a team of SwA subject matter experts (SMEs) that not only possess demonstrated expertise using state-of-the-art SwA tools and processes, but who are constantly conducting research and developing tools to identify and mitigate emerging SwA threats. This expertise can be used to assist organizations desiring to stand up an initial SwA capability, as well as those organizations that require SwA research expertise to counter evolving SwA threats from advanced adversaries.
Our SwA SMEs provide in depth SwA training to analysts and developers to ensure a thorough understanding of the underlying software issues that result in exploitable vulnerabilities, and the tools and techniques used to identify and mitigate them. Our SwA SMEs are available to support the establishment of a SwA capability by providing critical training and SwA expertise as required.
Put the “Sec” in your DevSecOps
Equip your analysts and developers with the knowledge and tools necessary for putting the “Sec” in your DevSecOps. When you work with the COLSA SwA team, you receive in-depth training and develop a thorough understanding of the underlying software issues that result in exploitable vulnerabilities. You also learn which tools and techniques are best suited for identifying and mitigating those vulnerabilities, all while having direct access to our Software Assurance experts.
Would you like a demonstration? Contact us:
L. Todd Wilson Chief Information Security Officer twilson@colsa.com 240-409-3540
Jonathan Hood, PhD Technical Fellow jhood@colsa.com 256-797-1580
[1] DoDI 5200.44, “Protection of Mission Critical Functions to Achieve Trusted Systems and Networks (TSN)”, Change 2, July 27, 2017, Page 13
