Making sense of the SAST, DAST, IAST, RASP soup #1
Software security scanner market today is now mature enough to have its own jargon, built largely by vendor marketing teams. As such, it’s primarily designed to help you get rid of your infosec budget rather than help you design a reasonable security assurance architecture. This is first part of series that looks at SAST and DAST advantages and limitations.
SAST (Static Application Security Testing)
“Static” in this context means source code, nothing else — no compiled objects, no binaries, no libraries, no running application instances.
The simplest SAST scanner is grep, although today I’d use ripgrep.
It’s a perfect example of a pattern-based SAST scanner. Want to find all occurences in C code where strcpy is used? Just run rg strcmp
and it will recursively scan your code base for all occurences of the function. This how first scanners operated back in 2000’s, and this is how some
scanners on the market still work, especially with more esoteric programming languages such as SAP ABAP. This method is however quite blind to
the context of the potentially dangerous function call and doesn’t account for any data flow spanning across multiple lines or files of the code.
This is why new generation of scanners was created that actually understands the application’s code and data flow, usually using Abstract Syntax Tree (AST) (AST) but also much more interesting techniques are employed such as code emulators etc. The scanner builds an in-memory model of all the possible application execution flows and then queries it for known vulnerabilities, often with a whole SQL-like query syntax. Scanners I much respect like Checkmarx and Semmle operate this way.
Some SAST scanners operate on source code only, some will scan compiled object files, most likely because it’s easier for them to build AST when the compiler did the initial job of parsing the source code. I know Veracode, HP Fortify and SpotBugs subscribe to this approach.
Advantages
- Static scanning can be performed from the very first line of the code written and you don’t need a working build environment, libraries, compiler etc (well, that is unless you use one of the “binary” SAST scanners). SAST works for auditors and other third-party code reviewers.
- These features allow to start scanning very early into the project, deploy scanner on developer workstations but also scan third-party code.
- Static scan will also cover all of the supplied code, so it can find potentially dangerous constructs even in code blocks that are executed under very unusual conditions. Yes, that’s about the test coverage.
Limitations
- Most static scanners will require a lot of careful tuning to make any sense: if you drop an existing, large application code into a static scanner you will be usually literally flooded with thousands of “high” and “medium” alerts, and most of them will be of little relevance. Amount of work needed to filter relevant ones is often not much less than finding them by manual code review. This aspect is probably the most frequent cause of SAST failures in real-world projects, and reason for objection from developers who were there before.
- Static scanner will just see the source code supplied. It will not see source code of third-party libraries used by your application, unless you explicitly scan them too. And yes, that’s also about the test coverage.
- It will not see vulnerabilities in infrastructure components such as load balancers, proxies, web servers, operating system and everything else that makes a “web application”. Duh!
DAST (Dynamic Application Security Testing)
“Dynamic” here means that there’s an application prototype (full-stack app, one microservice) running somewhere (developer workstation, QA/UAT environment) and a specialised security scanner is talking to it over HTTP and sending known attack vectors while watching for responses suggesting a successful exploitation. From functional point of view I see two primary groups here: interactive DAST scanners used for manual vulnerability assessment (BurpSuite, OWASP ZAP) and headless scanners (again, ZAP) that can be embedded into the CI/CD pipeline.
Most importantly, the purpose of DAST scanner is not to replace proper penetration testing, but rather ensure predictable and frequently repeated testing against known and newly discovered (yes, you need to update the scanner) attacks to catch them early in the development process, and approach sign-off pentest and production deployment with confidence and peace of mind.
Pros
- The scanner looks at the application as a whole. It doesn’t care about programming languages used to write the backend, load balancers, proxies, databases etc. It just sends evil inputs and looks at responses, which is a great way to catch ephemeral vulnerabilities arising on interfaces between these components or in infrastructure configuration.
- DAST is a great way to catch all these low-hanging fruits that are visible on the first look by someone who inspects your application from security point of view. If the developers can’t even get their basic HTTP headers or versions of Nginx right, how does the rest of the app look like?
- DAST can be easily run as part of or parallel to your functional testing suite (you do have one, don’t you?), it can be actually implemented as part of it without any dedicated security scanners. If the test is “when I send <script> does it come back in HTML” you can just use any native testing web application testing suite (like Django or Dropwizard) to implement it. The advantage is that the native test suites are usually light years ahead of dedicated security scanners in terms of performance and robustness.
- There are numerous SaaS DAST providers. GitHub now offers automated ZAP scans as part of their pipeline. YMMV, caveat emptor.
Limitations
- False positives. Is a late HTTP response indication of a time-based SQL attack or just network lag? Is this HTTP 200 OK response indication of a successful attack or it’s just the application sanitised your carefully mutated XXE vector or doesn’t care about it because there’s no XML parser on the other end?
- Large “security testing” companies made a significant semantic shift here: they started selling time of a junior operator equipped with an interactive DAST scanner as “penetration testing” (but it’s merely a “vulnerability assessment”), while actual penetration testers are sold as “red teams”. Just remember about this when you get an expensive “penetration test” report that looks just like a branded copy of a BurpSuite report. It’s not fault of the scanners, it’s how marketing works.
- DAST scanner needs to replay HTTP request a web browser does. If it’s a classic HTML-based web app, it can crawl it and then replay whatever requests it saw. Things get complicated with JavaScript: crawling is at best unreliable, so we usually record the user journey in a regular browser (Selenium, MITM proxy) and then just replay it in the DAST scanner, but then you need to re-record any changes or new functionality. And then you got all the tricky authentication schemes… Yes, test coverage again…
- Without authentication and user journey replay or crawling, the scanner will be only able to hit the URL you pointed it to. You need to understand precisely what the scanner is able to see, especially with DAST scanners offered as SaaS.
The below screenshot serves as a good example of DAST features and limitations at the same time: majority of the issues reported are either false positives or nominal best practice deviations not justifying a “medium risk” classification.
Where to DAST?
Usually you want to run a DAST scan against your QA/UAT environment as it’s where the scanner can be allowed to unfold the whole battery of truly evil tests. You don’t want these in production instance as they will inevitably pollute your comments with ugly XSS vectors and other test data.
At the same time it does make sense to run DAST scans against production to detect, again, low-hanging fruits left from deployment misconfigurations, vulnerable versions of software or JavaScript libraries.
The best balance here is to run a fully scripted, authenticated scan in offensive mode against a disposable UAT environment, and then an unauthenticated scan with passive rules against production instance.
Oh, and don’t forget both DAST and SAST scanners need to be updated frequently. A scanner last updated in 2010 will not see anything that happened in information security since then and real-world attackers rarely indulge themselves with decade old exploits. They are the first to adopt the bleeding edge techniques.
To be continued
This part covers the two classic tool groups: SAST and DAST. In the next part I will discuss IAST, dependency checkers and RASP. You might also have a look at the Choosing security scanners for your project article.