Benchmarks
The OWASP Benchmark
In 2015, the Open Web Application Security Project (OWASP) Benchmark Project was created to measure the speed, coverage, and accuracy of application security products. Funded in part by the Cyber Security Division of the United States Department of Homeland Security (DHS), the Benchmark Project lets organizations freely assess products they have or are planning to use. The results of running application security products through the Benchmark demonstrate that most organizations need to revisit their application security technology choices because they are using products that are relatively inaccurate and have high false positive rates.
The top-level benchmark results, shown in the figure above, are revealing and surprising. The most accurate dynamic application security testing (DAST) products had an 18% accuracy score. The most accurate static application security testing (SAST) products had a 33% score on the Benchmark. Contrast Assess, an interactive application security testing (IAST) product scored a 100%.
The Benchmark results call into question the way organizations are running their application security programs today, with such heavy reliance on SAST and DAST products. The results suggest that businesses need to look at the strengths and weaknesses of the products that they are using and seriously consider adding additional protections against the areas where their existing products are not delivering.