Application Security in DevOps: 3 Key Success Factors

In the traditional development model, application security has been hindered by an artificial separation between development and security teams. On one hand, the development team was trying to produce high-quality applications that responded to the needs of its users - whereas the security team was tasked with ensuring that applications weren’t subject to potential vulnerabilities. Security testing was often perceived as a complicated undertaking that could only be performed by experts. As a result, security testing was usually performed as the final step of the development process, just before the product was released.

This approach to workflow always contained flaws. Finding a severe vulnerability at the last moment resulted in delays to release-dates. Even worse, the workflow lent itself to delivering applications with known vulnerabilities. When development teams shifted to agile development and DevOps methodologies, these problems worsened. Development cycles were decreased to 2 or 3 weeks and new app versions were delivered more frequently. However, security teams couldn’t handle the fast pace and were often left out of the release process altogether.

Something had to change. The balance between meeting business growth objectives by releasing new app versions, and protecting the business from security breaches by detecting and mitigating security vulnerabilities, got disrupted. Security testing simply took too long. The solution to this problem came in the form of DevSecOps. This new approach made security a part of the operations side of DevOps by including the security team and security testing in the process.

DevSecOps, or Shall We Say SecDevOps?

DevSecOps (sometimes referred to as SecDevOps) has been around for a while, but it is still a relatively new buzzword. A Google search on the two terms yielded approximately 180,000 results on June 29, 2017. This may sound like a lot of results, but it is merely a fraction compared to the 20,600,000 hits that are received when searching for the term DevOps. There is agreement on what DevSecOps is and its purpose - to integrate security testing into DevOps to produce secure applications. However, there are different approaches to achieving this. I want to share my views on DevSecOps and how development teams can maximize available technologies to improve their application security posture, without disrupting their DevOps processes.

3 Key Success Factors

The value you will receive from integrating security testing into DevOps depends on how vastly it is adopted by your development teams and how many vulnerabilities remain in your applications when development is completed. Both of these metrics can be measured.

Given the above drawbacks with traditional methods, the need for adopting security testing as part of DevOps is clear. However, making it work requires buy-in from development teams, and that is never easy. In my years of experience in development roles and according to the customers that I meet with regularly in my current role, most developers prefer to spend all of their time within their Integrated Development Environment (IDE), with occasional peeks at their Defect Tracking System (DTS). Some developers are open to getting their hands dirty with build and quality tools, but the majority of developers keep their distance from those tools. Also, developers prefer to deliver new capabilities, since they are often measured by the timeliness of those deliveries. They typically have low tolerance for wasted time and will do their best to avoid tools that requires a lot of extra time commitment. Application security tools have a notorious reputation for being complicated and require a lot of hand-holding by experts. Those tools will likely be rejected by developers because the tools take them away from developing applications.

Success Factor #1: Automation

This brings me to the first success factor for integrating application security testing into DevOps: Automation. Any tools that are utilized must work automatically and must not disrupt the developers’ experience. Teams should be able to integrate the tools into their existing DevOps ecosystem, both technically and in spirit. It is not enough to automatically invoke a security scan from an IDE or Continuous Integration (CI) tool. If that scan relies on manual intervention, it will miss the point. Real automation means that security scans will automatically begin and complete - and no manual steps will be required from development or security teams (or the vendor supplying the solution, for that matter). Any other approach will derail the DevOps experience.

Success Factor #2: Speed

The second success factor is Speed. Any tool used for DevOps should respect companies’ desire for faster time-to-market, which cannot be satisfied if it takes days for scan results to be returned. But, speed assessments shouldn’t be limited to the time it takes for scans to be completed. Speed should also be measured by the total time it takes to detect and fix vulnerabilities. A good solution can’t just dump a high volume of findings (as accurate as they may be) on a development team with a “for help, please dial 1-800-VendorX" note attached to the findings. The output of scans should provide developers with all of the details they need to mitigate detected vulnerabilities as fast as possible.

Having a speedy and automated solution is important, but when it comes with the price of detecting fewer vulnerabilities, it misses the goal to produce secure applications. It is obvious that when you scan with fewer tests, the scan will complete faster. But does that mean that you should reduce the scope of your testing? In Forrester’s “The State Of Application Security: 2016 And Beyond,” Amy DeMartine wrote, “…static application security testing (SAST) rule sets are often decreased to limit the number of false positives so that developers can continue to work at speed.”

But where do you draw the line? Do you scan just 80% of your applications? Or maybe 50% ? Do you scan for just 75% of vulnerabilities? Or do you maybe just use one technology – and perhaps only concentrate on Static Analysis Security Testing (SAST) - and then have the security team run DAST (Dynamic Application Security Testing)?  But then we need to remember that security teams are not testing every release.

Success Factor #3: Coverage

I agree that partial scanning is better than no scanning at all, but it is a slippery slope and one that is not advisable. A good application security program should not ask for a relief on coverage to meet your DevOps goals. It should find a way to orchestrate the solution in a way that will give you the most coverage as possible for your needs. This coverage should offer the most of each scanning technology - and offer a variety of those technologies. There should be no more single technology solutions. Yes, Coverage is my third success factor.

Like a 3-legged stool, success is dependent on achieving all 3 factors. Automation cannot be achieved without speed and speed should not be achieved by sacrificing coverage.

Measure Your Program Success

There are many ways to measure adoption of your application security testing program. You can focus on different areas and different measurements, but most organizations focus first on these aspects:

-         How many groups participate in the program?

-         How many people joined the program?

-         How frequently are scans executed?

-         How many of applications are being scanned?

These metrics indicate your progress, of course - but knowing that an additional 20 applications were scanned this month doesn’t tell you if you are on the right track. You need to have a defined goal and measure yourself on that. And, compared with your current pace, how long will it take you to reach your goal?

Adoption is the obvious first step - but it doesn’t measure your return on investment:

-         How many vulnerabilities were found this week?

-         How many vulnerabilities were resolved this week?

-         What are my overall vulnerabilities trends?

If the metrics indicate that vulnerabilities are being resolved week-over-week, it means developers are getting value from the tool. If the pace of new vulnerability detection on new code declines, it may mean that developers have learned something from past fixes. But those metrics are not enough to know if your DevSecOps process succeeded. The data for these metrics is limited to the vulnerabilities collected by your automated testing. To fully measure your success, you need to know, after adding security testing, how better off you are in protecting your company from the next potential breach.

Integrating application security testing with DevOps doesn’t mean that security teams aren’t needed anymore. Even with the most effective solution, development teams won’t be able to detect and mitigate all of the security vulnerabilities in applications on their own. As security teams can’t test each version of an application before it is being released, integrating security testing into DevOps should significantly reduce the number of security vulnerabilities that remain in the application before it is shipped. The actual effectiveness of DevOps with application security can be measured by a reduction in the number of vulnerabilities detected by the security team. This, of course, is a difficult and expensive metric to track. Luckily you don’t have to do it for all of your applications. Comparing assessments of an application produced by a development team that leverages automated security testing with a similar application that wasn’t will let you know if things have improved - and it will also permit you to identify your remaining gaps. I assure you that the more application security testing technologies you use in your program, the smaller the gap will be.

Learn More About My Success Factors

There is a lot more information that I can share with you about the success factors outlined above. In fact, each of them probably deserves a post of its own. In upcoming posts, I will provide technical details on how these items came to be the foundation of Application Security on Cloud and how it empowers development teams to run SAST, DAST, and Open Source Testing as part of their web and mobile application development process.

This article is focused on how you can test your applications for security vulnerabilities as part of DevOps. However, it is important to be aware that running an application security program and protecting your business involves more than that. Teams are also encouraged to ensure that other practices, such as threat modeling and development training, are included. And they should invest in an infrastructure that will guarantee the security of their application in production (data and network protection, SIEM integrations, etc.) so that their application security programs are comprehensive.