A top challenge for DevSecOps is how to ensure development pipeline

Traditional security approaches which were incorporated to the waterfall-style software development were criticised to overlook information security and promote the “shift-right” approach in which the security was largely overlooked on the first steps of the software development, and “glued into the software” as the final step in the development process.

DevOps activities has been adopted to ensure higher quality software at a faster development rate. At the same time, the requirements for security have kept on rising. In large organizations the deployment rate of software changes is counted per day which, in turn, means that rapid actions are needed to secure both the development process and the product of it. The security of software is at substantial risk if security is not tied into each step of the DevOps process; both the development process as well as the software as an outcome of it need to be secured.

In this blog, we discuss the most essential findings of the literature review we carried out in 2022 (Leppänen, Honkaranta, Costing, 2022). The literature review was focused on the security postures in DevOps cycle. Our first focus was on the challenges encountered by the DevOps developers. Secondly, we focused to find out the most common postures for security that were taken by the DevOps developers.

Is DevOps security guaranteed with tools for monitoring and reviews?

Most challenges are met on securing the development pipeline, whereas developers are also burdened by overall complexity of contemporary cloud environments consisting of microservices, serverless processing and multitude of differing containers.

Most common way to ensure security by DevOps developers seems to be using diagnostic and automatic tools to monitor application and its health. This seems natural, since the most popular cloud environments provide such tools for environment diagnostics. Security reviews were also carried out (third most-common security posture), and it may be wondered if the challenge of interpreting security requirements on a right way (challenge number five) is related to this posture.

In this review we first discuss the essence of the DevOps software development approach. Then we present the 12 most typical challenges and the 14 most common postures for DevOps security. We wrap up by discussing if the current DevOps approach truly covers the whole development cycle, or not.

What is secure DevOps?

DevOps is a set of practices and tools aimed at bridging development and operations, embracing collaboration, and utilizing automated tools to enable continuous integration with required quality. The need for a concept like DevOps emerged in the early 2000s to improve software development processes. Since then, it has been adopted globally. During past 10 years the concerns related to security postures in the DevOps development have emerged.

In line with the concern on security, several terms related to secure DevOps have been formed by adding the abbreviation “Sec” as a part to the acronym (i.e. DevSecOps). The goal is to keep security as a key focus of the DevOps, throughout the whole DevOps cycle. “Shift left” principle emphasises the idea that security practices should be integrated to the process from the very beginning.

The security posture of DevOps is built on several practices covering both software development environment and process (deployed software). Example of actions that improve security in DevOps are:

  • Adopt management model (how to integrate security across DevOps lifecycle, incl. roles and responsibilities, e.g., security team).
  • Deploy penetration testing and automate security testing (identify main security gaps, run often, provide immediate feedback).
  • Establish risk-based security policies (e.g., code review, vulnerability testing, security tools).
  • Deploy vulnerability management throughout systems development life cycle: scan, evaluate, fix.
  • Collaborate and communicate between development, operations, and security teams to ensure a shared understanding of security requirements and challenges.

If the organization is not capable of adopting principles of secure DevOps (DevSecOps), both the software development pipeline as well as the outcoming software are not covered from the security perspective, leading to vulnerable software development practises and vulnerable software as an outcome.

The 12 challenges for DevSecOps

We updated a literature review that was carried out in 2019 and analysed 38 studies (of which 18 were scrutinized in more detail) [1]. Our interest was to find out the most common challenges and how they were tackled in DevOps projects. We utilised the BSIMM framework [3] as a tool in the review.  Building Security in Maturity Model (BSIMM) was created by Synopsis Software and is a framework that is comprised of activities observed in real-world software security programs on large observation projects throughout years. As a benchmark, it helps organizations to assess their security practices and guides them in enhancing their software security posture.

The main result of our review was a list covering 12 top challenges for secure DevOps developers.

A top challenge for DevSecOps is how to ensure the development pipeline.

In overall, the DevOps security seems to be challenged by how the pipeline is secured, as illustrated in the Figure below. In the figure, the numbers in brackets refer to the references number in which the topics was brought up. For example, “Ensuring pipeline security” was mentioned as a challenge in six of the literature references (references number 29,30,32, 33, 39 and 44), while “Including the security team in the development life cycle” was mentioned only in one reference.

Figure 1. The most common challenges for DevSecOps (Leppänen, Honkaranta, Costin, pp. 7)

Cloud and microservices environments are complex and difficult to secure.

The second on the list of top challenges was the complexity of the contemporary cloud environment, incorporating microservices, containers, and serverless procedures.

DevOps teams double the number of developers in the environment.

Increased insider access refers to the DevOps-specific style of publishing. While the software is constantly being published both the developers and maintainers operate on the same environment, which poses challenges for account management and stresses the need for monitoring the developers, maintainers and specifically the administrators’ operations on the environments. In practise, extensive logging in several levels (software, cloud, containers, databases, access to cloud etc.) is required to raise the Privileged Access Management (PAM) level to the adequate one. Extensive log analysis, and careful use of bastion hosts and related postures are needed, too.

Balancing with automatic and manual postures, which are complex to grasp.

The fast development pace and the developers’ knowledge about security requirements and capability to incorporate security postures into the fast-moving process is also a main concern to DevSecOps, reflected in a several challenges on the list. Incorporating automatic and manual security postures and combining them to the continuous delivery and publishing process is also a common challenge, which quite probably is a challenge for those using more traditional styles of software development, too.

Most typical security practices in DevOps processes

While we looked at the most common ways to secure the DevOps posture, we found out that the most common security posture was to utilise the application diagnostics and other automatised tools provided by the most common clouds, and to ensure that the basic postures for securing the networks were used. Security reviews were also carried out. The top 14 activities for securing the DevOps development are presented in Figure 2 (id in brackets before activity name refers to the corresponding BSIMM activity).

Figure 2. Top 14 activities for securing the DevOps software cycle and outcome.

Other popular postures for security seem also be to use virtualized environments, containers for applications and penetration testing. Use of virtualized environments, containers (such as Docker) also seem to bring out the most challenges for secure DevOps as each container core and microservice enlarges the attack surface scope.

Alarming was, that practices related to governance and intelligence domains in the BSIMM framework received the least attention. These domains contain software security practices and activities that help organize, manage, and measure a software security initiative (e.g., controls for compliance, supply chain risk management, security awareness training), and whose results end up in collections of corporate knowledge (e.g., attack intelligence, secure-by-design, secure coding standards). These collections include both proactive security guidance and organizational threat modelling.

Security practices are not in line with security goals

Our findings also revealed that security practices do not cover the whole software development cycle as they should.

The results revealed that despite the recommended approach to shift security everywhere, DevOps practitioners typically focus their security operations on the deployment and technical aspects of software security.

Collaborative, sharing, and transparent culture is the key to success in DevOps. Coordination between and within teams working for secure software development is crucial despite of the level of automated tools implemented. However, practices and activities are focused strongly on technology and how it is secured in the DevOps infrastructures.

Anne Honkaranta,
Senior Cyber Security  Consultant

Tiina Leppänen,
Senior Security Consultant


This blog is based on the literature review carried out by Leppänen, Honkaranta and Costin; see:

Leppänen, T., Honkaranta, A., Costin, A. (2022). Trends for the DevOps Security. A Systematic Literature Review. In: Shishkov, B. (eds) Business Modeling and Software Design. BMSD 2022. Lecture Notes in Business Information Processing, vol 453. Springer, Cham. (2022) https://doi.org/10.1007/978-3-031-11510-3_12

Link: Trends for the DevOps Security. A Systematic Literature Review | SpringerLink


Additional references for curious:

  1. Koskinen, A.: Devsecops: Building Security Into The Core Of Devops. University of Jyväskylä, Jyväskylä, Finland (2019).
  2. Kitchenham, B: Procedures for performing systematic reviews. Keele University, UK, 33, pp. 1-26 (2004).
  3. SynopsysSoftware: BSIMM12, 2021 Insights Trends Report. https://www.bsimm.com/
  4. Kitchenham, B., Brereton, O., Budgen, D., Turner, M., Bailey, J., Linkman, S.: Systematic literature reviews in software engineering–a systematic literature review. Information and software technology, 51(1), pp. 7–15 (2009).
  5. Rahman, A., Williams, L.: Software security in DevOps: Synthesizing practitioners’ perceptions and practices. In: 2016 IEEE/ACM International Workshop On Continuous Software Evolution And Delivery (CSED), pp. 70–76. IEEE (2016).