The State of Continuous Integration and Continuous Delivery Report 2024

The recently published State of CI/CD Report 2024 by SlashData, and the Continuous Delivery Foundation highlights the adoption and importance of strategic CI/CD tool implementation. There is a strong need for organizations to support their developers with education and streamlined tool integrations and process to fully leverage CI/CD DevOps practices. Watching the feedback loops for next level of measures can help improve engagement.

Observations: Growth, Performance, Challenges and Trends

The growth in engagement, with 83% of developers involved in DevOps activities, represents a strong advancement in culture. Deployment metrics that have not significantly changed over past years across many organizations - underscore the effectiveness of CI/CD toolchains in sustaining value generation through software delivery performance.

Data shows utilization of both managed and self-hosted CI/CD toolchains promotes adaptable environments for developers and changing technology landscape — and sustaining innovation at pace, while maintaining a level of control on risks.

Organizations would benefit from stronger investment in tool simplification and continous developer coaching, based on key concerns noted in the report, i.e.,

• The increase in issues such as inadequate training or integration complexities.
• Consolidation around fewer CI/CD and DevOps technologies suggests industry maturation but also highlights the need for focused competency development in these tools.
• The use of multiple tools of the same type (either all managed or all self-hosted) frequently leads to poorer performance due to interoperability issues.

Improvement Recommendations

It is crucial for organizations to invest into continously streamlining the developer tool ecosystems to minimize procedural complexity and enhance interoperability in the enterprise technology landscape. With emergence and greater adoption of AI, Cloud, On-Prem, SaaS providers, etc, this becomes more relevant now than ever.

Ongoing education and training for developers are essential to ensure effective utilization of CI/CD technologies as tools and integrations evolve, with new learnings emerging through measurable feedback loops.

Consider the strategic use of CI/CD tools to foster an environment of continuous improvement and innovation, rather than merely pushing metrics to drive adoption.

Advancing Metrics

Typically suggested metrics for successful CI/CD programs cover, leading measures like frequency of deployment, change lead time, percentage of failed deployments; and lagging metrics such as customer satisfaction scores, software uptime, and defect escape rate. Here are some new measures to think about in order to allow teams to incorporate feedback loops that will enable organizational leaders, and owners of CI/CD tools in an enterprise to further simplify tool chain integrations and processes, while improving the developer engagement by making it safe to move software change forward and by creating new oppurtunities for them through collaborative channels.

1. Third-Party Dependency Updates Response Time: Track how quickly the system adapts to updates or changes in third-party services or libraries, which is crucial for maintaining system stability and security.
2. Feature Development Velocity: Measure the time from ideation to feature deployment to assess how quickly new ideas are being turned into functional software for customers.
3. Automation Coverage Ratio: Evaluate the percentage of processes (testing, deployment, monitoring) that are automated versus those still performed manually, steering increase in automation over time.
4. Pull Request Time-to-Engagement: Track the time it takes for the first review or interaction with a pull request, as faster engagement can lead to quicker integration cycles and smaller changes that could be easily rolled back pose lower risk to disruption.
5. Feedback Loop Efficiency: Measure the speed and effectiveness of how user feedback is incorporated into the development process, which can drive continuous improvement and user satisfaction.
6. Ecosystem Contribution Index: Quantify contributions to external and/or internal organizational projects or community tools that your team uses, reflecting on how your team is supporting and leveraging the broader developer ecosystem.

Predictive Performance with AI/ML

AI/ML capabilities available today could offer avenues to drive powerful operational cost savings by avoiding failed software deliveries, non-compliance, business and customer impact, or lack of and/or deviations in developer engagement levels.

1. Predictive Failure Rates: Use machine learning models to predict the likelihood of a build or deployment failing based on historical data, allowing preemptive action.
2. Risk Assessment Scores: Implement scoring systems that analyze the risk of changes based on code complexity, change size, and historical impact, aiming to predict and mitigate potential disruptions.
3. Developer Sentiment Analysis: Apply sentiment analysis tools to commit messages, pull requests, and developer communications to gauge the mood and engagement levels within the team.

The goal of these new metrics is to make it safe and fast for the developers to delivering software changes, as well to show case measures that actually motivate the developers.

I would love to hear your insights on these trends and measures. What priorities are you setting in software delivery management? How do you perceive the proposed measures, and what additional metrics could drive continuous leadership and developer engagement?.