Welcome!

SDN Journal Authors: Pat Romanski, Destiny Bertucci, Liz McMillan, Elizabeth White, Amitabh Sinha

Related Topics: @DevOpsSummit, Java IoT, Microservices Expo, Linux Containers, @CloudExpo, SDN Journal

@DevOpsSummit: Blog Feed Post

Measuring Your DevOps Success

When you start delivering your DevOps initiatives, the measurement starts immediately and is constant

A survey from the Vanson Bourne market research agency (with CA) late in 2013 indicated that 39% of those surveyed had adopted some form of DevOps and 27% were planning to do so in the near future. Despite this being such a hot topic in the IT sector, and with a high level of take-up, the question we are still most commonly asked is: “Where do we start?”

Our answer is always that an organization’s current position must be baselined first. Having a baseline means you can build a business case, apply targets and goals to your projects and measure your success as you progress through your project with the ultimate goal being to report back to the board on how you used the money to save or make more money – and improved your teams’ satisfaction.

DevOps Metrics for Baselining and Measuring Success
There are hard, quantifiable technical and financial metrics we can track, such as:

  • Number and frequency of software releases
  • Volume of defects
  • Time/cost per release
  • MTTR*
  • Number and frequency of outages / performance issues
  • Revenue/profit impact of outages / performance issues
  • Number and cost of resources

It’s worth noting that one of the biggest inhibitors to success of DevOps and related tooling projects is people’s perceptions that they are at risk of losing their jobs as their work becomes automated. Often, particularly in areas like release and deployment management, we find that there are specific individuals who hold all the knowledge around a current process (they wrote all the scripts for example) and who are viewed as heroes when they are the only person who can fix an issue and often do it out of office hours and at short notice – but are, in fact, bottlenecks. These individuals are often highly talented, but feel secure in the indispensable role they have created for themselves. Though they will often be happier when freed up to do more creative and rewarding work, they are often fearful and this needs to be addressed.

Cultural Metrics
Although cultural metrics are difficult to apply hard dollar value to, DevOps is about resolving conflict in the workplace, eliminating stress and avoiding burnout – and they are measurable. Happy people are more productive – their health is better, they have more ideas, work more effectively and will put in the extra mile. You can measure across a number of key cultural indicators around feelings about change, failure, going to work, what a typical day’s work entails, in addition to a number of cultural attributes such as:

  • Cross-skilling, knowledge sharing and pairing between teams
  • Working in a fluid but focussed manner
  • Working in multidisciplinary teams
  • Organizing teams around projects rather than skill-sets
  • Constantly dancing on the edge of failure (in a good way)
  • Position around business demand
  • Extraneous lines of code
  • Attitude to continuous improvement
  • Obsession with metrics
  • Technological experimentation
  • Team autonomy

You can also look at a number of team features such as:

  • Rewards and feelings of success
  • Hierarchical and political obstacles and annoyances
  • Inspiring and fostering creativity

Process Metrics
DevOps is not a process or a tool – but there are a number of processes in the software development lifecycle (SDLC) that affect both traditional development and operations staff to greater or lesser degrees that need to be taken into consideration. All of these process components can be optimized, and all of them can then be improved upon further using appropriate software tooling. An ultimate goal of a typical DevOps project is often to attain true continuous delivery (CD) by linking these processes and tools together to allow fully tested, production-ready, committed code to proceed to live without impediment – we often refer to the software infrastructure piece of this as the DevOps toolchain. When baselining current state, it’s useful to measure these component processes and their relative maturity (taking into account use of existing tools and success of implementation). Typically, we look at:

  • Requirements elicitation and management
  • Agile development
  • Build
  • Release and deployment
  • Unit testing
  • User Acceptance Testing
  • Quality Assurance
  • Application Performance Monitoring
  • Cloud

How to Influence Metrics
Once you’ve baselined your current position, it’s time to think about your desired future state. Your baselining exercise will probably have highlighted where the key bottlenecks are and areas on which to concentrate. Although we preach, “People, then Process, then Tools”, there are tools, in particular Application Performance Management that can help discover bottlenecks and issues upfront – although it’s imperative you have the right people then using and acting on this data and put the right processes in behind in terms of defect tracking, corrective development, versioning build and deployment.

While you might be stumped on ideas for influencing cultural, especially in well-established enterprises, it is by no means impossible. The key is understanding the current culture. For instance, does a culture of “blame” exist? What happens when there is a production outage? How motivated and rewarded do staff feel? Once you’ve identified the prevailing culture you can establish a program of cultural initiatives to move the culture to one that is more productive and collaborative.

DevOps Tools for Change
There are a number of tools that can influence the harder and software metrics – for example:

Application Performance Monitoring (APM)

  • Reduces MTTR
  • Makes it easier to create a collaborative approach in teams dealing with issues
  • Identifies root cause fast; eliminating blame games

Application Release Automation (ARA)

  • Enables development to seamlessly transition code to operations who can quickly and consistently deploy into production
  • Enables instant rollback or redeploy when an error is identified in production
  • Reduces fear of failure as rollback/redeploy is so easy

Integration Testing & Virtualization

  • Mimics the production environment so successful test are guaranteed to run
  • Allows testers to ‘shift left’ in test process and collaborate with developers early
  • Fast testing enables fast, confident throughput of change

When to Measure, How to Tweak
When you start delivering your DevOps initiatives, the measurement starts immediately and is constant. You will be looking for upward trends, as well as downward trends. Make sure you share reports regularly with the team – try weekly with the core team and monthly with the extended team. Highlight success and elicit ideas for improvement where areas have proved more challenging. Try things – tweak, monitor, tweak again. But remember: “Any improvement not made at the constraint is an illusion.”

What to Do with Success
Celebrate success! Create rewards and incentive programmes for teams when metrics targets are achieved. Part of the DevOp’s agenda is about improving working conditions – depressurising and destressing environments and having everyone working together in harmony and eliminating disasters, catastrophes, blame and brinkmanship.

Why ask for an external DevOps Maturity Assessment
While no one’s going to understand your business as well as yourselves, we often meet organizations who are struggling to find the time – they know there are improvements to be made but they are so busy with firefighting they can’t conceive of stopping and taking stock of their current position. Also, as human beings are emotional creatures — it’s natural for all working environments to have some level of politics or hierarchies going on — it’s often helpful to have an outsider take a pragmatic, neutral view of a situation. So if you are ready to baseline your current DevOps state and identify the DevOps initiatives that will have real, positive impact on your business, but feel you do not have the time to figure it all out yourselves – please do get in touch with us @ranger4ltd.

* The MTTR is the Mean Time to Repair, Resolve or Resolution – each of the definitions mean the same thing and can be used interchangeably. This term is more commonly used when talking about Application Performance Management and the speed at which an outage or performance issue can be fixed, but equally can be used when talking about testing and eliminating defects

 

Take five minutes to get complete visibility into the performance of your production applications with AppDynamics today.

This is guest post by our partners at Ranger 4. Find out more about them on by following @ranger4ltd on twitter.

The post Measuring Your DevOps Success written by Helen Beal appeared first on Application Performance Monitoring Blog from AppDynamics.

Read the original blog entry...

More Stories By AppDynamics Blog

In high-production environments where release cycles are measured in hours or minutes — not days or weeks — there's little room for mistakes and no room for confusion. Everyone has to understand what's happening, in real time, and have the means to do whatever is necessary to keep applications up and running optimally.

DevOps is a high-stakes world, but done well, it delivers the agility and performance to significantly impact business competitiveness.

@CloudExpo Stories
In his general session at 21st Cloud Expo, Greg Dumas, Calligo’s Vice President and G.M. of US operations, discussed the new Global Data Protection Regulation and how Calligo can help business stay compliant in digitally globalized world. Greg Dumas is Calligo's Vice President and G.M. of US operations. Calligo is an established service provider that provides an innovative platform for trusted cloud solutions. Calligo’s customers are typically most concerned about GDPR compliance, application p...
"I focus on what we are calling CAST Highlight, which is our SaaS application portfolio analysis tool. It is an extremely lightweight tool that can integrate with pretty much any build process right now," explained Andrew Siegmund, Application Migration Specialist for CAST, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
As many know, the first generation of Cloud Management Platform (CMP) solutions were designed for managing virtual infrastructure (IaaS) and traditional applications. But that's no longer enough to satisfy evolving and complex business requirements. In his session at 21st Cloud Expo, Scott Davis, Embotics CTO, explored how next-generation CMPs ensure organizations can manage cloud-native and microservice-based application architectures, while also facilitating agile DevOps methodology. He expla...
SYS-CON Events announced today that Evatronix will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Evatronix SA offers comprehensive solutions in the design and implementation of electronic systems, in CAD / CAM deployment, and also is a designer and manufacturer of advanced 3D scanners for professional applications.
SYS-CON Events announced today that Synametrics Technologies will exhibit at SYS-CON's 22nd International Cloud Expo®, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. Synametrics Technologies is a privately held company based in Plainsboro, New Jersey that has been providing solutions for the developer community since 1997. Based on the success of its initial product offerings such as WinSQL, Xeams, SynaMan and Syncrify, Synametrics continues to create and hone inn...
Cloud Expo | DXWorld Expo have announced the conference tracks for Cloud Expo 2018. Cloud Expo will be held June 5-7, 2018, at the Javits Center in New York City, and November 6-8, 2018, at the Santa Clara Convention Center, Santa Clara, CA. Digital Transformation (DX) is a major focus with the introduction of DX Expo within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive ov...
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. In his session at @BigDataExpo, Jack Norris, Senior Vice President, Data and Applications at MapR Technologies, reviewed best practices to ...
Continuous Delivery makes it possible to exploit findings of cognitive psychology and neuroscience to increase the productivity and happiness of our teams. In his session at 22nd Cloud Expo | DXWorld Expo, Daniel Jones, CTO of EngineerBetter, will answer: How can we improve willpower and decrease technical debt? Is the present bias real? How can we turn it to our advantage? Can you increase a team’s effective IQ? How do DevOps & Product Teams increase empathy, and what impact does empath...
DevOps promotes continuous improvement through a culture of collaboration. But in real terms, how do you: Integrate activities across diverse teams and services? Make objective decisions with system-wide visibility? Use feedback loops to enable learning and improvement? With technology insights and real-world examples, in his general session at @DevOpsSummit, at 21st Cloud Expo, Andi Mann, Chief Technology Advocate at Splunk, explored how leading organizations use data-driven DevOps to close th...
Smart cities have the potential to change our lives at so many levels for citizens: less pollution, reduced parking obstacles, better health, education and more energy savings. Real-time data streaming and the Internet of Things (IoT) possess the power to turn this vision into a reality. However, most organizations today are building their data infrastructure to focus solely on addressing immediate business needs vs. a platform capable of quickly adapting emerging technologies to address future ...
With tough new regulations coming to Europe on data privacy in May 2018, Calligo will explain why in reality the effect is global and transforms how you consider critical data. EU GDPR fundamentally rewrites the rules for cloud, Big Data and IoT. In his session at 21st Cloud Expo, Adam Ryan, Vice President and General Manager EMEA at Calligo, examined the regulations and provided insight on how it affects technology, challenges the established rules and will usher in new levels of diligence arou...
Most technology leaders, contemporary and from the hardware era, are reshaping their businesses to do software. They hope to capture value from emerging technologies such as IoT, SDN, and AI. Ultimately, irrespective of the vertical, it is about deriving value from independent software applications participating in an ecosystem as one comprehensive solution. In his session at @ThingsExpo, Kausik Sridhar, founder and CTO of Pulzze Systems, discussed how given the magnitude of today's application ...
There is a huge demand for responsive, real-time mobile and web experiences, but current architectural patterns do not easily accommodate applications that respond to events in real time. Common solutions using message queues or HTTP long-polling quickly lead to resiliency, scalability and development velocity challenges. In his session at 21st Cloud Expo, Ryland Degnan, a Senior Software Engineer on the Netflix Edge Platform team, will discuss how by leveraging a reactive stream-based protocol,...
Mobile device usage has increased exponentially during the past several years, as consumers rely on handhelds for everything from news and weather to banking and purchases. What can we expect in the next few years? The way in which we interact with our devices will fundamentally change, as businesses leverage Artificial Intelligence. We already see this taking shape as businesses leverage AI for cost savings and customer responsiveness. This trend will continue, as AI is used for more sophistica...
In his session at 21st Cloud Expo, Raju Shreewastava, founder of Big Data Trunk, provided a fun and simple way to introduce Machine Leaning to anyone and everyone. He solved a machine learning problem and demonstrated an easy way to be able to do machine learning without even coding. Raju Shreewastava is the founder of Big Data Trunk (www.BigDataTrunk.com), a Big Data Training and consulting firm with offices in the United States. He previously led the data warehouse/business intelligence and B...
Digital transformation is about embracing digital technologies into a company's culture to better connect with its customers, automate processes, create better tools, enter new markets, etc. Such a transformation requires continuous orchestration across teams and an environment based on open collaboration and daily experiments. In his session at 21st Cloud Expo, Alex Casalboni, Technical (Cloud) Evangelist at Cloud Academy, explored and discussed the most urgent unsolved challenges to achieve f...
"Digital transformation - what we knew about it in the past has been redefined. Automation is going to play such a huge role in that because the culture, the technology, and the business operations are being shifted now," stated Brian Boeggeman, VP of Alliances & Partnerships at Ayehu, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
You know you need the cloud, but you're hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You're looking at private cloud solutions based on hyperconverged infrastructure, but you're concerned with the limits inherent in those technologies. What do you do?
"We started a Master of Science in business analytics - that's the hot topic. We serve the business community around San Francisco so we educate the working professionals and this is where they all want to be," explained Judy Lee, Associate Professor and Department Chair at Golden Gate University, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.