Welcome!

SDN Journal Authors: Liz McMillan, Yeshim Deniz, Elizabeth White, Pat Romanski, TJ Randall

Related Topics: @DevOpsSummit, Linux Containers, Containers Expo Blog, SDN Journal

@DevOpsSummit: Blog Post

Logging and Continuous Delivery By @FloMotlik | @DevOpsSummit [#DevOps]

Continuous Delivery is the future of building high class software

Guest blog post by Florian Motlik, Cofounder & CTO of Codeship Inc.

Why Great Logging Is Key to Continuous Delivery

Over the last years Continuous Delivery has gained a massive following with many development teams embracing the style. Companies have chosen (as with many other modern developer tools), to either build their own embrace a hosted service likeCodeship. In the end though, no matter if you go with a hosted service or roll it on your own, the goal is to move faster and build a product that your customers really love. For that you need to iterate quickly, get feedback and iterate again.

Successfully rolling out that process depends on many variables. Proper logging is one of those variables, and can be a helpful tool to remove fear.

Why great logging is key to continuous delivery

Thou shalt not be afraid
As we've moved into the age of cloud software development, it is all about team productivity. Getting started on infrastructure is virtually free, so every team plays on the same level playing field and you need to constantly increase productivity to win.

By far the biggest killer of productivity is fear of moving faster because you might break things.

Then your team, processes or technology are not built with constant change in mind and you decrease the speed in which you release to have better control. This is a downward spiral that will only lead to slower processes, less innovation and you losing your market.

Fear stops experiments and promotes stagnation.

Having a repeatable and easily automated process can drastically reduce that fear. The more you execute that automated process the safer you feel.  Over time this will become stronger as all potential issues get discovered and fixed.

A second very important improvement is getting deep insights into any processes and workflows happening in your application.

When you continuously deploy changes to your application, being able to trace any step your application makes becomes your main tool to debug your production system.

The insight you can gain quickly from looking through your logs will often show you the problem a recent deploy brought into your infrastructure immediately.

This is indispensable with Continuous Delivery.

While Metrics are a great and very important part of getting insight into your infrastructure they only represent the state of the system. Additionally to understand the state you need to be able to trace and deeply understand anything happening in your infrastructure.

Integration with Pager systems provide an additional level of security on top of that to always be aware of problems happening.

Let there be light
We've grown accustomed to have full insight into our testing and deployment process as we are and have been using Codeship to build Codeship for a very long time. We understood we needed that same insight into our application as well to build the kind of infrastructure that supports the quality we want to deliver.

A good logging strategy and overview on your most important workflows is necessary.

Defining a graph of all the states that a workflow can have in your system makes it easy to add logging to each of those states and the transitions between them. At that point logging is not an afterthought, but integral part of your software development effort.

This needs to be clearly communicated to your team so everyone follows it thoroughly.

For example we test and deploy code for thousands of companies with many different language and infrastructure requirements. Those companies connect to GitHub or Bitbucket as their source code management system and various hosting providers like Heroku or AWS to deploy to. There are many moving parts in that system so we need to be able to detect and debug problems anytime without effort. A well thought out logging strategy can help tremendously and make it easy to fix issues when they come up. We can follow any build through all of our infrastructure and correlate issues between builds or our infrastructure anytime.

You can read more about how we use Logentries at Codeship in an earlier post.

Grand Central Logging
Being able to follow any workflow means collecting logs of various services in one place. If your developers have to look through various sources for correlating potential problems productivity and the speed to resolve an issue become very low.

Conclusion
Continuous Delivery is the future of building high class software. Automated testing and deployment build the basis of the workflow, but many other tools like centralized logging and error reporting are important building blocks of that workflow as well.

When your system feels like a black box you will hesitate to release changes to that infrastructure. Make sure you're not stuck with that black box and build a workflow that makes your team more productive and increases your products quality.

If you want to learn more about Continuous Delivery you can also take a look at our crash course which can be found on the Codeship homepage.

More Stories By Trevor Parsons

Trevor Parsons is Chief Scientist and Co-founder of Logentries. Trevor has over 10 years experience in enterprise software and, in particular, has specialized in developing enterprise monitoring and performance tools for distributed systems. He is also a research fellow at the Performance Engineering Lab Research Group and was formerly a Scientist at the IBM Center for Advanced Studies. Trevor holds a PhD from University College Dublin, Ireland.

CloudEXPO Stories
Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throughout enterprises of all sizes. We are offering early bird savings on all ticket types where you can save significant amount of money by purchasing your conference tickets today.
Daniel Jones is CTO of EngineerBetter, helping enterprises deliver value faster. Previously he was an IT consultant, indie video games developer, head of web development in the finance sector, and an award-winning martial artist. Continuous Delivery makes it possible to exploit findings of cognitive psychology and neuroscience to increase the productivity and happiness of our teams.
CloudEXPO | DevOpsSUMMIT | DXWorldEXPO Silicon Valley 2019 will cover all of these tools, with the most comprehensive program and with 222 rockstar speakers throughout our industry presenting 22 Keynotes and General Sessions, 250 Breakout Sessions along 10 Tracks, as well as our signature Power Panels. Our Expo Floor will bring together the leading global 200 companies throughout the world of Cloud Computing, DevOps, IoT, Smart Cities, FinTech, Digital Transformation, and all they entail.
Eric Taylor, a former hacker, reveals what he's learned about cybersecurity. Taylor's life as a hacker began when he was just 12 years old and playing video games at home. Russian hackers are notorious for their hacking skills, but one American says he hacked a Russian cyber gang at just 15 years old. The government eventually caught up with Taylor and he pleaded guilty to posting the personal information on the internet, among other charges. Eric Taylor, who went by the nickname Cosmo the God, also posted personal information of celebrities and government officials, including Michelle Obama, former CIA director John Brennan, Kim Kardashian and Tiger Woods. Taylor recently became an advisor to cybersecurity start-up Path which helps companies make sure their websites are properly loading around the globe.
ClaySys Technologies is one of the leading application platform products in the ‘No-code' or ‘Metadata Driven' software business application development space. The company was founded to create a modern technology platform that addressed the core pain points related to the traditional software application development architecture. The founding team of ClaySys Technologies come from a legacy of creating and developing line of business software applications for large enterprise clients around the world.