Welcome!

SDN Journal Authors: Elizabeth White, Yeshim Deniz, Liz McMillan, Pat Romanski, TJ Randall

Related Topics: @DevOpsSummit, Linux Containers, Containers Expo Blog, SDN Journal

@DevOpsSummit: Blog Post

Logging and Continuous Delivery By @FloMotlik | @DevOpsSummit [#DevOps]

Continuous Delivery is the future of building high class software

Guest blog post by Florian Motlik, Cofounder & CTO of Codeship Inc.

Why Great Logging Is Key to Continuous Delivery

Over the last years Continuous Delivery has gained a massive following with many development teams embracing the style. Companies have chosen (as with many other modern developer tools), to either build their own embrace a hosted service likeCodeship. In the end though, no matter if you go with a hosted service or roll it on your own, the goal is to move faster and build a product that your customers really love. For that you need to iterate quickly, get feedback and iterate again.

Successfully rolling out that process depends on many variables. Proper logging is one of those variables, and can be a helpful tool to remove fear.

Why great logging is key to continuous delivery

Thou shalt not be afraid
As we've moved into the age of cloud software development, it is all about team productivity. Getting started on infrastructure is virtually free, so every team plays on the same level playing field and you need to constantly increase productivity to win.

By far the biggest killer of productivity is fear of moving faster because you might break things.

Then your team, processes or technology are not built with constant change in mind and you decrease the speed in which you release to have better control. This is a downward spiral that will only lead to slower processes, less innovation and you losing your market.

Fear stops experiments and promotes stagnation.

Having a repeatable and easily automated process can drastically reduce that fear. The more you execute that automated process the safer you feel.  Over time this will become stronger as all potential issues get discovered and fixed.

A second very important improvement is getting deep insights into any processes and workflows happening in your application.

When you continuously deploy changes to your application, being able to trace any step your application makes becomes your main tool to debug your production system.

The insight you can gain quickly from looking through your logs will often show you the problem a recent deploy brought into your infrastructure immediately.

This is indispensable with Continuous Delivery.

While Metrics are a great and very important part of getting insight into your infrastructure they only represent the state of the system. Additionally to understand the state you need to be able to trace and deeply understand anything happening in your infrastructure.

Integration with Pager systems provide an additional level of security on top of that to always be aware of problems happening.

Let there be light
We've grown accustomed to have full insight into our testing and deployment process as we are and have been using Codeship to build Codeship for a very long time. We understood we needed that same insight into our application as well to build the kind of infrastructure that supports the quality we want to deliver.

A good logging strategy and overview on your most important workflows is necessary.

Defining a graph of all the states that a workflow can have in your system makes it easy to add logging to each of those states and the transitions between them. At that point logging is not an afterthought, but integral part of your software development effort.

This needs to be clearly communicated to your team so everyone follows it thoroughly.

For example we test and deploy code for thousands of companies with many different language and infrastructure requirements. Those companies connect to GitHub or Bitbucket as their source code management system and various hosting providers like Heroku or AWS to deploy to. There are many moving parts in that system so we need to be able to detect and debug problems anytime without effort. A well thought out logging strategy can help tremendously and make it easy to fix issues when they come up. We can follow any build through all of our infrastructure and correlate issues between builds or our infrastructure anytime.

You can read more about how we use Logentries at Codeship in an earlier post.

Grand Central Logging
Being able to follow any workflow means collecting logs of various services in one place. If your developers have to look through various sources for correlating potential problems productivity and the speed to resolve an issue become very low.

Conclusion
Continuous Delivery is the future of building high class software. Automated testing and deployment build the basis of the workflow, but many other tools like centralized logging and error reporting are important building blocks of that workflow as well.

When your system feels like a black box you will hesitate to release changes to that infrastructure. Make sure you're not stuck with that black box and build a workflow that makes your team more productive and increases your products quality.

If you want to learn more about Continuous Delivery you can also take a look at our crash course which can be found on the Codeship homepage.

More Stories By Trevor Parsons

Trevor Parsons is Chief Scientist and Co-founder of Logentries. Trevor has over 10 years experience in enterprise software and, in particular, has specialized in developing enterprise monitoring and performance tools for distributed systems. He is also a research fellow at the Performance Engineering Lab Research Group and was formerly a Scientist at the IBM Center for Advanced Studies. Trevor holds a PhD from University College Dublin, Ireland.

CloudEXPO Stories
Your job is mostly boring. Many of the IT operations tasks you perform on a day-to-day basis are repetitive and dull. Utilizing automation can improve your work life, automating away the drudgery and embracing the passion for technology that got you started in the first place. In this presentation, I'll talk about what automation is, and how to approach implementing it in the context of IT Operations. Ned will discuss keys to success in the long term and include practical real-world examples. Get started on automating your way to a brighter future!
The challenges of aggregating data from consumer-oriented devices, such as wearable technologies and smart thermostats, are fairly well-understood. However, there are a new set of challenges for IoT devices that generate megabytes or gigabytes of data per second. Certainly, the infrastructure will have to change, as those volumes of data will likely overwhelm the available bandwidth for aggregating the data into a central repository. Ochandarena discusses a whole new way to think about your next-gen applications and how to address the challenges of building applications that harness all data types and sources.
Whenever a new technology hits the high points of hype, everyone starts talking about it like it will solve all their business problems. Blockchain is one of those technologies. According to Gartner's latest report on the hype cycle of emerging technologies, blockchain has just passed the peak of their hype cycle curve. If you read the news articles about it, one would think it has taken over the technology world. No disruptive technology is without its challenges and potential impediments that frequently get lost in the hype. The panel will discuss their perspective on what they see as they key challenges and/or impediments to adoption, and how they see those issues could be resolved or mitigated.
CloudEXPO New York 2018, colocated with DevOpsSUMMIT and DXWorldEXPO New York 2018 will be held November 12-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI and Machine Learning to one location.
DXWorldEXPO LLC announced today that Nutanix has been named "Platinum Sponsor" of CloudEXPO | DevOpsSUMMIT | DXWorldEXPO New York, which will take place November 12-13, 2018 in New York City. Nutanix makes infrastructure invisible, elevating IT to focus on the applications and services that power their business. The Nutanix Enterprise Cloud Platform blends web-scale engineering and consumer-grade design to natively converge server, storage, virtualization and networking into a resilient, software-defined solution with rich machine intelligence.