Welcome!

SDN Journal Authors: Elizabeth White, Yeshim Deniz, Liz McMillan, Pat Romanski, TJ Randall

Related Topics: @DevOpsSummit, Containers Expo Blog, Apache, SDN Journal

@DevOpsSummit: Blog Post

DevOps Teams Get Docker Flexibility @Ruxit | @DevOpsSummit [#DevOps]

If your applications are distributed via Docker containers to give your DevOps teams some flexibility

Network in the Cloud is No Free Lunch

If you have your applications running on AWS or a similar cloud-based solution, you’ve effectively “outsourced” your networking to the cloud as well. Of course, this can be of great value. Most significantly because it frees you from maintaining physical network infrastructure. Not having physical access to your network doesn’t however mean that you’re free from taking care of your network.

A bit of history

In traditional application architectures, network infrastructure was kept under the strict control of network teams. These teams were responsible for upgrading overloaded equipment before problems arose, identifying and replacing weak network links, resolving bottlenecks, observing latency metrics and delayed data delivery, and even detecting security threats. In other words, traditional network teams looked after all seven OSI layers.

Modern architectures need more networking than ever

In cloud-based architectures, the situation is different and the network has become even more important. Let’s imagine a typical cloud-based architecture situation. You run a datacenter with a flexible number of allocated computing instances (for example, due to the pricing model and volatile demands for CPU). Your datacenter serves distributed applications that are backed by, for example, microservices. Additionally, let’s say that your applications are distributed via Docker containers to give your DevOps teams some flexibility. In situations like this you need more networking than ever. Your network must shoulder all the communications required between the microservices. It serves as a virtual nervous system for your applications.

Even though the network is not physically available to system administrators, it still exists and requires attention. It’s often difficult to know where exactly your machines are physically hosted and how they’re connected with other hosts in your network. Related virtual machines and services can even be hosted on the same virtual host, in which case your network exists only as a memory read operation. This means, the physical network very often coexists with multiple virtual networks.

Challenges in cloud-based networks

The inability to physically access a network (OSI layers 1-2) makes it hard for DevOps teams to keep an eye on it. They can use the monitoring tools offered by their cloud provider, for example Cloud Watch, to fetch network metrics like NetworkIn and NetworkOut, but these metrics can be insufficient in detecting network problems.

Here are some of the key challenges that DevOps faces in maintaining virtual network performance:

  • Competing processes for network resources (for example, the TCP Incast problem)
  • Variable network infrastructure through new or stopped instances
  • Scalability of network load via elastic network interfaces
  • Quality of connections inside your data center
  • Quality of connections to private networks outside your data center

Monitoring network usage

Your network monitoring must be able to react to infrastructure changes such as mentioned above. In particular, it needs to be able to handle virtual network interfaces. Hence, your monitoring needs to run on your hosts and consistently keep watch for changes of your virtual infrastructure. From this position it can observe network connections between processes that communicate with other processes and services, thereby monitoring actual network usage instead of just network devices.

Infrastructure with all levels

Resource monitoring is key… and simple!

With this monitoring approach your network won’t be seen simply as a collection of network interfaces, routing tables, and security groups. Rather your network will be viewed as a limited resource used by processes and applications. This resource can be monitored along with CPU, memory, and storage, and even measured at process level. This enables full-stack application performance monitoring and the ability to trace network problems all the way up to the application level.

Network connection quality

There are a few basic network performance metrics you should keep in mind:

  • The traffic of network data (throughput) is the basic indicator for network performance.
  • The Connectivity metric provides the percentage of successfully established TCP (Transmission Control Protocol) connections and indicates accessibility of services. TCP connections may be refused or end up in timeouts, so connectivity is a good indicator of network problems between sender and receiver.
  • With respect to the quality of established TCP connections, the retransmission rate is also worth monitoring. The TCP protocol is a reliable and error-checked protocol. This means the receiver must confirm the packets sent over a network link; otherwise they are considered lost and then retransmitted by the sender. Therefore, retransmission rate is a good indicator of poor network links and overloaded network infrastructure.

The bottom line is that you shouldn’t blindly trust cloud providers regarding the health of your “outsourced” virtual network infrastructure. Virtualized networks cannot be monitored in a more or less traditional manner. They should at least be monitored from the point of view of your hosts and processes so that you have some meaningful network performance indicators.

The post Network in the Cloud is No Free Lunch appeared first on The ruxit blog.

Read the original blog entry...

More Stories By Dynatrace Blog

Building a revolutionary approach to software performance monitoring takes an extraordinary team. With decades of combined experience and an impressive history of disruptive innovation, that’s exactly what we ruxit has.

Get to know ruxit, and get to know the future of data analytics.

CloudEXPO Stories
Digital Transformation and Disruption, Amazon Style - What You Can Learn. Chris Kocher is a co-founder of Grey Heron, a management and strategic marketing consulting firm. He has 25+ years in both strategic and hands-on operating experience helping executives and investors build revenues and shareholder value. He has consulted with over 130 companies on innovating with new business models, product strategies and monetization. Chris has held management positions at HP and Symantec in addition to advisory roles at startups. He has worked extensively on monetization, SAAS, IoT, ecosystems, partnerships and accelerating growth in new business initiatives.
Whenever a new technology hits the high points of hype, everyone starts talking about it like it will solve all their business problems. Blockchain is one of those technologies. According to Gartner's latest report on the hype cycle of emerging technologies, blockchain has just passed the peak of their hype cycle curve. If you read the news articles about it, one would think it has taken over the technology world. No disruptive technology is without its challenges and potential impediments that frequently get lost in the hype. The panel will discuss their perspective on what they see as they key challenges and/or impediments to adoption, and how they see those issues could be resolved or mitigated.
Lori MacVittie is a subject matter expert on emerging technology responsible for outbound evangelism across F5's entire product suite. MacVittie has extensive development and technical architecture experience in both high-tech and enterprise organizations, in addition to network and systems administration expertise. Prior to joining F5, MacVittie was an award-winning technology editor at Network Computing Magazine where she evaluated and tested application-focused technologies including app security and encryption-related solutions. She holds a B.S. in Information and Computing Science from the University of Wisconsin at Green Bay, and an M.S. in Computer Science from Nova Southeastern University, and is an O'Reilly author.
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more business becomes digital the more stakeholders are interested in this data including how it relates to business. Some of these people have never used a monitoring tool before. They have a question on their mind like "How is my application doing" but no idea how to get a proper answer.
Having been in the web hosting industry since 2002, dhosting has gained a great deal of experience while working on a wide range of projects. This experience has enabled the company to develop our amazing new product, which they are now excited to present! Among dHosting's greatest achievements, they can include the development of their own hosting panel, the building of their fully redundant server system, and the creation of dhHosting's unique product, Dynamic Edge.