SDN Journal Authors: Liz McMillan, Yeshim Deniz, Elizabeth White, Pat Romanski, TJ Randall

Related Topics: @CloudExpo, Linux Containers, Containers Expo Blog, Cognitive Computing , SDN Journal

@CloudExpo: Article

Latency Beyond Throughput | @CloudExpo #DataCenter #Storage #SSD #SCM

Consider the analogy of a highway. If it is a one-lane road. Latency is one vehicle making a round-trip between two end points

Storage is moving to flash, and flash is getting faster, so people keep asking me why I keep talking about latency as if there is a problem.  Isn't faster flash going to just make everything faster?  Won't "the rising tide lift all boats"?

Flash media as a storage media is indeed "faster" than the spinning hard-disks we've all been using for decades.  But when it is used to simulate a hard disk, as is the case with SSD products, there are software layers which prevent it from reaching its full potential.  That explanation always gets heads nodding, because it is obvious.  But what about when the flash media is not simply packaged into an "SSD" and connected over SATA or SAS, but instead can be addressed via NVMe over PCIe?  Doesn't that make the problem of hard disk drive emulation go away?

Not entirely.  For one thing, in some cases the SSD abstraction is maintained despite being connected via PCIe.  Faster than SAS or SATA, but still those software speed bumps to keep data from driving too fast through the storage parking lot.

But let's suppose that you have a flash card designed specifically for NVMe and allowing more sophisticated memory-addressing software mechanisms to unleash its greater potential.  And what about the SCM (Storage Class Memory) products coming to market blurring the distinction between DRAM and non-volatile media previously relegated to the storage layer?  Hasn't hardware solved the performance problem?

I wish it were so.  But it comes back to the subtle distinction between latency and throughput, and what it means to the software inside the kernel.

At a high level, it's easy to think of latency as the inverse of throughput (and vice versa).  For a simple, single-threaded series of operations, that should be literally true.  But it is more complicated when you have many operations occurring in parallel.

Consider the analogy of a highway.  If it is a one-lane road. Latency is one vehicle making a round-trip between two end points. Let's imagine a 4-passenger car traveling on a single-lane road for 50 kilometers across a desert between two depots. Each round trip is an event and its completion time is its latency.  If we want to improve its latency we could make the car faster.

Latency - A Faster Car
If we want to improve its throughput we could make the car bigger - replace it with a huge, slow bus - but the net result would be worse latency with more passenger round trips for greater throughput.  We could further increase throughput by building more lanes on the highway and running more vehicles.  Optimally, we would make all the vehicles faster, whether they became even larger trains of trailers pulled by a truck or swarms of speedy motorcycles spreading out over the ever-multiplying number of new lanes.

Throughput - Many Jammed Lanes
The point here is that we really have three performance dimensions to consider:

  1. How quick is the round trip for any one passenger? That is latency.
  2. How many passengers in aggregate per unit time? That is throughput.
  3. How many independent events (vehicles)? That is accesses.

Enabling more accesses will be a natural consequence of lower latency, because the number of lanes is fixed (in the analogy) and the number of queues is finite (applying the analogy to software).

To optimize the use of emerging hardware technology we shouldn't merely rely on building more lanes in the highway (increasing throughput potential via flash media capability).  We should also be making the vehicles faster (improve latency).  Enhancing the hardware is in the hands of Intel, Micron, Samsung, and all the rest of the players in that space.

Simply making use of the bigger/faster/cheaper nonvolatile "flash" hardware components coming to market in wave after wave of impressive innovation is straightforward. Everybody in the storage industry is doing it.  And adding more of it is like adding lanes to the highway to get more throughput.

But doing something meaningful about latency is not easy.  It's hard.  It means rethinking the fundamentals, changing the innards of the kernel, ripping out cruft with both hands and designing new streamlined code to handle storage I/O for the 21st century.

More Stories By Amit Golander

Dr. Amit Golander is the Chief Technology Officer (CTO) and R&D Manager for Plexistor. His responsibilities are to develop the product and work with CEO, Sharon Azulai, on the vision for the technology and products.

Golander brings a rich research, development, and leadership background to Plexistor where he has distinguished himself in both the corporate, startup and higher educational realms.In addition to his work in the business and academic sectors, Golander holds over 50 patents and has published a number of technology articles in prestigious engineering journals.

Prior to Plexistor, Golander was VP of Systems and Product for Primary Data where he was responsible for strategic partnerships, alliances and beta customers as well as worked closely with the R&D teams on the day-to-day product management.Golander also worked for IBM for over twelve years on data center and cloud infrastructure.

Golander has also mentored M.Sc. students and taught computer architecture and quantitative analysis at Tel Aviv University.

Golander received his B.Sc. in C.S.and EE and his Ph.D. from Tel Aviv University in the field of computer architecture. His thesis won the Intel Research Award. Prior to his academic studies, Golander served as an intelligence officer in the Israeli Defense Force (IDF).

CloudEXPO Stories
All in Mobile is a mobile app agency that helps enterprise companies and next generation startups build the future of digital. We offer mobile development and design for smartphones, tablets and wearables. Our projects cover the latest and most innovative technologies - voice assistants, AI, AR/VR and more. We excel at solutions for sports, fintech and retail industries.
NanoVMs is the only production ready unikernel infrastructure solution on the market today. Unikernels prevent server intrusions by isolating applications to one virtual machine with no users, no shells and no way to run other programs on them. Unikernels run faster and are lighter than even docker containers.
The dream is universal: heuristic driven, global business operations without interruption so that nobody has to wake up at 4am to solve a problem. Building upon Nutanix Acropolis software defined storage, virtualization, and networking platform, Mark will demonstrate business lifecycle automation with freedom of choice and consumption models. Hybrid cloud applications and operations are controllable by the Nutanix Prism control plane with Calm automation, which can weave together the following: database as a service with Era, micro segmentation with Flow, event driven lifecycle operations with Epoch monitoring, and both financial and cloud governance with Beam. Combined together, the Nutanix Enterprise Cloud OS democratizes and accelerates every aspect of your business with simplicity, security, and scalability.
CloudEXPO | DevOpsSUMMIT | DXWorldEXPO Silicon Valley 2019 will cover all of these tools, with the most comprehensive program and with 222 rockstar speakers throughout our industry presenting 22 Keynotes and General Sessions, 250 Breakout Sessions along 10 Tracks, as well as our signature Power Panels. Our Expo Floor will bring together the leading global 200 companies throughout the world of Cloud Computing, DevOps, IoT, Smart Cities, FinTech, Digital Transformation, and all they entail. As your enterprise creates a vision and strategy that enables you to create your own unique, long-term success, learning about all the technologies involved is essential. Companies today not only form multi-cloud and hybrid cloud architectures, but create them with built-in cognitive capabilities.
Darktrace is the world's leading AI company for cyber security. Created by mathematicians from the University of Cambridge, Darktrace's Enterprise Immune System is the first non-consumer application of machine learning to work at scale, across all network types, from physical, virtualized, and cloud, through to IoT and industrial control systems. Installed as a self-configuring cyber defense platform, Darktrace continuously learns what is ‘normal' for all devices and users, updating its understanding as the environment changes.