SDN Journal Authors: Liz McMillan, Yeshim Deniz, Elizabeth White, Pat Romanski, TJ Randall

Related Topics: @CloudExpo, Linux Containers, Containers Expo Blog, Cognitive Computing , SDN Journal

@CloudExpo: Article

Latency Beyond Throughput | @CloudExpo #DataCenter #Storage #SSD #SCM

Consider the analogy of a highway. If it is a one-lane road. Latency is one vehicle making a round-trip between two end points

Storage is moving to flash, and flash is getting faster, so people keep asking me why I keep talking about latency as if there is a problem.  Isn't faster flash going to just make everything faster?  Won't "the rising tide lift all boats"?

Flash media as a storage media is indeed "faster" than the spinning hard-disks we've all been using for decades.  But when it is used to simulate a hard disk, as is the case with SSD products, there are software layers which prevent it from reaching its full potential.  That explanation always gets heads nodding, because it is obvious.  But what about when the flash media is not simply packaged into an "SSD" and connected over SATA or SAS, but instead can be addressed via NVMe over PCIe?  Doesn't that make the problem of hard disk drive emulation go away?

Not entirely.  For one thing, in some cases the SSD abstraction is maintained despite being connected via PCIe.  Faster than SAS or SATA, but still those software speed bumps to keep data from driving too fast through the storage parking lot.

But let's suppose that you have a flash card designed specifically for NVMe and allowing more sophisticated memory-addressing software mechanisms to unleash its greater potential.  And what about the SCM (Storage Class Memory) products coming to market blurring the distinction between DRAM and non-volatile media previously relegated to the storage layer?  Hasn't hardware solved the performance problem?

I wish it were so.  But it comes back to the subtle distinction between latency and throughput, and what it means to the software inside the kernel.

At a high level, it's easy to think of latency as the inverse of throughput (and vice versa).  For a simple, single-threaded series of operations, that should be literally true.  But it is more complicated when you have many operations occurring in parallel.

Consider the analogy of a highway.  If it is a one-lane road. Latency is one vehicle making a round-trip between two end points. Let's imagine a 4-passenger car traveling on a single-lane road for 50 kilometers across a desert between two depots. Each round trip is an event and its completion time is its latency.  If we want to improve its latency we could make the car faster.

Latency - A Faster Car
If we want to improve its throughput we could make the car bigger - replace it with a huge, slow bus - but the net result would be worse latency with more passenger round trips for greater throughput.  We could further increase throughput by building more lanes on the highway and running more vehicles.  Optimally, we would make all the vehicles faster, whether they became even larger trains of trailers pulled by a truck or swarms of speedy motorcycles spreading out over the ever-multiplying number of new lanes.

Throughput - Many Jammed Lanes
The point here is that we really have three performance dimensions to consider:

  1. How quick is the round trip for any one passenger? That is latency.
  2. How many passengers in aggregate per unit time? That is throughput.
  3. How many independent events (vehicles)? That is accesses.

Enabling more accesses will be a natural consequence of lower latency, because the number of lanes is fixed (in the analogy) and the number of queues is finite (applying the analogy to software).

To optimize the use of emerging hardware technology we shouldn't merely rely on building more lanes in the highway (increasing throughput potential via flash media capability).  We should also be making the vehicles faster (improve latency).  Enhancing the hardware is in the hands of Intel, Micron, Samsung, and all the rest of the players in that space.

Simply making use of the bigger/faster/cheaper nonvolatile "flash" hardware components coming to market in wave after wave of impressive innovation is straightforward. Everybody in the storage industry is doing it.  And adding more of it is like adding lanes to the highway to get more throughput.

But doing something meaningful about latency is not easy.  It's hard.  It means rethinking the fundamentals, changing the innards of the kernel, ripping out cruft with both hands and designing new streamlined code to handle storage I/O for the 21st century.

More Stories By Amit Golander

Dr. Amit Golander is the Chief Technology Officer (CTO) and R&D Manager for Plexistor. His responsibilities are to develop the product and work with CEO, Sharon Azulai, on the vision for the technology and products.

Golander brings a rich research, development, and leadership background to Plexistor where he has distinguished himself in both the corporate, startup and higher educational realms.In addition to his work in the business and academic sectors, Golander holds over 50 patents and has published a number of technology articles in prestigious engineering journals.

Prior to Plexistor, Golander was VP of Systems and Product for Primary Data where he was responsible for strategic partnerships, alliances and beta customers as well as worked closely with the R&D teams on the day-to-day product management.Golander also worked for IBM for over twelve years on data center and cloud infrastructure.

Golander has also mentored M.Sc. students and taught computer architecture and quantitative analysis at Tel Aviv University.

Golander received his B.Sc. in C.S.and EE and his Ph.D. from Tel Aviv University in the field of computer architecture. His thesis won the Intel Research Award. Prior to his academic studies, Golander served as an intelligence officer in the Israeli Defense Force (IDF).

CloudEXPO Stories
The current age of digital transformation means that IT organizations must adapt their toolset to cover all digital experiences, beyond just the end users’. Today’s businesses can no longer focus solely on the digital interactions they manage with employees or customers; they must now contend with non-traditional factors. Whether it's the power of brand to make or break a company, the need to monitor across all locations 24/7, or the ability to proactively resolve issues, companies must adapt to the new world.
DXWorldEXPO LLC announced today that ICC-USA, a computer systems integrator and server manufacturing company focused on developing products and product appliances, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City. ICC is a computer systems integrator and server manufacturing company focused on developing products and product appliances to meet a wide range of computational needs for many industries. Their solutions provide benefits across many environments, such as datacenter deployment, HPC, workstations, storage networks and standalone server installations. ICC has been in business for over 23 years and their phenomenal range of clients include multinational corporations, universities, and small businesses.
René Bostic is the Technical VP of the IBM Cloud Unit in North America. Enjoying her career with IBM during the modern millennial technological era, she is an expert in cloud computing, DevOps and emerging cloud technologies such as Blockchain. Her strengths and core competencies include a proven record of accomplishments in consensus building at all levels to assess, plan, and implement enterprise and cloud computing solutions. René is a member of the Society of Women Engineers (SWE) and a member of the Society of Information Management (SIM) Atlanta Chapter. She received a Business and Economics degree with a minor in Computer Science from St. Andrews Presbyterian University (Laurinburg, North Carolina). She resides in metro-Atlanta (Georgia).
In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. His expertise is in automating deployment, management, and problem resolution in these environments, allowing his teams to run large transactional applications with high availability and the speed the consumer demands.
Everyone wants the rainbow - reduced IT costs, scalability, continuity, flexibility, manageability, and innovation. But in order to get to that collaboration rainbow, you need the cloud! In this presentation, we'll cover three areas: First - the rainbow of benefits from cloud collaboration. There are many different reasons why more and more companies and institutions are moving to the cloud. Benefits include: cost savings (reducing on-prem infrastructure, reducing data center foot print, reducing IT support costs), enabling growth (ensuring a highly available, highly scalable infrastructure), increasing employee access & engagement (by having collaboration tools that are usable and available globally regardless of location there will be an increased connectedness amongst teams and individuals that will help increase both efficiency and productivity.)