Welcome!

SDN Journal Authors: TJ Randall, Yeshim Deniz, Liz McMillan, Elizabeth White, Pat Romanski

Related Topics: SDN Journal, Containers Expo Blog, @CloudExpo

SDN Journal: Blog Post

Scaling with Distributed Pollers By @MJannery | @CloudExpo [#SDN #Cloud]

Not all network management solutions are alike, even though they may sound that way sometimes

Enterprise in Disguise - Scaling with Distributed Pollers

Not all network management solutions are alike, even though they may sound that way sometimes. A sure sign of a network management tool trying to pass itself off as an enterprise solution is when it implements distributed polling as a way to scale.

These systems scale server capacity by adding distributed pollers, each sharing a portion of the overall CPU load. But any network monitoring architect can tell you that the bottleneck in infrastructure management is I/O to the database. Having multiple pollers simultaneously send data back to a single data store does not solve the issue but can exacerbate it.

This is why network management systems that claim to scale using distributed polling engines can only achieve small increases for each engine—typically only 7,000 to 10,000 additional objects each versus up to 70,000 of an intelligently architected enterprise-capable system. Metaphorically speaking, distributed pollers do allow a network management application to pour more water in the top of the funnel, but the neck of the funnel is the problem.

Other risks include:

  • Single point of failure—If the central database fails, the ability of polling servers to collect data will be impacted.
  • WAN link failure—A failure of a WAN link between remote pollers and the central data store will cause loss of data.
  • Expensive WAN links—If data is sent to the central server over expensive and/or low capacity WAN links then pricy upgrades to these links may be needed.
  • Lack of real-time data—If the remote pollers simply gather data and forward it without real-time analysis, the benefits of immediate notification of anomalies are lost.

A true enterprise class solution distributes not only the polling but also the data storage. These multi-server solutions allow each server the visibility to the data stores of the other servers and therefore can scale infinitely. This is an architecture designed from its inception for enterprise computing.

More Stories By Michael Jannery

Michael Jannery is CEO of Entuity. He is responsible for setting the overall corporate strategy, vision, and direction for the company. He brings more than 30 years of experience to Entuity with 25 years in executive management.

Prior to Entuity, he was Vice President of Marketing for Proficiency, where he established the company as the thought, technology, and market leader in a new product lifecycle management (PLM) sub-market. Earlier, Michael held VP of Marketing positions at Gradient Technologies, where he established them as a market leader in the Internet security sector, and Cayenne Software, a leader in the software and database modeling market. He began his career in engineering.

CloudEXPO Stories
DevOps is a world surrounded by information, starting from a single commit and ending in roll out to production. In this talk, I'll introduce you to the world of Taboola DevOps data collection, to better understand what goes on under the hood. The system we've developed in-house helps us collect and analyse the entire DevOps process from the very first commit all the way to production. It provides us a full clear view with a drill-down toolset that helps keep us away from the dark side. Our KPI's moved from being abstracted ideas to data driven goals, which we can measure and act upon. We're living in a data driven world when all business components are based on our clients action and reaction, why not doing the same thing within our DevOps eco-system?
After years of investments and acquisitions, CloudBlue was created with the goal of building the world's only hyperscale digital platform with an increasingly infinite ecosystem and proven go-to-market services. The result? An unmatched platform that helps customers streamline cloud operations, save time and money, and revolutionize their businesses overnight. Today, the platform operates in more than 45 countries and powers more than 200 of the world's largest cloud marketplaces, managing more than 27 million enterprise cloud subscriptions valued at more than $1 billion in revenue.
In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, will discuss how to use Kubernetes to setup a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. His expertise is in automating deployment, management, and problem resolution in these environments, allowing his teams to run large transactional applications with high availability and the speed the consumer demands.
Containerized software is riding a wave of growth, according to latest RightScale survey. At Sematext we see this growth trend via our Docker monitoring adoption and via Sematext Docker Agent popularity on Docker Hub, where it crossed 1M+ pulls line. This rapid rise of containers now makes Docker the top DevOps tool among those included in RightScale survey. Overall Docker adoption surged to 35 percent, while Kubernetes adoption doubled, going from 7% in 2016 to 14% percent.
In an age of borderless networks, security for the cloud and security for the corporate network can no longer be separated. Security teams are now presented with the challenge of monitoring and controlling access to these cloud environments, as they represent yet another frontier for cyber-attacks. Complete visibility has never been more important-or more difficult. Powered by AI, Darktrace's Enterprise Immune System technology is the only solution to offer real-time visibility and insight into all parts of a network, regardless of its configuration. By learning a ‘pattern of life' for all networks, devices, and users, Darktrace can detect threats as they arise and autonomously respond in real time - all without impacting server performance.