Welcome!

SDN Journal Authors: Liz McMillan, Pat Romanski, Elizabeth White, Yeshim Deniz, TJ Randall

Related Topics: SDN Journal, Containers Expo Blog, @CloudExpo

SDN Journal: Blog Post

Scaling with Distributed Pollers By @MJannery | @CloudExpo [#SDN #Cloud]

Not all network management solutions are alike, even though they may sound that way sometimes

Enterprise in Disguise - Scaling with Distributed Pollers

Not all network management solutions are alike, even though they may sound that way sometimes. A sure sign of a network management tool trying to pass itself off as an enterprise solution is when it implements distributed polling as a way to scale.

These systems scale server capacity by adding distributed pollers, each sharing a portion of the overall CPU load. But any network monitoring architect can tell you that the bottleneck in infrastructure management is I/O to the database. Having multiple pollers simultaneously send data back to a single data store does not solve the issue but can exacerbate it.

This is why network management systems that claim to scale using distributed polling engines can only achieve small increases for each engine—typically only 7,000 to 10,000 additional objects each versus up to 70,000 of an intelligently architected enterprise-capable system. Metaphorically speaking, distributed pollers do allow a network management application to pour more water in the top of the funnel, but the neck of the funnel is the problem.

Other risks include:

  • Single point of failure—If the central database fails, the ability of polling servers to collect data will be impacted.
  • WAN link failure—A failure of a WAN link between remote pollers and the central data store will cause loss of data.
  • Expensive WAN links—If data is sent to the central server over expensive and/or low capacity WAN links then pricy upgrades to these links may be needed.
  • Lack of real-time data—If the remote pollers simply gather data and forward it without real-time analysis, the benefits of immediate notification of anomalies are lost.

A true enterprise class solution distributes not only the polling but also the data storage. These multi-server solutions allow each server the visibility to the data stores of the other servers and therefore can scale infinitely. This is an architecture designed from its inception for enterprise computing.

More Stories By Michael Jannery

Michael Jannery is CEO of Entuity. He is responsible for setting the overall corporate strategy, vision, and direction for the company. He brings more than 30 years of experience to Entuity with 25 years in executive management.

Prior to Entuity, he was Vice President of Marketing for Proficiency, where he established the company as the thought, technology, and market leader in a new product lifecycle management (PLM) sub-market. Earlier, Michael held VP of Marketing positions at Gradient Technologies, where he established them as a market leader in the Internet security sector, and Cayenne Software, a leader in the software and database modeling market. He began his career in engineering.

CloudEXPO Stories
Organizations planning enterprise data center consolidation and modernization projects are faced with a challenging, costly reality. Requirements to deploy modern, cloud-native applications simultaneously with traditional client/server applications are almost impossible to achieve with hardware-centric enterprise infrastructure. Compute and network infrastructure are fast moving down a software-defined path, but storage has been a laggard. Until now.
Adding public cloud resources to an existing application can be a daunting process. The tools that you currently use to manage the software and hardware outside the cloud aren’t always the best tools to efficiently grow into the cloud. All of the major configuration management tools have cloud orchestration plugins that can be leveraged, but there are also cloud-native tools that can dramatically improve the efficiency of managing your application lifecycle. In his session at 18th Cloud Expo, Alex Lovell-Troy, Director of Solutions Engineering at Pythian, presented a roadmap that can be leveraged by any organization to plan, analyze, evaluate, and execute on moving from configuration management tools to cloud orchestration tools. He also addressed the three major cloud vendors as well as some tools that will work with any cloud.
Transformation Abstract Encryption and privacy in the cloud is a daunting yet essential task for both security practitioners and application developers, especially as applications continue moving to the cloud at an exponential rate. What are some best practices and processes for enterprises to follow that balance both security and ease of use requirements? What technologies are available to empower enterprises with code, data and key protection from cloud providers, system administrators, insiders, government compulsion, and network hackers? Join Ambuj Kumar (CEO, Fortanix) to discuss best practices and technologies for enterprises to securely transition to a multi-cloud hybrid world.
With the proliferation of both SQL and NoSQL databases, organizations can now target specific fit-for-purpose database tools for their different application needs regarding scalability, ease of use, ACID support, etc. Platform as a Service offerings make this even easier now, enabling developers to roll out their own database infrastructure in minutes with minimal management overhead. However, this same amount of flexibility also comes with the challenges of picking the right tool, on the right provider and with the proper expectations. In his session at 18th Cloud Expo, Christo Kutrovsky, a Principal Consultant at Pythian, compared the NoSQL and SQL offerings from AWS, Microsoft Azure and Google Cloud, their similarities, differences and use cases for each one based on client projects.
In his session at 21st Cloud Expo, Raju Shreewastava, founder of Big Data Trunk, provided a fun and simple way to introduce Machine Leaning to anyone and everyone. He solved a machine learning problem and demonstrated an easy way to be able to do machine learning without even coding. Raju Shreewastava is the founder of Big Data Trunk (www.BigDataTrunk.com), a Big Data Training and consulting firm with offices in the United States. He previously led the data warehouse/business intelligence and Big Data teams at Autodesk. He is a contributing author of book on Azure and Big Data published by SAMS.