SDN Journal Authors: Pat Romanski, Elizabeth White, Yeshim Deniz, Destiny Bertucci, Liz McMillan

Related Topics: SDN Journal, Microservices Expo, Containers Expo Blog, @CloudExpo, Cloud Security, @DXWorldExpo

SDN Journal: Article

Software-Automated Testing for Software-Defined Networks

You should already be deploying software-driven testing processes, driven by a test and lab automation framework

Software-defined networks are all the rage these days - and why not? They offer the promised benefit of making networks far more agile and responsive to dynamically changing application requirements. Consequently, in theory, the entire networking-computing-storage-applications ecosystem of IT and telecom will dance nimbly to the contrapuntal rhythms of the top and bottom line. And let's not forget that there will be big-time winners and losers in the networking technology industry, making not only a fascinating business issue, but something of a blood sport as well, and who doesn't like to watch competitors bash each other silly?

Amid all the hype and hyperbole, there remains plenty of gritty detail in building, deploying, and maintaining networks, software-defined or not. One immensely important detail is making sure all this new-fangled networking will work, and that means testing. Networks may become software driven, but unless we find another dimension within which to switch communications data around, there will still be hardware. The vendor logo on the faceplate of that hardware may change over time, but hardware will still be humming away in millions of racks in thousands of data centers, central offices, points of presence, and telco closets. Inside those millions upon millions of pieces of hardware will be billions of virtual resources - hypervisors, virtual machines, virtual switches, and trillions of application instances. All of that, hardware included, has to be tested. Today, many software functions are collocated within the hardware - for example, the computation of network paths has been delivered within the switch or router.

The new frontier of SDN makes the picture more complex. Separate network components perform control-plane path calculations (now network-critical control plane functions) based on requests from applications and then propagate those paths to data-plane switches. From a testing point of view, functions that formerly ran in the software of a switch or router are now network applications, requiring testing against a matrix of network topologies and conditions. If the control plane breaks, the entire network will break catastrophically. If that doesn't qualify as something that needs extremely thorough testing, then maybe nothing does.

Let's consider what is involved in testing a service provider network today by taking the simplest case - adding a new piece of hardware or a software upgrade to something in the network. There is a huge, nay, massive matrix of hardware components, subcomponents, OS versions, firmware versions, configurations, etc., that are deployed in the network. Each hardware component amortizes over a very long time so they're not going away anytime soon. Consequently, each new bit must be tested against all the bits that are already deployed. This is a very large, time-consuming and expensive undertaking involving all the representative equipment sitting in a lab to do the job. Know of any service providers that only run one service? All those different services with all their features need to be tested, at different loads, etc.

Ironically, network testing is probably one of the least "software-enabled" functions in IT today, making testing more complex and less efficient. Imagine a large business that operated using manual phone switchboards for telephony, its messaging via sticky notes, and with no calendaring system. That doesn't exist in the front office of any business of scale today, but such a primitive state of IT automation is very common in large test labs, despite millions of dollars of capital, space and power being poured into labs annually. Very costly devices under test (DUTs) and test generators are often connected together with a spaghetti of cables and manual patch panels. Testing teams coordinate their usage of equipment through "hands off" post-it notes placed on equipment that is in use, and through emails. Visibility to and calendaring of resources is typically non-existent as well - it's all handled in a very manual fashion.

The lack of automation increases the time it takes to connect test equipment together and to configure the test devices so that they're ready for testing. Sorting through a tangle of cables and physically connecting the various test components into a testing topology can take hours or even days. A test engineer may take additional hours or days to load the right OS or firmware versions and configure the devices in a large test bed topology. Since nobody wants to make a mistake during setup and configuration, equipment is often kept locked down and powered up during that entire period. And that's when the new features arrive on time. If the new gear to be tested is delayed, given the onerous time investment just to prepare for the test, engineers often will simply keep test topologies locked up to avoid the risk of losing access to a critical test component or having their configuration messed up. The not-so-shocking result is that the ratio of time spent in setup and configuration vs. actual testing can be as lopsided as 80% vs. 20%. Ever hear of "process re-engineering?" This is a business process begging to be optimized. To recap, software-defined networks will add complexity to an already daunting testing task against a massive matrix of legacy and continually added new hardware and virtual network components. Most network test labs aren't software-driven today. What would it look like to software-enable network testing? Enter test and lab automation solutions, which are ideally flexible frameworks that can take these massive CAPEX and OPEX investments from the dark ages into the bright, post-modern light of efficiency and savings. Test and lab automation software does this by changing the way that test labs operate:

  • Connecting to, controlling and inventorying all the equipment in the lab, including DUTs, test devices L1 switches, and even power distribution units. All lab personnel now find gear and build their test topologies through a GUI rather than by manually hunting through telecom racks.
  • Enabling testers to build and schedule the precise resources needed in a test topology, including the design and activation of their connectivity, provisioning of proper OS and firmware versions onto devices, and configuration of logical parameters and features. This advanced scheduling and automated provisioning minimizes both physical and logical setup time, speeds test execution, improves sharing of resources, and enables test labs to deliver more test coverage in less time.
  • Automated administrative functions such as the ability to save and restore an entire test topology once it is designed, or to restore devices to a baseline state, cuts down on repeated design work, and prevents errors from tests run on the wrong device configurations
  • Tracking scheduling and activation indicators of device utilization and test completion provide vital business intelligence reports to help managers and architects continue to refine best practices and continuously improve efficiency.

What's the take away? If you're considering deploying software-defined networks, you should already be deploying software-driven testing processes, driven by a test and lab automation framework. To attempt to roll out such complex new technology without proper automation could lead to experiencing some hard realities.

More Stories By Alex Henthorn-Iwane

Alex Henthorn-Iwane joined QualiSystems in February 2013 and is responsible for worldwide marketing and public relations. Prior to joining QualiSystems, he was Vice-President of Marketing and Product Management at Packet Design, Inc., a provider of IP/MPLS route analytics network management software, and has 20+ years of experience in senior management, marketing and technical roles at telecom-oriented networking and security startups. Alex holds a BA from U.C. Berkeley.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

@CloudExpo Stories
The best way to leverage your Cloud Expo presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering Cloud Expo and @ThingsExpo will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at Cloud Expo. Product announcements during our show provide your company with the most reach through our targeted audiences.
DevOpsSummit New York 2018, colocated with CloudEXPO | DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City. Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of bus...
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, @CloudEXPO and DXWorldEXPO are two of the most important technology events of the year. Since its launch over eight years ago, @CloudEXPO and DXWorldEXPO have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, we provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading...
As you move to the cloud, your network should be efficient, secure, and easy to manage. An enterprise adopting a hybrid or public cloud needs systems and tools that provide: Agility: ability to deliver applications and services faster, even in complex hybrid environments Easier manageability: enable reliable connectivity with complete oversight as the data center network evolves Greater efficiency: eliminate wasted effort while reducing errors and optimize asset utilization Security: implemen...
DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
DXWorldEXPO | CloudEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
@DevOpsSummit New York 2018, colocated with CloudEXPO | DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises - and delivering real results.
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
DXWorldEXPO LLC announced today that Dez Blanchfield joined the faculty of CloudEXPO's "10-Year Anniversary Event" which will take place on November 11-13, 2018 in New York City. Dez is a strategic leader in business and digital transformation with 25 years of experience in the IT and telecommunications industries developing strategies and implementing business initiatives. He has a breadth of expertise spanning technologies such as cloud computing, big data and analytics, cognitive computing, m...
Digital Transformation and Disruption, Amazon Style - What You Can Learn. Chris Kocher is a co-founder of Grey Heron, a management and strategic marketing consulting firm. He has 25+ years in both strategic and hands-on operating experience helping executives and investors build revenues and shareholder value. He has consulted with over 130 companies on innovating with new business models, product strategies and monetization. Chris has held management positions at HP and Symantec in addition to ...
Cloud-enabled transformation has evolved from cost saving measure to business innovation strategy -- one that combines the cloud with cognitive capabilities to drive market disruption. Learn how you can achieve the insight and agility you need to gain a competitive advantage. Industry-acclaimed CTO and cloud expert, Shankar Kalyana presents. Only the most exceptional IBMers are appointed with the rare distinction of IBM Fellow, the highest technical honor in the company. Shankar has also receive...
DXWorldEXPO LLC announced today that Kevin Jackson joined the faculty of CloudEXPO's "10-Year Anniversary Event" which will take place on November 11-13, 2018 in New York City. Kevin L. Jackson is a globally recognized cloud computing expert and Founder/Author of the award winning "Cloud Musings" blog. Mr. Jackson has also been recognized as a "Top 100 Cybersecurity Influencer and Brand" by Onalytica (2015), a Huffington Post "Top 100 Cloud Computing Experts on Twitter" (2013) and a "Top 50 C...
There is a huge demand for responsive, real-time mobile and web experiences, but current architectural patterns do not easily accommodate applications that respond to events in real time. Common solutions using message queues or HTTP long-polling quickly lead to resiliency, scalability and development velocity challenges. In his session at 21st Cloud Expo, Ryland Degnan, a Senior Software Engineer on the Netflix Edge Platform team, will discuss how by leveraging a reactive stream-based protocol,...
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities - ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups.
Daniel Jones is CTO of EngineerBetter, helping enterprises deliver value faster. Previously he was an IT consultant, indie video games developer, head of web development in the finance sector, and an award-winning martial artist. Continuous Delivery makes it possible to exploit findings of cognitive psychology and neuroscience to increase the productivity and happiness of our teams.
Poor data quality and analytics drive down business value. In fact, Gartner estimated that the average financial impact of poor data quality on organizations is $9.7 million per year. But bad data is much more than a cost center. By eroding trust in information, analytics and the business decisions based on these, it is a serious impediment to digital transformation.
The standardization of container runtimes and images has sparked the creation of an almost overwhelming number of new open source projects that build on and otherwise work with these specifications. Of course, there's Kubernetes, which orchestrates and manages collections of containers. It was one of the first and best-known examples of projects that make containers truly useful for production use. However, more recently, the container ecosystem has truly exploded. A service mesh like Istio addr...
Predicting the future has never been more challenging - not because of the lack of data but because of the flood of ungoverned and risk laden information. Microsoft states that 2.5 exabytes of data are created every day. Expectations and reliance on data are being pushed to the limits, as demands around hybrid options continue to grow.
Business professionals no longer wonder if they'll migrate to the cloud; it's now a matter of when. The cloud environment has proved to be a major force in transitioning to an agile business model that enables quick decisions and fast implementation that solidify customer relationships. And when the cloud is combined with the power of cognitive computing, it drives innovation and transformation that achieves astounding competitive advantage.
Digital Transformation: Preparing Cloud & IoT Security for the Age of Artificial Intelligence. As automation and artificial intelligence (AI) power solution development and delivery, many businesses need to build backend cloud capabilities. Well-poised organizations, marketing smart devices with AI and BlockChain capabilities prepare to refine compliance and regulatory capabilities in 2018. Volumes of health, financial, technical and privacy data, along with tightening compliance requirements by...