SDN Journal Authors: Elizabeth White, Yeshim Deniz, Liz McMillan, Pat Romanski, TJ Randall

Related Topics: @CloudExpo, SDN Journal

@CloudExpo: Blog Post

Training Wheels and Protective Gear By @PlexxiInc | @CloudExpo [#SDN]

This balancing act is part of what as made networking as complex as it has become

Throughout the development cycle of new features and functions for any network platform (or probably most other products not targeted at the mass market consumer) this one question will always come up: should we protect the user of our product from doing this? And “this” is always something that would allow the user of the product to really mess things up if not done right. As a product management organization you almost have to take a philosophical stand when it comes to these questions.

Protect the user
Sure enough, the question came up last week as part of the development of one our features. When putting the finishing touches on a feature that allows very direct control over some of the fundamental portions of what creates a Plexxi fabric, our QA team (very appropriately) raised the concern: if the user does this, bad things can happen, should we not allow the user to change this portion of the feature?

This balancing act is part of what as made networking as complex as it has become. As an industry we have been extremely flexible in what we have exposed to our users. We have given access to portions of our products that 99.9% of customers will never need, but unfortunately because of that 0.1% every networking product has tons of these little tweaks and knobs that could wreak havoc if used the wrong way.

We take a lot of pride in creating a network solution that is simple to use, simple to interact with, but extremely powerful under the hood. Direct access to all that power will lead to not only giving the customer a powerful weapon, but also the ammunition to use it. And like handling any weapon, you can really hurt yourself if you are not careful. Which comes back to the question at hand, how many safety valves do you put in place to make sure the user cannot hurt themselves?

The reason why
Some of these controls are buried fairly deep inside our products. They are meant for true power users and for the support teams of the vendors. And even beyond the support teams, there are tools and tricks inside our products that only the engineering teams know about, hidden even beyond the knowledge of support teams. Several years ago (in a previous job), we had a customer with a complex problem. Traffic was inconsistently forwarded and the belief was that there were communication problems between line cards and the main CPU card that would create inconsistant tables (the biggest challenge for any chassis based system).

Of course our development teams had tools embedded in the code to carefully examine and manipulate the tables and communications between these cards. Not exposed to a regular user, because they were potentially dangerous. And we proved that they were. During the execution of the command by one of my developers, he made a small typo in one of the arguments and boom went the switch. Crash and reboot. Customer very upset (for good reason, this was a production network), executive management very upset (also for good reason) and worse, the problem disappeared without us collecting the information we needed to attempt to fix it.

Different Answers for Different Tools
There is a difference between debug tools that allow engineers to look deep inside the switch versus common features that may have significant service consequences if not in expert hands. No matter how hard we try, the first category will continue to exist. As vendors we will bring portions of these tools to the user or support visible spectrum, but at the same time we will create new ones buried deep.

The latter category though is one where I favor a less protective approach. There are many ways by which you can completely disrupt your network service. Most of the services your network provide have been created with your own hands through provisioning and configuration and can therefore be disrupted by those same actions. When we create features and functions that are potentially dangerous, it is on us the vendor to make sure it is properly documented and explained. This way when you do make that mistake (and it will happen) we can refer to that 4 letter “read the documentation” response.

Off come the training wheels
When it comes to user configurable features and functions, every single one of them has the potential to disrupt service when used the wrong way. We as vendors should not shy away from giving you all the tools you need to create (and destroy) the service you need. And I do not believe anyone wants to step through one “Are you sure (Y/N)?” after another. Of course we need to make creating services easier for you. If you are a frequent reader of our blogs you know that is what we stand for. But we should not take away features because we are afraid you can shoot yourself in the foot. Any time in the past where we opted to give you a gun but keep the bullets behind a locked door, we have found someone that legitimately explained that he or she needed the bullets to solve their specific problem. And we unlocked the door.

There are ways to teach someone how to ride a bike without providing permanent training wheels. Documentation (for those few that read it), workflow based provisioning and configuration and solid default behaviors with predictable results can steer you clear of the dangers we have provided. And when you do fall off the bike and hurt your knee or elbow, well, you are less likely to try that maneuver again next time. That is how most of us learn. Including those developers that crash a customer production switch during a debug session. For every one of those “oops” moments there will be many where those hidden gems may have saved your network from disaster. Just like there is one customer for whom having the bullets makes the difference between a working service and one that just limps along.

[Today's fun fact: You burn more calories sleeping than watching TV. I enjoy combining the two, especially during some of the last few Thursday night NFL games.]

The post Training Wheels and Protective Gear appeared first on Plexxi.

Read the original blog entry...

More Stories By Marten Terpstra

Marten Terpstra is a Product Management Director at Plexxi Inc. Marten has extensive knowledge of the architecture, design, deployment and management of enterprise and carrier networks.

CloudEXPO Stories
DXWorldEXPO LLC announced today that Big Data Federation to Exhibit at the 22nd International CloudEXPO, colocated with DevOpsSUMMIT and DXWorldEXPO, November 12-13, 2018 in New York City. Big Data Federation, Inc. develops and applies artificial intelligence to predict financial and economic events that matter. The company uncovers patterns and precise drivers of performance and outcomes with the aid of machine-learning algorithms, big data, and fundamental analysis. Their products are deployed by some of the world's largest financial institutions. The company develops and applies innovative machine-learning technologies to big data to predict financial, economic, and world events. The team is a group of passionate technologists, mathematicians, data scientists and programmers in Silicon Valley with over 100 patents to their names. Big Data Federation was incorporated in 2015 and is ...
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more business becomes digital the more stakeholders are interested in this data including how it relates to business. Some of these people have never used a monitoring tool before. They have a question on their mind like "How is my application doing" but no idea how to get a proper answer.
All in Mobile is a place where we continually maximize their impact by fostering understanding, empathy, insights, creativity and joy. They believe that a truly useful and desirable mobile app doesn't need the brightest idea or the most advanced technology. A great product begins with understanding people. It's easy to think that customers will love your app, but can you justify it? They make sure your final app is something that users truly want and need. The only way to do this is by researching target group and involving users in the designing process.
CloudEXPO New York 2018, colocated with DevOpsSUMMIT and DXWorldEXPO New York 2018 will be held November 12-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI and Machine Learning to one location.
CloudEXPO | DevOpsSUMMIT | DXWorldEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.