Welcome!

SDN Journal Authors: Destiny Bertucci, Liz McMillan, Pat Romanski, Elizabeth White, Amitabh Sinha

Related Topics: @CloudExpo, Microservices Expo, @DXWorldExpo, SDN Journal, @ThingsExpo

@CloudExpo: Blog Post

How to Avoid the #BigData Black Hole | @BigDataExpo [#IoT]

It's hard to avoid the world of opportunities that has been opened in data collection and analysis

It takes a lot of will power, in our data obsessed world to say "too much!" However, there are many ways where too much information is destroying productivity, and actually causing bad decision making, not good. But it is hard to avoid the world of opportunities that has been opened in data collection and analysis. So how do you balance the two? The first step is to understand there is a big difference between data collection, and it's utilization. While it seems subtle, the difference is key, and utilization is where many make mistakes.

Many confuse data collection and data utilization as the same thing, or at least being very similar.
Collection merely indicates the storage and analysis of data by automated systems, maybe never viewed by human eye or mind. It is when you spend the time to access and understand the data collected where there is a human cost.

"When information is cheap, attention becomes expensive." - James Gleick

New tools mask human effort by taking raw data (in hard to consume formats) and making it much easier to understand, and more importantly; query upon. This makes users feel like they need to look at it all, and the UI is saving them all the time they need. But, in most cases, we did not previously collect this information, nor analyze it. So the opportunity is higher, but any effort is net new.

big-data-blackhole

When I get in front of a great log management tool, I sometimes get that same feeling as when I get addicted to a new iPad game; crushing candies, or tending to farms. Because there is a sort of gamification when you dig, and if I just dig a little deeper I will reach the goal. Next thing you know, you have spent three hours without any results.

"Information is not knowledge and knowledge is not wisdom." - James Gleick,

It is a people problem too
There is effort required to first understand, and then act on the information we receive. But the enthusiasm for learning more, and "information is power" creates a sort of blindness. A blindness that skirts the need to quantify our effort spent in analyzing the data against the results (decisions, actions items) that from it. To make things worse, the amount of data it takes for one decision is often way more than we realize.

So, what is the impact of spending too much time trying to utilize the new pile of information?

  • Time suck: Being taken away from daily tasks, by asking a question of the platform that takes way more time than expected.
  • Dwelling on details that do not impact the business: Leading to zero actionable results.
  • The anchoring effect: If you are familiar with the psychological anchoring effect, where new decisions are based on an existing foundation of positions, and opinions. With all the data in your log analysis platform, if you are not aware of anchoring, you will the ask questions that produce the answers you seek.
  • "That is what the numbers say ...." Platforms that only produce time-series dashboards and numbers lead to the habit of using this data as an default answer to all questions, and deflecting mechanism. It's easy to put the decisions on the system, instead of the people who were previously responsible for them.

These are not light topics I know, but I have seen time and time again where the abuse of data, and the time wasted on collecting it eats organizations from within.

Does the tool save your time or steal your eyeballs?
Part of this whole situation has a lot to do with the tools. Tools can be good or bad at surfacing information. All the latest APM, Log Analysis, Error Logging, and usage analytics tools have ways for you to dig-in deep. Putting the burden of when enough is enough on the user. Who often is too absorbed to keep track.

In fact, many of these tools encourage you to go deep, eventually spending hours sifting thought information that provides no real value real, but perhaps a lot of little "oh that is interesting". Those little hits are just like pulling the lever on a casino machine. If you have to become specialized to use the platform, and make it a large portion of your daily effort, you are committed. After all, churn is a major concern for tech companies. But should it be?

"Business is not just doing deals; business is having great products, doing great engineering, and providing tremendous service to customers." - Ross Perot

A great tool should help you get the data, make decisions (or prepare them for the expanded team), and move on.

It should prevent you from entering the data black hole, and guide you to what is most useful. The tool should also put a layer between you and the data, not to encourage the diving-in too deep which we know can happen quickly.

Here are some ways to avoid these problems:

  • Hire a dedicated analysis person: Definitely a nice-to-have. But really, this is just a workaround for a bigger problem.
  • Ask discrete questions: If you keep your questions simple and small, then you can move faster. But we all know that there are bigger questions we want answered.
  • Plan your logs in advance for utilization, not collection: If you think about how you want to use the information, instead of just gathering data, you will make the analysis go much faster. In a crunch, this seems like a wasted effort.
  • Focusing: There are a lot of "Fun" questions you can ask. But time is better spent on questions decided-on in advance and related to current initiatives and activity. At the same time, wouldn't it be nice for the data to tell you what is interesting without you having to predict it in advance?
  • FIND THE RIGHT TOOL! Many analysis tools only help "send information and put in a nice format ", aka data collection. While this is a must, the focus of the tool should be on utilization. And this should be their roadmap focus.

Find it, act on it, move on
Part of creating this value layer is discovering what is happening, but also what is not happening, and what is out of normal. Watching the flow of data can be memorizing, but decisions are made on things that are not consistent with the steady state of systems and applications. Or with sudden changes. Things that normally occur in the system, but for some recently did not.

Data collection should be synthesized into meaningful events. Getting users addicted to a platform by the quality and frequency of decisions versus encouraging them to spin the wheel to see what happens and becoming a 5th limb.

First, admit to yourself that there is too much information, more than you could every analyze on your own. And then find a tool that takes over analysis, and only shows you what is valuable so that you and your team can get on with your day.

Conference Schedule Announced

Are you ready to put your data in the cloud?

What is the future of security in the cloud?

Does Docker quickly advance the development of an IoT application?

What are the implications of Moore's Law on Hadoop deployments?

Get all these questions and hundreds more like them answered at the 15th Cloud Expo, November 4-6, 2014, at the Santa Clara Convention Center, in Santa Clara, CA. The Cloud Expo / Big Data Expo / @ThingsExpo / DevOps Summit programs are now available for you to inspect and investigate in advance.

Our upcoming November 4-6 event in Santa Clara, California will present a total of 10 simultaneous tracks by an all-star faculty, over three days, plus a two-day "Cloud Computing Bootcamp" presented by Janakiram MSV, an Analyst with the Gigaom Research analyst network, where he covers the Cloud Services landscape.

Cloud and Big Data topics and tracks include: Enterprise Cloud Adoption, APM & Cloud Computing | Hot Topics, Cloud APIs & Business, Cloud Security | Mobility, Big Data | Analytics.

@ThingsExpo content tripled from a single track in New York to three simultaneous tracks: Consumer IoT, Enterprise IoT, IoT Developer | WebRTC Convergence.

DevOps Summit also doubled from a single track in New York to two simultaneous tracks: "Dev" Developer Focus and "Ops" Operations Focus.

Schedule for Cloud Expo / Big Data Expo / @ThingsExpoHere

Schedule for DevOps SummitHere

Now that we have published the full conference schedule, please check back for daily updates as we finalize new session abstracts by working with our distinguished faculty members. For your questions please contact us at events (at) sys-con.com. Last but not least we will announce our keynotes on the hottest subjects to be delivered by world-class speakers!

The largest 'Internet of Things' event in the world has announced "sponsorship opportunities" and "call for papers."

The 1st International Internet of @ThingsExpo was launched this June at the Javits Center in New York City with over 6,000 delegates in attendance. The 2nd International Internet of @ThingsExpo will take place November 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, California, with an estimated 7,000 plus delegates attending over three days.

@ThingsExpo is co-located with 15th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading IoT industry players in the world. In 2014, more than 200 companies will be present on the @ThingsExpo show floor, including global players and the hottest new technology pioneers.

Sponsorship and Exhibit Opportunities for @ThingsExpo Silicon Valley and New York Are Now Available
Sponsors and exhibitors of Internet of @ThingsExpo will benefit from unmatched branding, profile building and lead generation opportunities through:

  • Featured on-site presentation and ongoing on-demand webcast exposure to a captive audience of industry decision-makers.
  • Showcase exhibition during our new extended dedicated expo hours
  • Breakout Session Priority scheduling for sponsors that have been guaranteed a 35-minute technical session
  • Online advertising in SYS-CON's i-Technology publications
  • Capitalize on our comprehensive marketing efforts leading up to the show with print mailings, e-newsletters and extensive online media coverage.
  • Unprecedented PR Coverage: Editorial coverage on IoT.sys-con.com, Tweets to our 75,000 plus followers, press releases sent on major wire services to over 500 combined analysts and press members who attended Internet of @ThingsExpo - making it the best-covered "Internet of Things" conference in the world

For more information on sponsorship, exhibit, and keynote opportunities contact Carmen Gonzalez by email at events (at) sys-con.com, or by phone 201 802-3021. Book both events for additional savings!

@ThingsExpo Silicon Valley (November 4-6, 2014, Santa Clara, CA)
@ThingsExpo New York (June 9-11, 2015, New York, NY)

Secure Your VIP Pass to Attend @ThingsExpo Silicon Valley
Internet of @ThingsExpo announced today a limited time free "Expo Plus" registration option. The onsite registration price of $600 will be set to 'free' for delegates who register during September.

To take advantage of this opportunity, attendees can use the coupon code "IoTSeptember" and secure their "@ThingsExpo Plus" registration to attend all keynotes and general sessions, as well as a limited number of technical sessions each day of the show, in addition to full access to the expo floor and the @ThingsExpo hackathon.

The registration page is located at the @ThingsExpo site here.


@ThingsExpo New York 2015 'Call for Papers' Now Open
The 3rd International Internet of @ThingsExpo, to be held June 9-11, 2015, at the Javits Center in New York City, New York announces that its 'Call for Papers' is now open. The event will feature a world class, all-star faculty with the hottest IoT topics covered in three distinct tracks.

Track 1 - Consumer IoT and Wearables: Smart Appliances, Wearables, Smart Cars, Smartphones 2.0, Smart Travel, Personal Fitness, Health Care, Personalized Marketing, Customized Shopping, Personal Finance, The Digital Divide, Mobile Cash & Markets, Games & the IoT, The Future of Education, Virtual Reality

Track 2 - Enterprise IoT: The Business Case for IoT, Smart Grids, Smart Cities, Smart Transportation, The Smart Home, M2M, Authentication/Security, Wiring the IoT, The Internet of Everything, Digital Transformation of Enterprise IT, Agriculture, Transportation, Manufacturing, Local & State Government, Federal Government

Track 3 - Developer IoT: WebRTC, Eclipse Foundation, Cloud Foundry, Docker & Linux Containers, Node-Red, Open Source Hardware, Leveraging SOA, Multi-Cloud IoT, Evolving Standards, WebSockets, Security & Privacy Protocols, GPS & Proximity Services, Bluetooth/RFID/etc., XMPP, Nest Labs


@ThingsExpo billboard is viewed by more than 1.3 million motorists per week on Highway 101, in the heart of Silicon Valley

Help plant your flag in the fast-expanding business opportunity that is the Internet of Things: Submit your speaking proposal today here!

Download @ThingsExpo Newsletter Today Here

Chris Matthieu Named @ThingsExpo Tech Chair

Internet of @ThingsExpo named Chris Matthieu tech chair of Internet of @ThingsExpo 2014 Silicon Valley.

Chris Matthieu has two decades of telecom and web experience. He launched his Teleku cloud communications-as-a-service platform at eComm in 2010, which was acquired by Voxeo. Next he built an open source Node.JS PaaS called Nodester, which was acquired by AppFog. His latest startups include Twelephone. Leveraging HTML5 and WebRTC, Twelephone's BHAG (Big Hairy Audacious Goal) is to become the next generation telecom company running in the Web browser. Chris is currently co-founder and CTO of Octoblu.

Website: http://www.ThingsExpo.com

Twitter: http://www.Twitter.com/ThingsExpo

About SYS-CON Media & Events
SYS-CON Media (www.sys-con.com) has since 1994 been connecting technology companies and customers through a comprehensive content stream - featuring over forty focused subject areas, from Cloud Computing to Web Security - interwoven with market-leading full-scale conferences produced by SYS-CON Events. The company's internationally recognized brands include among others Cloud Expo® (CloudComputingExpo.com / @CloudExpo), Big Data Expo (BigDataExpo.net / @BigDataExpo), DevOps Summit (DevOpsSummit.sys-con.com / @DevOpsSummit), Internet of @ThingsExpo (ThingsExpo.com / @ThingsExpo) and Cloud Computing Bootcamp (CloudComputingBootcamp.com).

Cloud Expo® and Big Data Expo® are registered trademarks of Cloud Expo, Inc., a SYS-CON Events company.

More Stories By Trevor Parsons

Trevor Parsons is Chief Scientist and Co-founder of Logentries. Trevor has over 10 years experience in enterprise software and, in particular, has specialized in developing enterprise monitoring and performance tools for distributed systems. He is also a research fellow at the Performance Engineering Lab Research Group and was formerly a Scientist at the IBM Center for Advanced Studies. Trevor holds a PhD from University College Dublin, Ireland.

@CloudExpo Stories
Digital transformation is about embracing digital technologies into a company's culture to better connect with its customers, automate processes, create better tools, enter new markets, etc. Such a transformation requires continuous orchestration across teams and an environment based on open collaboration and daily experiments. In his session at 21st Cloud Expo, Alex Casalboni, Technical (Cloud) Evangelist at Cloud Academy, explored and discussed the most urgent unsolved challenges to achieve f...
The dynamic nature of the cloud means that change is a constant when it comes to modern cloud-based infrastructure. Delivering modern applications to end users, therefore, is a constantly shifting challenge. Delivery automation helps IT Ops teams ensure that apps are providing an optimal end user experience over hybrid-cloud and multi-cloud environments, no matter what the current state of the infrastructure is. To employ a delivery automation strategy that reflects your business rules, making r...
The 22nd International Cloud Expo | 1st DXWorld Expo has announced that its Call for Papers is open. Cloud Expo | DXWorld Expo, to be held June 5-7, 2018, at the Javits Center in New York, NY, brings together Cloud Computing, Digital Transformation, Big Data, Internet of Things, DevOps, Machine Learning and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding busin...
In a recent survey, Sumo Logic surveyed 1,500 customers who employ cloud services such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). According to the survey, a quarter of the respondents have already deployed Docker containers and nearly as many (23 percent) are employing the AWS Lambda serverless computing framework. It’s clear: serverless is here to stay. The adoption does come with some needed changes, within both application development and operations. Tha...
SYS-CON Events announced today that Synametrics Technologies will exhibit at SYS-CON's 22nd International Cloud Expo®, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. Synametrics Technologies is a privately held company based in Plainsboro, New Jersey that has been providing solutions for the developer community since 1997. Based on the success of its initial product offerings such as WinSQL, Xeams, SynaMan and Syncrify, Synametrics continues to create and hone in...
Smart cities have the potential to change our lives at so many levels for citizens: less pollution, reduced parking obstacles, better health, education and more energy savings. Real-time data streaming and the Internet of Things (IoT) possess the power to turn this vision into a reality. However, most organizations today are building their data infrastructure to focus solely on addressing immediate business needs vs. a platform capable of quickly adapting emerging technologies to address future ...
In his general session at 21st Cloud Expo, Greg Dumas, Calligo’s Vice President and G.M. of US operations, discussed the new Global Data Protection Regulation and how Calligo can help business stay compliant in digitally globalized world. Greg Dumas is Calligo's Vice President and G.M. of US operations. Calligo is an established service provider that provides an innovative platform for trusted cloud solutions. Calligo’s customers are typically most concerned about GDPR compliance, application p...
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
Kubernetes is an open source system for automating deployment, scaling, and management of containerized applications. Kubernetes was originally built by Google, leveraging years of experience with managing container workloads, and is now a Cloud Native Compute Foundation (CNCF) project. Kubernetes has been widely adopted by the community, supported on all major public and private cloud providers, and is gaining rapid adoption in enterprises. However, Kubernetes may seem intimidating and complex ...
In his session at 21st Cloud Expo, Michael Burley, a Senior Business Development Executive in IT Services at NetApp, described how NetApp designed a three-year program of work to migrate 25PB of a major telco's enterprise data to a new STaaS platform, and then secured a long-term contract to manage and operate the platform. This significant program blended the best of NetApp’s solutions and services capabilities to enable this telco’s successful adoption of private cloud storage and launching ...
The past few years have brought a sea change in the way applications are architected, developed, and consumed—increasing both the complexity of testing and the business impact of software failures. How can software testing professionals keep pace with modern application delivery, given the trends that impact both architectures (cloud, microservices, and APIs) and processes (DevOps, agile, and continuous delivery)? This is where continuous testing comes in. D
You know you need the cloud, but you’re hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You’re looking at private cloud solutions based on hyperconverged infrastructure, but you’re concerned with the limits inherent in those technologies.
Nordstrom is transforming the way that they do business and the cloud is the key to enabling speed and hyper personalized customer experiences. In his session at 21st Cloud Expo, Ken Schow, VP of Engineering at Nordstrom, discussed some of the key learnings and common pitfalls of large enterprises moving to the cloud. This includes strategies around choosing a cloud provider(s), architecture, and lessons learned. In addition, he covered some of the best practices for structured team migration an...
With tough new regulations coming to Europe on data privacy in May 2018, Calligo will explain why in reality the effect is global and transforms how you consider critical data. EU GDPR fundamentally rewrites the rules for cloud, Big Data and IoT. In his session at 21st Cloud Expo, Adam Ryan, Vice President and General Manager EMEA at Calligo, examined the regulations and provided insight on how it affects technology, challenges the established rules and will usher in new levels of diligence arou...
Most technology leaders, contemporary and from the hardware era, are reshaping their businesses to do software. They hope to capture value from emerging technologies such as IoT, SDN, and AI. Ultimately, irrespective of the vertical, it is about deriving value from independent software applications participating in an ecosystem as one comprehensive solution. In his session at @ThingsExpo, Kausik Sridhar, founder and CTO of Pulzze Systems, discussed how given the magnitude of today's application ...
The “Digital Era” is forcing us to engage with new methods to build, operate and maintain applications. This transformation also implies an evolution to more and more intelligent applications to better engage with the customers, while creating significant market differentiators. In both cases, the cloud has become a key enabler to embrace this digital revolution. So, moving to the cloud is no longer the question; the new questions are HOW and WHEN. To make this equation even more complex, most ...
As you move to the cloud, your network should be efficient, secure, and easy to manage. An enterprise adopting a hybrid or public cloud needs systems and tools that provide: Agility: ability to deliver applications and services faster, even in complex hybrid environments Easier manageability: enable reliable connectivity with complete oversight as the data center network evolves Greater efficiency: eliminate wasted effort while reducing errors and optimize asset utilization Security: imple...
Mobile device usage has increased exponentially during the past several years, as consumers rely on handhelds for everything from news and weather to banking and purchases. What can we expect in the next few years? The way in which we interact with our devices will fundamentally change, as businesses leverage Artificial Intelligence. We already see this taking shape as businesses leverage AI for cost savings and customer responsiveness. This trend will continue, as AI is used for more sophistica...
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, led attendees through the exciting evolution of the cloud. He looked at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering m...
In his session at 21st Cloud Expo, Raju Shreewastava, founder of Big Data Trunk, provided a fun and simple way to introduce Machine Leaning to anyone and everyone. He solved a machine learning problem and demonstrated an easy way to be able to do machine learning without even coding. Raju Shreewastava is the founder of Big Data Trunk (www.BigDataTrunk.com), a Big Data Training and consulting firm with offices in the United States. He previously led the data warehouse/business intelligence and B...