Welcome!

SDN Journal Authors: Pat Romanski, Liz McMillan, Elizabeth White, Mark Hoover, Stefan Bernbo

Related Topics: @BigDataExpo, Microservices Expo, IBM Cloud, Agile Computing, @CloudExpo, Cloud Security, SDN Journal

@BigDataExpo: Article

Innovative Analytics - Changing the IT Landscape

How log analysis is transforming how products are monitored

A common theme heard in the IT marketplace is that innovative technologies are enabling new insights to be harvested from data. Cloud computing provides the necessary infrastructure that allows smaller organizations to enjoy the value from these insights. As cloud computing becomes mainstream, analytics driven by new sources are leveraged by formation of new partnerships. GE and Google recently announced a partnership for the utility business that integrates Google map functions with GE's geospatial analytical tools to improve visualization. Any way you look at it, integrating consumer and industrial assets with speed and ease is game changing.

Recently, I attended the Strata and IBM Pulse 2013 Conferences that were relatively different but that both incorporated common analytics themes. The Strata Conference is independently organized and focused on data analytics while IBM Pulse, run by the company's Tivoli organization, primarily targets systems management.

Pulse 2013 focused more attention on analytics than last year's event.  While the Strata Conference focused on start-ups that demonstrated interesting analytic products, IBM is looking to leverage analytics as a core competency by embedding those processes into current solutions. Speaking at IBM's recent investor meeting, CEO Ginni Rometty projected $20B in analytics and big data revenues by 2015. To meet this goal, all IBM brands seem to be adding analytic capabilities.

One analytics area I found interesting at both Strata and IBM Pulse were solutions focused on system logs. Computer data logging is a common practice and supports the goal of keeping a history of administrator activities and problem diagnoses.  After an outage, logs are thoroughly analyzed by technicians who scour for the root cause then take appropriate steps to fix malfunctioning components.

The number of inter-connected components in a cloud computing infrastructure significantly increases the volume and variety of log data produced, as well as its importance. Traditional log file solutions leveraged legacy database technologies that are unable to process data being created at the variety and velocity today in a timely and cost effective manner. At the Strata Conference, Glassbeam and Splunk offered interesting solutions; and at Pulse, IBM released a new product capability in their SmartCloud portfolio that analyzes logs, events and metrics in an integrated manner. Let's discuss each in turn.

  • Glassbeam's goal is to provide a platform to help companies get intelligence from machine data. Their partnerships with LogiXML, OpSource (now part of Dimension Data) and Vertica (now part of HP) give them the capability to quickly assemble an innovative solution. Capabilities that focus on customer intelligence, product engineering and service revenue will be of interest to any organization looking to leverage intelligence and value from log data.
  • Splunk surprised a lot of folks when their stock price doubled on the first day of trading after they went public, offering insight to investors' thinking regarding Splunk's capabilities and the value of log analytics to customers. While Glassbeam is a niche and vertical solution, Splunk is a broad solution that covers the gamut of logs, from call centers to click streams.
  • IBM is a late entrant into developing a core product focused on log analytics leveraging new technologies. Their new product is promising as it integrates competencies from multiple software divisions as well as the recent Vivisimo acquisition. This combination of products brings the challenge of integration and installation, while allowing the new product to pull strengths of multiple best-of-breed products. By integrating systems manuals as an additional source to identify specific problems, this new workload analytics capability will be a good addition for existing and new IBM Tivoli customers.

Application logs have offered ways to diagnose system problems for a long time. Analyzing logs from multiple operational processes is a new challenge, and new technologies like Hadoop provide an excellent way to proactively act on intelligence gained. As technologies mature, technology giants will be forced to introduce new products to meet customer needs.

The ability to create new products rapidly is demonstrated by newcomers like Glassbeam and Splunk, changing the IT landscape. Pulse 2013 exhibited legacy product improvements along with new products bundling multiple capabilities from a diverse portfolio to address customer demand. Existing technology vendors need to quickly build their own products or acquire companies that will allow them to fill in this void in their portfolios. Predictive analytics from log data is a good example of how businesses can add significant value to their customers.

This post was first published on Robustcloud.com. Republished with permission.

More Stories By Larry Carvalho

Larry Carvalho runs Robust Cloud LLC, an advisory services company helping various ecosystem players develop a strategy to take advantage of cloud computing. As the 2010-12 Instructor of Cloud Expo's popular Cloud Computing Bootcamp, he has already led the bootcamp in New York, Silicon Valley, and Prague, receiving strong positive feedback from attendees about the value gained at these events. Carvalho has facilitated all-day sessions at customer locations to set a clear roadmap and gain consensus among attendees on strategy and product direction. He has participated in multiple discussion panels focused on cloud computing trends at information technology events, and he has delivered all-day cloud computing training to customers in conjunction with CloudCamps. To date, his role has taken him to clients in three continents.

@CloudExpo Stories
The cloud market growth today is largely in public clouds. While there is a lot of spend in IT departments in virtualization, these aren’t yet translating into a true “cloud” experience within the enterprise. What is stopping the growth of the “private cloud” market? In his general session at 18th Cloud Expo, Nara Rajagopalan, CEO of Accelerite, explored the challenges in deploying, managing, and getting adoption for a private cloud within an enterprise. What are the key differences between wh...
Due of the rise of Hadoop, many enterprises are now deploying their first small clusters of 10 to 20 servers. At this small scale, the complexity of operating the cluster looks and feels like general data center servers. It is not until the clusters scale, as they inevitably do, when the pain caused by the exponential complexity becomes apparent. We've seen this problem occur time and time again. In his session at Big Data Expo, Greg Bruno, Vice President of Engineering and co-founder of StackIQ...
Security, data privacy, reliability, and regulatory compliance are critical factors when evaluating whether to move business applications from in-house, client-hosted environments to a cloud platform. Quality assurance plays a vital role in ensuring that the appropriate level of risk assessment, verification, and validation takes place to ensure business continuity during the migration to a new cloud platform.
"Tintri was started in 2008 with the express purpose of building a storage appliance that is ideal for virtualized environments. We support a lot of different hypervisor platforms from VMware to OpenStack to Hyper-V," explained Dan Florea, Director of Product Management at Tintri, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Containers have changed the mind of IT in DevOps. They enable developers to work with dev, test, stage and production environments identically. Containers provide the right abstraction for microservices and many cloud platforms have integrated them into deployment pipelines. DevOps and containers together help companies achieve their business goals faster and more effectively. In his session at DevOps Summit, Ruslan Synytsky, CEO and Co-founder of Jelastic, reviewed the current landscape of Dev...
One of the hottest areas in cloud right now is DRaaS and related offerings. In his session at 16th Cloud Expo, Dale Levesque, Disaster Recovery Product Manager with Windstream's Cloud and Data Center Marketing team, will discuss the benefits of the cloud model, which far outweigh the traditional approach, and how enterprises need to ensure that their needs are properly being met.
The security needs of IoT environments require a strong, proven approach to maintain security, trust and privacy in their ecosystem. Assurance and protection of device identity, secure data encryption and authentication are the key security challenges organizations are trying to address when integrating IoT devices. This holds true for IoT applications in a wide range of industries, for example, healthcare, consumer devices, and manufacturing. In his session at @ThingsExpo, Lancen LaChance, vic...
WebRTC has had a real tough three or four years, and so have those working with it. Only a few short years ago, the development world were excited about WebRTC and proclaiming how awesome it was. You might have played with the technology a couple of years ago, only to find the extra infrastructure requirements were painful to implement and poorly documented. This probably left a bitter taste in your mouth, especially when things went wrong.
In their general session at 16th Cloud Expo, Michael Piccininni, Global Account Manager - Cloud SP at EMC Corporation, and Mike Dietze, Regional Director at Windstream Hosted Solutions, reviewed next generation cloud services, including the Windstream-EMC Tier Storage solutions, and discussed how to increase efficiencies, improve service delivery and enhance corporate cloud solution development. Michael Piccininni is Global Account Manager – Cloud SP at EMC Corporation. He has been engaged in t...
You have great SaaS business app ideas. You want to turn your idea quickly into a functional and engaging proof of concept. You need to be able to modify it to meet customers' needs, and you need to deliver a complete and secure SaaS application. How could you achieve all the above and yet avoid unforeseen IT requirements that add unnecessary cost and complexity? You also want your app to be responsive in any device at any time. In his session at 19th Cloud Expo, Mark Allen, General Manager of...
WebRTC is bringing significant change to the communications landscape that will bridge the worlds of web and telephony, making the Internet the new standard for communications. Cloud9 took the road less traveled and used WebRTC to create a downloadable enterprise-grade communications platform that is changing the communication dynamic in the financial sector. In his session at @ThingsExpo, Leo Papadopoulos, CTO of Cloud9, discussed the importance of WebRTC and how it enables companies to focus o...
Big Data engines are powering a lot of service businesses right now. Data is collected from users from wearable technologies, web behaviors, purchase behavior as well as several arbitrary data points we’d never think of. The demand for faster and bigger engines to crunch and serve up the data to services is growing exponentially. You see a LOT of correlation between “Cloud” and “Big Data” but on Big Data and “Hybrid,” where hybrid hosting is the sanest approach to the Big Data Infrastructure pro...
In his General Session at DevOps Summit, Asaf Yigal, Co-Founder & VP of Product at Logz.io, will explore the value of Kibana 4 for log analysis and will give a real live, hands-on tutorial on how to set up Kibana 4 and get the most out of Apache log files. He will examine three use cases: IT operations, business intelligence, and security and compliance. This is a hands-on session that will require participants to bring their own laptops, and we will provide the rest.
"We're bringing out a new application monitoring system to the DevOps space. It manages large enterprise applications that are distributed throughout a node in many enterprises and we manage them as one collective," explained Kevin Barnes, President of eCube Systems, in this SYS-CON.tv interview at DevOps at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
In his General Session at 16th Cloud Expo, David Shacochis, host of The Hybrid IT Files podcast and Vice President at CenturyLink, investigated three key trends of the “gigabit economy" though the story of a Fortune 500 communications company in transformation. Narrating how multi-modal hybrid IT, service automation, and agile delivery all intersect, he will cover the role of storytelling and empathy in achieving strategic alignment between the enterprise and its information technology.
Buzzword alert: Microservices and IoT at a DevOps conference? What could possibly go wrong? In this Power Panel at DevOps Summit, moderated by Jason Bloomberg, the leading expert on architecting agility for the enterprise and president of Intellyx, panelists peeled away the buzz and discuss the important architectural principles behind implementing IoT solutions for the enterprise. As remote IoT devices and sensors become increasingly intelligent, they become part of our distributed cloud enviro...
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo 2016 in New York. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place June 6-8, 2017, at the Javits Center in New York City, New York, is co-located with 20th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry p...
@DevOpsSummit at Cloud taking place June 6-8, 2017, at Javits Center, New York City, is co-located with the 20th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long developm...
A look across the tech landscape at the disruptive technologies that are increasing in prominence and speculate as to which will be most impactful for communications – namely, AI and Cloud Computing. In his session at 20th Cloud Expo, Curtis Peterson, VP of Operations at RingCentral, will highlight the current challenges of these transformative technologies and share strategies for preparing your organization for these changes. This “view from the top” will outline the latest trends and developm...
The 20th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held June 6-8, 2017, at the Javits Center in New York City, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Containers, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal ...