Welcome!

SDN Journal Authors: TJ Randall, Yeshim Deniz, Liz McMillan, Elizabeth White, Pat Romanski

Related Topics: @CloudExpo, Microservices Expo, Containers Expo Blog, Cloud Security, @DXWorldExpo, SDN Journal

@CloudExpo: Article

Six Characteristics of Best-in-Class Object Storage

Six “must haves” that are required to create best-in-class object storage solutions and why they matter to you

The adoption of cloud storage technology for a broad range of consumer and business applications is transforming the storage landscape by transitioning away from traditional disk arrays to object-based storage systems that have the scalability, availability, resiliency and accessibility to enable cloud-scale storage and instant access.

A recent IDC report predicts that the market for File- and Object-Based Storage (FOBS) will experience an annual growth rate of 24.5% through 2017, reaching $38 billion. "Increased versatility will result in more diverse use cases for FOBS," said IDC.

Software based, object storage is not saddled with the cost, complexity and vendor lock-in of legacy storage arrays or the scalability limitations of traditional file system storage. But all object storage systems are not created equal. Here are six "must haves" that are required to create best-in-class object storage solutions.

1. Granular, Automated Scalability:
Best-in-class object stores should support highly flexible scalability, spanning the addition of a single disk all the way up to multiple nodes to extend the capacity or performance of the solution.

Why this matters to you: Granular scalability lets you scale as you grow and eliminates the need to over purchase hardware because of the technical limitations of the storage solution.

2. No Single Point of Failure:
The most efficient object storage systems are built on a symmetric architecture where all nodes run the same code, resulting in high availability and unprecedented scalability, eliminating any single point of failure.

Why this matters to you: When you hear management node, controller node, or database this means more management and the addition of single points of failure that can critically impact performance, stability and fault tolerance. In highly available object storage solutions all nodes do the same thing so that if one fails, the others can immediately remedy the issue. This also eliminates the need for specialized hardware that needs to be physically shipped if an issue is discovered.

3. Flexible Data Protection on a Per-Object Basis:
Data protection flexibility is critical as no single data protection scheme can be optimized for every use case. Object storage systems need both replication and erasure coding as well as the ability to move between them, all available in the same cluster to ensure comprehensive, efficient data protection.

Why this matters to you: One size fits all just does not work in real life. Different use cases require different combinations of replication and erasure coding. Object solutions that constrain the transition from one protection scheme to the other or lock the protection scheme to specific hardware ultimately hinder growth and your ability to optimize resources. Support for both protection schemes on the same server means you can optimize for access, data protection and resource utilization system wide - without constraint.

4. Support for Large and Small Files:
Object stores must be designed with the versatility and flexibility to handle a broad range of applications and workloads without performance impact, equally adept at storing and accessing billions of small files, documents and emails or very large files like high-definition videos.

Why this matters to you: This is primarily about performance from an access perspective. The variation in file sizes will continue. While compression algorithms get more efficient in making files smaller, technological advancements will continue to add to the complexity and depth of some file types resulting in larger files. An object storage solution that ensures rapid access and efficient storage, regardless of file size or object count will increase the number of use cases reducing the number of point solutions you need to purchase.

5. Continuous Integrity Checks and Fast Volume Recovery:
Best of breed solutions continuously check content integrity from a protection scheme and content perspective. If a bad disk is discovered, recovery should be distributed with the rate of repair accelerating as the storage solution grows.

Why this matters to you: Content you store should always be available. Some object solutions only check data integrity on reads - the worst time to ensure data integrity. Others rely on specialized nodes to identify and repair issues that limit scale.

6. Instant Content Lookup and Retrieval:
Best-of-breed solutions allow queries against the object store based on object attributes or customizable metadata "tags" stored with the object.

Why this matters to you: As the amount of content grows from millions to billions of objects and management resources change (hardware migration and employee turnover) efficient content lookup and retrieval becomes a challenge. Some object solutions store metadata in a database, which introduces an additional layer of complexity between content requests and content delivery - a textbook bottleneck. Databases also become unwieldy with size and require investment in specialized management resources. By storing metadata with the object, content is self-contained and security, authentication and all identifying information is always available regardless of application, employee turnover, technological obsolescence or even time.

More Stories By Paul Carpentier

Paul Carpentier is CTO and Founder of Caringo. Known as the father of the Content Addressing concept, He invented the patent pending scalable and upgradeable security that is at the heart of Caringo. He was the architect of SequeLink — the first client/server middleware product to connect heterogeneous front ends running over multiple networks to multiple databases on the server side.

Paul founded Wave Research and conceived FileWave, the first fully automated, model-driven software distribution and management system. At FilePool, he invented the technology that created the Content Addressed Storage industry. FilePool, was sold to EMC who turned CAS into a multi-billion dollar marketplace. Caringo CAStor, based on two of Mr. Carpentier's six patents promises to revolutionize the data storage business in much the same manner that CAS created a whole new marketplace.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


CloudEXPO Stories
Using serverless computing has a number of obvious benefits over traditional application infrastructure - you pay only for what you use, scale up or down immediately to match supply with demand, and avoid operating any server infrastructure at all. However, implementing maintainable and scalable applications using serverless computing services like AWS Lambda poses a number of challenges. The absence of long-lived, user-managed servers means that states cannot be maintained by the service. Longer function invocation times (referred to as cold starts) become very important to track, because they impact the response time of the service and will impose additional cost. Additionally, the transition to smaller individual components (much like breaking a monolithic application into microservices) results in a simpler deployment model, but makes the system as a whole increasingly complex.
Here to help unpack insights into the new era of using containers to gain ease with multi-cloud deployments are our panelists: Matt Baldwin, Founder and CEO at StackPointCloud, based in Seattle; Nic Jackson, Developer Advocate at HashiCorp, based in San Francisco, and Reynold Harbin, Director of Product Marketing at DigitalOcean, based in New York. The discussion is moderated by Dana Gardner, principal analyst at Interarbor Solutions.
Using serverless computing has a number of obvious benefits over traditional application infrastructure - you pay only for what you use, scale up or down immediately to match supply with demand, and avoid operating any server infrastructure at all. However, implementing maintainable and scalable applications using serverless computing services like AWS Lambda poses a number of challenges. The absence of long-lived, user-managed servers means that states cannot be maintained by the service. Longer function invocation times (referred to as cold starts) become very important to track, because they impact the response time of the service and will impose additional cost. Additionally, the transition to smaller individual components (much like breaking a monolithic application into microservices) results in a simpler deployment model, but makes the system as a whole increasingly complex.
With the rise of Docker, Kubernetes, and other container technologies, the growth of microservices has skyrocketed among dev teams looking to innovate on a faster release cycle. This has enabled teams to finally realize their DevOps goals to ship and iterate quickly in a continuous delivery model. Why containers are growing in popularity is no surprise — they’re extremely easy to spin up or down, but come with an unforeseen issue. However, without the right foresight, DevOps and IT teams may lose a lot of visibility into these containers resulting in operational blind spots and even more haystacks to find the presumptive performance issue needle.
Isomorphic Software is the global leader in high-end, web-based business applications. We develop, market, and support the SmartClient & Smart GWT HTML5/Ajax platform, combining the productivity and performance of traditional desktop software with the simplicity and reach of the open web. With staff in 10 timezones, Isomorphic provides a global network of services related to our technology, with offerings ranging from turnkey application development to SLA-backed enterprise support. Leading global enterprises use Isomorphic technology to reduce costs and improve productivity, developing & deploying sophisticated business applications with unprecedented ease and simplicity.