Welcome!

SDN Journal Authors: Esmeralda Swartz, Elizabeth White, Rex Morrow, Datical, Pat Romanski, Carmen Gonzalez

Related Topics: Java, SOA & WOA, .NET, Red Hat, Cloud Expo, SDN Journal

Java: Blog Feed Post

NoSQL and Partition Tolerance

While NoSQL implementations are aware that nodes have left, they are not aware that said nodes have formed a separate partition

I came across an article discussing NoSQL and partition tolerance.

The NoSQL Partition Tolerance Myth (link)

I may not entirely agree with the author.

But what most NoSQL systems offer is a peculiar behavior that is not partition tolerant, but partition oblivious instead.

 

No argument here. While NoSQL implementations are aware that nodes have left, they are not aware that said nodes have formed a separate partition.

In this case, we would want failure detection and carry out those transfers where the accounts are both on the same side of the partition, while denying or deferring transfers that cross the chasm.

The author is assuming that the account is only on one side of the partition. If that is that is case, is doesn’t matter whether the NoSQL implementation is eventually consistent or not. If the account is on both sides of the partition, the solution the author provides still results in an inconsistent state.

In such cases, it is almost always better to build services that degrade gracefully under partitions.

Bingo! The author implies that all NoSQL implementations sacrifice consistency in the event of a partition. That is not true. There are AP implementations (available & partition tolerant), and there are CP implementations (consistent & partition tolerant). However, an AP implementation can function as a CP implementation depending on the configuration and the application. For example, if accounts do not have multiple owners and the application can not withdraw funds from or deposit funds to an account if it can not access the account.

This post is not perfect and it is a bit outdated, but I think its helpful nonetheless.

Visual Guide to NoSQL Systems (link)

If I were to build a bank based on Dynamo, the granddaddy of all first-generation NoSQL data stores, it would silently split into two halves, like a lobotomized patient.

I would not say that Amazon Dynamo is the grandparent of all NoSQL implementations. I would say that there are two parents: Amazon Dynamo and Google BigTable. Then there are the grandparents…

A Brief History of NoSQL (link)

In this scenario, the hypothetical backend for Banko Dynamo would not only not provide any indication of failure, but allow a customer to create as many new accounts as there are partitions, one in each.

Why is the author now using account creation instead of withdrawals and deposits, and what is the relevance of creating multiple accounts? If my debit card does not work, I do not create a new account. That, and I maintain two checking accounts and one savings account with the same bank.

Let’s go back to withdrawals and deposits. If the accounts do not have multiple owners, it does not matter whether the NoSQL implementation is eventually consistent or not. If the accounts do have multiple owners, it depends on the NoSQL implementation. If it is inspired by Google BigTable (e.g. Apache HBase) or both Google BigTable and Amazon Dynamo (e.g. Apache Cassandra), it does not matter. These NoSQL implementations are CP, or can be configured to be CP. If it inspired only by Amazon Dynamo and it is eventually consistent, it may or may not matter…

Let’s assume that account withdrawals / deposits are separate from the accounts themselves and that the account is both consistent and available during a partition. The account has multiple owners but it is more or less read only.

My account has a balance of $100 (calculated from the withdrawals and deposits). Now, there are two partitions: A and B. I purchase $50 of St. Bernardus Abt 12 at Binny’s via partition A. Partition A now has withdrawal #1. I have dinner at Baume & Brix for $75 via partition B. Partition B now has withdrawal #2. My account has a balance of $50 in partition A. It has a balance of $25 in partition B. My account should have a balance of minus $25.

Does it matter? My account may not have a balance of minus $25, but it will. When the partition is repaired, the application will be able to access all of the withdrawals and deposits on my account. I may be charged an overlimit fee.

What if the NoSQL implementation sacrificed availability? My payment at Binny’s did not go through. That’s not a problem. No St. Bernardus Abt 12 for me. My payment at Baume & Brix did not go through. That’s a problem. I can’t pay for dinner. Baume & Brix can’t accept my payment nor that of any other customer paying with a debit card from the same bank as me via partition B.

What if I made a deposit of $25 at an ATM via partition A? My account will have a balance of $0 after the partition is repaired. I will not be charged an overlimit fee.

There are other scenarios. Perhaps I’m charged an insufficient funds fee and Baume & Brix does not receive payment. Perhaps Baume & Brix later resubmits the payment and receives payment.

Do you really want to sell tickets from both halves of your system? By definition, there is no way you can guarantee uniqueness of those tickets. There will be customers holding identical tickets with identical seat numbers.

Maybe, maybe not. If there is only a single owner per ticket, then yes. However, there may be availability issues. For example, partition A has tickets 1-150 and partition B has tickets 151-200. If all the tickets in partition B have been purchased, visitors may be unable to purchase tickets despite the fact that there may be tickets available in partition A. If there are multiple owners per ticket, I would prefer a NoSQL implementation that is CP. In this case, I would prefer to sacrifice availability rather than consistency.

Here is a better example. What if I report my debit card stolen? Sacrificing availability is not appropriate. What if customer service is accessing my account via the partition with no availability? My debit card must be reported stolen or the thief can continue to make purchases with it. Sacrificing consistency is not appropriate. The thief can continue to make purchases with my debit card via the partition where my account has not been reported stolen. Perhaps account information should not be stored in a distributed system.

And if they did, the first-generation NoSQL stores usually take the ultimate punt by presenting all versions of the divergent objects to the application, and let the application resolve the mess.

No argument here.

But if your data is that soft and inconsequential, why not just use memcached? It’s wicked fast, far faster than Mongo.

Perhaps because MongoDB is a document store and as such provides features that are not provided by key / value stores.

A lot of NoSQL developers pretend that being partition oblivious is a difficult thing to implement. This is false. It’s easy to make a program oblivious to a particular event; namely, you write no code to handle that event.

No argument here.

The thing that greatly helps first generation NoSQL data stores, the thing that enables them to package partition obliviousness as if it were equivalent to partition tolerance, is that they provide a very weak service guarantee in the first place. These systems cannot guarantee that, on a good day, your GET will return the latest PUT.

Sure they can.

In fact, eventual consistency means that a GET can return any previous value, including the Does Not Exist response from the very initial state of the system.

No argument here. Of course, not all NoSQL implementations are eventually consistent.


With all this being said, a NoSQL implementation may or may not be appropriate. To be more specific, a NoSQL implementation that is eventually consistent and sacrifices consistency in the event of a partition may or may not be appropriate. The behaviour is determined by the NoSQL implementation, its configuration, and the application that reads and writes to it. Whether that behaviour is appropriate or not depends on the business requirements.

Read the original blog entry...

More Stories By Daniel Thompson

I curate the content on this page, but the credit goes to my talented colleagues for the posts that you see here. Much of what you read on this page is the work of friends at How to JBoss, and I encourage you to drop by the site at http://www.howtojboss.com for some of the best JBoss technical and non-technical content for developers, architects and technology executives on the Web.

@CloudExpo Stories
Compute virtualization has been transformational, yet security policy implementation and enforcement has lagged behind in agility and automation. There are a number of key considerations when implementing policy in private and hybrid clouds. In his session at 15th Cloud Expo, Holland Barry, VP of Technology at Catbird, will discuss the impact of this new paradigm and what organizations can do today to safely move to software-defined network and compute architectures, including: How normal ope...
Samsung VP Jacopo Lenzi, who headed the company's recent SmartThings acquisition under the auspices of Samsung's Open Innovaction Center (OIC), answered a few questions we had about the deal. This interview was in conjunction with our interview with SmartThings CEO Alex Hawkinson. IoT Journal: SmartThings was developed in an open, standards-agnostic platform, and will now be part of Samsung's Open Innovation Center. Can you elaborate on your commitment to keep the platform open? Jacopo Lenzi: S...
How do APIs and IoT relate? The answer is not as simple as merely adding an API on top of a dumb device, but rather about understanding the architectural patterns for implementing an IoT fabric. There are typically two or three trends: Exposing the device to a management framework Exposing that management framework to a business centric logic • Exposing that business layer and data to end users. This last trend is the IoT stack, which involves a new shift in the separation of what stuff hap...
SYS-CON Events announced today that SOA Software, an API management leader, will exhibit at SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. SOA Software is a leading provider of API Management and SOA Governance products that equip business to deliver APIs and SOA together to drive their company to meet its business strategy quickly and effectively. SOA Software’s technology helps businesses to accel...
SYS-CON Events announced today that Utimaco will exhibit at SYS-CON's 15th International Cloud Expo®, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Utimaco is a leading manufacturer of hardware based security solutions that provide the root of trust to keep cryptographic keys safe, secure critical digital infrastructures and protect high value data assets. Only Utimaco delivers a general-purpose hardware security module (HSM) as a customiz...
Almost everyone sees the potential of Internet of Things but how can businesses truly unlock that potential. The key will be in the ability to discover business insight in the midst of an ocean of Big Data generated from billions of embedded devices via Systems of Discover. Businesses will also need to ensure that they can sustain that insight by leveraging the cloud for global reach, scale and elasticity.
SYS-CON Events announced today that ElasticBox is holding a Hackathon at DevOps Summit, November 6 from 12 pm -4 pm at the Santa Clara Convention Center in Santa Clara, CA. You can enter as an individual or team of up to 10 developers. A New Star Is Born Every Month! All completed ElasticBoxes will then be sent to a judging panel - 12 winners will be featured on the ElasticBox website in 2015. All entrants will receive five full enterprise licenses for one year + ElasticBox headphones + Elasti...
Once the decision has been made to move part or all of a workload to the cloud, a methodology for selecting that workload needs to be established. How do you move to the cloud? What does the discovery, assessment and planning look like? What workloads make sense? Which cloud model makes sense for each workload? What are the considerations for how to select the right cloud model? And how does that fit in with the overall IT tranformation? In his session at 15th Cloud Expo, John Hatem, head of V...
Cloud services are the newest tool in the arsenal of IT products in the market today. These cloud services integrate process and tools. In order to use these products effectively, organizations must have a good understanding of themselves and their business requirements. In his session at 15th Cloud Expo, Brian Lewis, Principal Architect at Verizon Cloud, will outline key areas of organizational focus, and how to formalize an actionable plan when migrating applications and internal services to...
SAP is delivering break-through innovation combined with fantastic user experience powered by the market-leading in-memory technology, SAP HANA. In his General Session at 15th Cloud Expo, Thorsten Leiduck, VP ISVs & Digital Commerce, SAP, will discuss how SAP and partners provide cloud and hybrid cloud solutions as well as real-time Big Data offerings that help companies of all sizes and industries run better. SAP launched an application challenge to award the most innovative SAP HANA and SAP ...
Ixia develops amazing products so its customers can connect the world. Ixia helps its customers provide an always-on user experience through fast, secure delivery of dynamic connected technologies and services. Through actionable insights that accelerate and secure application and service delivery, Ixia's customers benefit from faster time to market, optimized application performance and higher-quality deployments.
SYS-CON Events announced today that Calm.io has been named “Bronze Sponsor” of DevOps Summit Silicon Valley, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Calm.io is a cloud orchestration platform for AWS, vCenter, OpenStack, or bare metal, that runs your CL tools puppet, Chef, shell, git, Jenkins, nagios, and will soon support New Relic and Docker. It can run hosted, or on premise and provides VM automation / expiry, self-service portals,...
In her General Session at 15th Cloud Expo, Anne Plese, Senior Consultant, Cloud Product Marketing, at Verizon Enterprise, will focus on finding the right mix of renting vs. buying Oracle capacity to scale to meet business demands, and offer validated Oracle database TCO models for Oracle development and testing environments. Anne Plese is a marketing and technology enthusiast/realist with over 19+ years in high tech. At Verizon Enterprise, she focuses on driving growth for the Verizon Cloud pla...
SYS-CON Events announced today that Aria Systems, the recurring revenue expert, has been named "Bronze Sponsor" of SYS-CON's 15th International Cloud Expo®, which will take place on November 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Aria Systems helps leading businesses connect their customers with the products and services they love. Industry leaders like Pitney Bowes, Experian, AAA NCNU, VMware, HootSuite and many others choose Aria to power their recurring revenue bu...
The Internet of Things (IoT) is going to require a new way of thinking and of developing software for speed, security and innovation. This requires IT leaders to balance business as usual while anticipating for the next market and technology trends. Cloud provides the right IT asset portfolio to help today’s IT leaders manage the old and prepare for the new. Today the cloud conversation is evolving from private and public to hybrid. This session will provide use cases and insights to reinforce t...
As Platform as a Service (PaaS) matures as a category, developers should have the ability to use the programming language of their choice to build applications and have access to a wide array of services. Bluemix is IBM's open cloud development platform that enables users to easily build cloud-based, creative mobile and web applications without having to spend large amounts of time and resources on configuring infrastructure and multiple software licenses. In this track, you will learn about the...
Blue Box has closed a $10 million Series B financing. The round was led by a strategic investor and included participation from prior investors including Voyager Capital and Founders Collective, as well as the Blue Box executive team. This round follows a $4.3 million Series A closed in December of 2012 and led by Voyager Capital. In May of this year, the company announced general availability of its private cloud as a service offering, Blue Box Cloud. Since that release, the company has dem...
SYS-CON Events announced today that Verizon has been named "Gold Sponsor" of SYS-CON's 15th International Cloud Expo®, which will take place on November 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Verizon Enterprise Solutions creates global connections that generate growth, drive business innovation and move society forward. With industry-specific solutions and a full range of global wholesale offerings provided over the company's secure mobility, cloud, strategic network...
SimpleECM is the only platform to offer a powerful combination of enterprise content management (ECM) services, capture solutions, and third-party business services providing simplified integrations and workflow development for solution providers. SimpleECM is opening the market to businesses of all sizes by reinventing the delivery of ECM services. Our APIs make the development of ECM services simple with the use of familiar technologies for a frictionless integration directly into web applicat...
The only place to be June 9-11 is Cloud Expo & @ThingsExpo 2015 East at the Javits Center in New York City. Join us there as delegates from all over the world come to listen to and engage with speakers & sponsors from the leading Cloud Computing, IoT & Big Data companies. Cloud Expo & @ThingsExpo are the leading events covering the booming market of Cloud Computing, IoT & Big Data for the enterprise. Speakers from all over the world will be hand-picked for their ability to explore the economic...