If I’m honest, I think of MongoDB as more of a “data store” than an actual database. This may be a controversial opinion these days, but I don’t think it’s a hard position to support. Granted, MongoDB does a great job storing data for a particular use case and can be highly optimized for that purpose. But, as applications inevitably evolve, Mongo’s single-use case formula where applications leverage the document store to optimize one access path to the data starts to unravel. To be a true general purpose database, you need to be able to support many different paths through the same data set to support different use cases, whether your goal is reporting, inline analytics or just supporting a new business process that wasn’t envisioned when the document structure was designed.
Support For Multi-Document ACID Transactions
Based on some of their recent comments, I think the team at MongoDB actually know this. This is why they’re tying themselves in knots trying to become a general purpose database. The first step towards this objective is MongoDB’s recent announcement that it will finally support multi-document ACID transactions in MongoDB 4.0. Even with this development, it’s unclear whether ACID is really a priority for them, however. MongoDB executives have this to say on the matter:
- “... document model negates the need for ACID transactions”[1]— Eliot Horowitz, MongoDB CTO
- “... 80%-90% of applications don't need multi-document transactions at all”[2]— Grigori Melnik, MongoDB VP of Products, Server, and Enterprise Tools
- “... multi-document, fully ACID documents are coming to MongoDB 4.0 this summer”[3]— Eliot Horowitz, MongoDB
-
“With this final piece of the puzzle with multi-document ACID transaction guarantees, it’s game over”[4]— Seong Park, MongoDB VP of Strategy and Product Marketing
If you read these quotes and find yourself confused about MongoDB’s belief in the true impact of multi-document ACID, rest assured you’re not alone. With the inconsistency in these statements — all made in 2018 by the way! — it’s unclear to us if ACID is really a priority for MongoDB or if they think it will be a priority for users.
A Surprise or a Long Time Coming?
While Postgres offers native support for ACID compliance, MongoDB has taken years to achieve support for multi-document ACID transactions. This development is a surprise to many, but it has been seen as a long time coming for others.
According to Alex Woodie, Managing Editor of Datanami, “MongoDB ... has always supported single-document ACID transaction guarantees, but it still pushed the technical complexity required to deliver full ACID guarantees for transactions that touched more than one document out to the user.”[5] However, Woodie added that MongoDB has actually been “... working toward improving its consistency guarantees ever since it bought WiredTiger back in late 2014.” [6]
And some observers thought that MongoDB was no closer to this goal in 2016, either. Entrepreneur, functional programmer, and speaker John De Goes published his thoughts on the matter at the time. “In the relational world, all the grown-up operational databases have ACID,” he said. “Given MongoDB’s apparent reluctance to follow the footsteps of other NoSQL databases like MarkLogic, Couchbase, and Aerospike, and push for greater ACID, I’d say there’s zero chance that an ACID compliant storage engine will be announced at MongoDB World 2016.”[7]
But that all changed in 2018 when MongoDB co-founder and CTO Eliot Horowitz announced the company’s support for multi-document ACID transaction guarantees with MongoDB version 4.0. So, what took so long? Horowitz stated, “The reason we didn’t need this is because the document model negates the need for ACID transactions — not for all of them, but for most of them.” [8]
This opinion was further endorsed by Grigori Melnik, VP of Products, Server, and Enterprise Tools at MongoDB. “Because of this fundamental difference in data modeling, MongoDB's existing atomicity guarantees are able to meet the data integrity needs of most applications,” he stated. “In fact, we estimate 80%-90% of applications don't need multi-document transactions at all.” [9]
What is most surprising is that Melnik says support for multi-document ACID transactions has actually been a “massive multi-year engineering investment”[10] for MongoDB, but CTO Eliot Horowitz does not think this feature would be used a great deal. He said, “I don’t expect this to be a common way to write to MongoDB,”[11] and stated that developers will only enable it for “very specific use cases.”[12]
An Exercise in Providing Assurance
Yes, it seems like MongoDB has invested multiple years of engineering into a feature they state is unnecessary up to 90% of the time and will likely be underutilized.
Why? This inconsistency appears to be an exercise in providing just-in-case assurances. CTO Eliot Horowitz has stated that some developers working with MongoDB “have this nagging fear that they may need it in the future.”[13]
And, does MongoDB’s delivery of ACID come with a catch? Technologist Dennis Duckworth explored the fine print in a recent blog post. Duckworth writes, “During this announcement, MongoDB executives did acknowledge that there are a few restrictions for the first implementation of transactions. In V4.0, MongoDB’s transactions will be limited to single-shards, though they announced that V4.2 will include the ability to do transactions across multi-shard implementations.”[14] Duckworth also spotted another catch: “Later, while exploring MongoDB’s documentation, I found another restriction: while MongoDB has supported multiple storage engines for data in the past, only WiredTiger (one of MongoDB’s storage engines) could be used if transactions were desired.”[15]
So, is ACID really important to MongoDB? We’re unsure, but one thing we are clear on is EDB’s commitment to the importance of ACID transactions. Postgres’ long-term native support for ACID compliance, Multiversion Concurrency Control, data and referential integrity, server side programs, and fine-grained access controls are mature and stable. This is especially true compared to alternatives that have only just recently started implementing such features.
The Strength of a True General Purpose Database
MongoDB’s efforts around multi-document ACID transactions were certainly a necessary step towards being a general purpose database, but they are by no means sufficient. The issue is that MongoDB is still not fundamentally architected for this purpose. Even ignoring the limitations in the newly enhanced transaction support, it still organizes data in a very application-centric way, limiting reuse. And despite all the NoSQL hype, there’s still a lot of value in being able to query your data using an industry standard query language like SQL once you move beyond your first use case. In the end, while adding multi-document transactions certainly enables you to do some things with MongoDB that you couldn’t before, it’s still best suited for use as an application-specific data store.
When all data for document access is stored together, sure, you’ll achieve performance, but it will be oriented to a single access path. As applications evolve, this model does not hold up against the rigors of pulling data to support analytics, statistics, and business insights. Soon, you’ll be designing multiple applications to access your data, leading to application sprawl and increased complexity. Postgres remains the most ideal general purpose database, giving users the flexibility to treat their database as a single pool and enable access through many different access patterns.