The Road to CouchDB 3.0: Smarter Smarter I/O Queue

This is a post in a series about the Apache CouchDB 3.0 release. Check out the other posts in this series.

Hand in hand with smarter compaction and view indexing, 3.0 introduces the third major Cloudant contribution: A brand new I/O Queue.

CouchDB 1.x did not have a way to manage disk I/O in any way. All requests external and internal (like view indexing or compaction) were treated equally. This meant that while running compaction, for example, all other requests to the database were impacted.

CouchDB 2.x introduced a basic I/O Queue that differentiated between interactive requests and background tasks, and made sure that background tasks did not take away any performance from interactive requests. That is, your application would receive full performance whenever it needed it, and CouchDB would do maintenance when there were spare CPU cycles.

However, this paints a very simple picture, and with the I/O Queue system being very basic, many caveats applied.

CouchDB 3.0 introduces a more sophisticated I/O Queue system that lets you control all types of requests and tasks into queues with different performance characteristics.

See the documentation for all details.

The Road to CouchDB 3.0

This is the first in a series of blog posts introducing the Apache CouchDB 3.0 release.

The CouchDB 3.x series of releases is going to be the best we have ever released using the “Classic CouchDB Architecture”. Of course, the newest releases should also always be “the best”, but in this case, we really mean it.

2.x finally fulfilled CouchDB’s original vision of a replicating, clusterable, highly scalable data store that is easy to use, and it was rather successful at that.

The main focus of the 3.x release line is addressing most of the shortcomings that our otherwise happy users have reported over the past years.

The 3.x release series is particularly focused on a few areas of improvement over 2.x:

  • Security
  • Missing, obvious features
  • Even smoother operational behaviour
  • Prepare for CouchDB 4.0

Coinciding with the 3.0 release, we are dedicating a blog post to the changes in each of these areas:

Let us drill into what “Prepare for CouchDB 4.0 means”, because we can’t properly describe CouchDB 3.x without talking about CouchDB 4.x. Earlier, we mentioned “The Classic CouchDB Architecture”. What do we mean by that?

In CouchDB 4.0 and beyond we are going to take the tried and trusted underpinnings of CouchDB: the append-only B+-tree storage engine and the dynamo clustering layer, and we are going to replace it with FoundationDB.

There will be more blog posts explaining the details of that future release, when the time comes. For 3.x, we need to know that a big change is coming, so we have both an incentive and an opportunity to build the best version of CouchDB on the existing architecture, and that a number of changes are going to prepare our users in 3.x for this future migration.

But as with all CouchDB releases, past, present, and future, we are going to focus hard on strong API compatibility, so your applications can start using the newer features without being rewritten.