This is a post in a series about the Apache CouchDB 3.0 release. Check out the other posts in this series.

Querying in CouchDB has always been a little different than in other databases. One such aspect is index creation and updating. In most other databases, an index is usually created upon definition, and updated when new data arrives. In CouchDB, when making a query against an index, that index is created and updated on demand at query time.

The underlying reason is a performance trade-off: in other databases, you are encouraged to have as many indexes as you need, but no more, because each additional index makes inserting and updating any data more resource intensive. In CouchDB on the other hand, you can have as many indexes as you like, only the ones that are actually used are built at query time.

The trade-off in particular is the following: updating many database updates at once is a lot more efficient than doing it one-by-one. However, if an index has not been used in a while, it can take quite some time to process all updates that have happened in the meantime. So the trade-off is data insertion speed for query latency.

Over the lifetime of CouchDB 1.x and 2.x users have built their own little cron jobs that periodically query all indexes to make sure each real query has at most a little database-update gap to bridge, making query responses more predictable.

CouchDB 3.0 introduces Ken, an automatic background indexing service that does this for you. No need to keep maintaining those view update cron jobs. Ken has been on duty at Cloudant for a long time, and is finally available in Open Source CouchDB as of 3.0
See the documentation for more details.

This is a post in a series about the Apache CouchDB 3.0 release. Check out the other posts in this series.

Hand in hand with smarter compaction and view indexing, 3.0 introduces the third major Cloudant contribution: A brand new I/O Queue.

CouchDB 1.x did not have a way to manage disk I/O in any way. All requests external and internal (like view indexing or compaction) were treated equally. This meant that while running compaction, for example, all other requests to the database were impacted.

CouchDB 2.x introduced a basic I/O Queue that differentiated between interactive requests and background tasks, and made sure that background tasks did not take away any performance from interactive requests. That is, your application would receive full performance whenever it needed it, and CouchDB would do maintenance when there were spare CPU cycles.

However, this paints a very simple picture, and with the I/O Queue system being very basic, many caveats applied.

CouchDB 3.0 introduces a more sophisticated I/O Queue system that lets you control all types of requests and tasks into queues with different performance characteristics.

See the documentation for all details.