Category Archives: News

The Road to CouchDB 3.0: Goodbye Travis, Hello Jenkins, Much Improved Continuous Integration

This is a post in a series about the Apache CouchDB 3.0 release. Check out the other posts in this series.

In the lead up to CouchDB 3.0, we have noticed that our needs for running automated testing had outgrown what the fine folks at Travis CI provide for free, so we started looking for alternatives.

We had multiple goals for this new CI solution:

  • quick startup time: during busy times, Travis’ free offering sometimes takes an hour to start working, which is entirely fair, but we needed something better.
  • better performance: we spent quite some time hardening our test suite against Travis’s environment of resource constrained machines, to ensure we have no false-negative reports due to timeouts and other issues. In some cases, this meant writing a test case that did not represent real-world usage of the code it was testing. We are also always happy for a faster test runtime, even on modern machines, our test suite takes about 20 minutes to run, add a matrix of different configurations, and we need some serious compute to move quickly and confidently.
  • broader platform support: CouchDB 3.0 now officially supports ARM and PPC platforms in 32bit and 64bit, mainly because our new solution can automatically run tests on these platforms each time we change code. We also now (again) test automatically on FreeBSD systems, and a macOS build machine is in the works.
  • nightly binaries: we wanted to be able to build binaries for all supported platforms for PRs and master for improved testing of as-of-yet unreleased features. This now happens automatically. You can subscribe to the Developer mailing list to get access to these binaries.

Our new CI solution is based on Jenkins via CloudBees with worker servers donated by IBM/Cloudant and MacStadium and it supports all of the above requirements

However, none of this would have been made possible without the untiring efforts of Joan “wohali” Touzet who pulled all the strings together to making this happen, from organising and configuring donation hardware, to sorting out ASF policy to make sure we are all aligned, to writing oh so many build scripts to tie the whole service together. Some of Joan’s time was donated by Neighbourhoodie Software.

Meet the new https://ci.couchdb.org.

The Road to CouchDB 3.0: Introducing Partitions to CouchDB for More Efficient Querying

This is a post in a series about the Apache CouchDB 3.0 release. Check out the other posts in this series.

Apache CouchDB 3.0 comes equipped with a new partitioned database feature, offering  more performant, scalable, and efficient querying of secondary indexes.

Users decide, at database creation time, whether or not to create the database with partitions. All documents in a partitioned database require a partition key, and all documents within a partition are grouped together. Common partition keys could include usernames, IoT device IDs, or locations. Partitioned database documents also have an _id field, but the _id field is in two parts, the partition key and the document key, separated by a “:” character:

Querying against a secondary index for a partition scans only the specified partition range within a single shard.  Compared against a “global” query that requires collating the results of reading a copy of every shard, partition queries require much fewer resources and respond faster at scale. To visualize the difference in database operations, compare a global query on left with a partition query on right:

Querying a partitioned database with a partition key can be done against the Primary Index (_all_docs), as well as a MapReduce view, Search (now available in Apache CouchDB 3.0), or Mango index. Users can combine both partitioned and global indexes within the same database to meet their querying requirements. 

For more information on leveraging Partitioned Databases in Apache CouchDB, check out the following resources: