This is a post in a series about the Apache CouchDB 3.0 release. Check out the other posts in this series.
In the lead up to CouchDB 3.0, we have noticed that our needs for running automated testing had outgrown what the fine folks at Travis CI provide for free, so we started looking for alternatives.
We had multiple goals for this new CI solution:
- quick startup time: during busy times, Travis’ free offering sometimes takes an hour to start working, which is entirely fair, but we needed something better.
- better performance: we spent quite some time hardening our test suite against Travis’s environment of resource constrained machines, to ensure we have no false-negative reports due to timeouts and other issues. In some cases, this meant writing a test case that did not represent real-world usage of the code it was testing. We are also always happy for a faster test runtime, even on modern machines, our test suite takes about 20 minutes to run, add a matrix of different configurations, and we need some serious compute to move quickly and confidently.
- broader platform support: CouchDB 3.0 now officially supports ARM and PPC platforms in 32bit and 64bit, mainly because our new solution can automatically run tests on these platforms each time we change code. We also now (again) test automatically on FreeBSD systems, and a macOS build machine is in the works.
- nightly binaries: we wanted to be able to build binaries for all supported platforms for PRs and master for improved testing of as-of-yet unreleased features. This now happens automatically. You can subscribe to the Developer mailing list to get access to these binaries.
Our new CI solution is based on Jenkins via CloudBees with worker servers donated by IBM/Cloudant and MacStadium and it supports all of the above requirements
However, none of this would have been made possible without the untiring efforts of Joan “wohali” Touzet who pulled all the strings together to making this happen, from organising and configuring donation hardware, to sorting out ASF policy to make sure we are all aligned, to writing oh so many build scripts to tie the whole service together. Some of Joan’s time was donated by Neighbourhoodie Software.
Meet the new https://ci.couchdb.org.