Monthly Archives: July 2016

The Road to CouchDB 2.0

This is the first in a series of blog posts introducing the Apache CouchDB 2.0 release.

C.o.u.c.h.D.B. is famously a backronym for “Cluster of unreliable commodity hardware”. However, the 1.x series of CouchDB has been a single-node database system. While it had been designed to be used in a cluster, and while there are clustering strategies and solutions on top of CouchDB 1.x, none are built-in.

In 2008 the startup Cloudant, founded by CouchDB contributor Adam Kocoloski together with his particle physicist colleagues Mike Miller and Alan Hoffman, started building a proprietary layer on top of CouchDB, using the same core technology, Erlang, turning CouchDB into a clustered database.

The Cloudant founders had built custom data storage systems during the tenure as particle physicists working with CERN’s Large Hadron Collider really big data sets (years before “big data” was thing).

Only one year before, in 2007, Amazon had published a paper about “Dynamo” that outlined Amazon’s solution to scale their database layer to their ever growing needs.

Building on the principles on the Dynamo paper, Cloudant’s clustering layer turned CouchDB into a genuine Big Data capable database.

Cloudant’s core business model is a managed database service using the CouchDB clustering technology they have developed. But their key value proposition is the management of that database, not the code itself, so in 2010 Cloudant made their clustering technology available as the open source project BigCouch.

And in summer 2013 they donated the BigCouch project to the Apache Software Foundation to be integrated into Apache CouchDB proper, fulfilling CouchDB’s original promise of supporting clusters of unreliable commodity hardware as Apache CouchDB 2.0.

In the past pretty exactly three years, the two now widely diverging codebases have been unified into one (with a few warts we are aiming to remove for 3.0 and beyond). The three milestones, roughly one per year, were:

  1. The Initial “Windsor” Merge: Named after the final of two hacking sessions of CouchDB core contributors and Cloudant employees Robert Newson and Paul Davis to bring the BigCouch source code into the Apache CouchDB repository and `master` branch of development.
  2. At the time of the branching off of BigCouch, CouchDB was at version 1.0.1. At the time of the Windsor merge, CouchDB was at version 1.4.0 and several substantial new features hadn’t been added to the codebase yet, so we had to do some catch up.
  3. With all the pieces in place, we had to make sure CouchDB 2.0 was a coherent project: Installation, documentation, all tests working, etc. So we spent the last year polishing off the final experience.

With 2.0 in release candidate phase, we already identified the upcoming areas of work and we are not going to delay releases again for this long.

Earlier this year, Cloudant began to upgrade their production clusters to the newly merged 2.0 codebases and on top of that smooth transition, we are now ready to release CouchDB 2.0 after a thorough release candidate process.

You can download the latest release candidate from http://couchdb.apache.org/release-candidate/2.0/. Files with -RC in their name a special release candidate tags, and the files with the git hash in their name are builds off of every commit to CouchDB master.

We are inviting the community to thoroughly test their applications with CouchDB 2.0 release candidates. See the testing and setup instructions for more details.

CouchDB Weekly News, July 21, 2016

Major Discussions

Starting 2.0 Release Candidates (see thread)

We are now at a point where we can start the CouchDB 2.0 release candidate phase, see the thread for more information on how you can help with testing the tarball.

Releases in the CouchDB Universe

  • beanbag 4.8.0 – Generic CouchDB driver built upon the teepee module
  • cdbcli 0.1.1 (Python) – Interactive command line shell for CouchDB
  • clusterpost-provider 1.2.2 (Hapi) – Transfer job description and data to CouchDB. Submit task to the computing grid.
  • couchdbjs 1.0.1 – A Node client for CouchDB.
  • lazers 0.1.0 (Rust) – laze.rs is a CouchDB toolkit written in Rust
  • logstash-input-couchdb_changes 3.0.2 (Ruby Gem) – This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname.
  • poms 2.1.2 (Ruby Gem) – Interface to POMS CouchDB API

PouchDB

Opinions and other News in the CouchDB Universe

… and in the PouchDB Universe

CouchDB Use Cases, Questions and Answers

Use Case:

  • nano-records – A module for interacting with CouchDB through nano with Node.js.

Stack Overflow:

no public answer yet:

PouchDB Use Cases, Questions and Answers

Use Case:

  • pouch-clerk: PouchDB worker reacting to document changes. Each document has a state. A clerk listens to changes in one or more PouchDB databases and react to document state changes.

No public answer on Stack Overflow yet:

For more new questions and answers about CouchDB, see these search results and about PouchDB, see these.

Get involved!

If you want to get into working on CouchDB:

  • We have an infinite number of open contributor positions on CouchDB. Submit a pull request and join the project!
  • Do you want to help us with the work on the new CouchDB website? Get in touch on our new website mailing list and join the website team! – www@couchdb.apache.org
  • The CouchDB advocate marketing programme is just getting started. Join us in CouchDB’s Advocate Hub!
  • CouchDB has a new wiki. Help us move content from the old to the new one!
  • Can you help with Web Design, Development or UX for our Admin Console? No Erlang skills required! – Get in touch with us.
  • Do you want to help moving the CouchDB docs translation forward? We’d love to have you in our L10n team! See our current status and languages we’d like to provide CouchDB docs in on this page. If you’d like to help, don’t hesitate to contact the L10n mailing list on l10n@couchdb.apache.org or ping Andy Wenk (awenkhh on IRC).

We’d be happy to welcome you on board!

Events

Job opportunities for people with CouchDB skills

Time to relax! Once in a “buck moon” edition

  • “A few hundred years ago, naming moons was more about practically tracking the seasons than coming up with a catchy hashtag. For instance, according to the Farmer’s Almanac June’s ‘strawberry moon’ and September’s ‘corn moon’ both correspond with ideal harvesting times.” – Here’s Everything You Need To Know About Tonight’s “Buck Moon”
  • “A baby planet at least as large as New Jersey that smashed into the moon billions of years ago might have created the mysterious grooves that score the lunar surface, a new study suggests.” – Long-Ago Giant Impact May Have Created Moon’s Mysterious Grooves
  • “In the case of the America’s ambition to land on the moon, the backup plan was a little bit bonkers because it relied on a cramped, two-person spacecraft never really intended for use in lunar orbit. It was a plan (well, really, a series of plans) that would have made a lunar landing possible with less of everything—astronauts, fuel, mass, interior volume, launch thrust, and, in one particularly exciting variant, the entire outer body of a lunar lander.” – America’s Crazy Backup Plan To Get To The Moon
  • “A new study says much smaller quakes are happening deep down below the earth’s crust almost on a regular schedule that are tied to the phases of the moon. Just as the moon’s gravitational pull affects the ocean tides, scientists say they now have hard data to show how it has the same effect on hard ground.” – New Study Indicates Link Between Moon Phases And Earthquakes
  • “On Earth, there would be a significant amount of long winters, and ‘a general cooling off,’ of the world at first, he says, perhaps followed by a warming. We’d see ‘wild extremes’ that are more drawn out and last much longer than the planet has previously exhibited. Petro emphasizes that these climatic effects would certainly not be felt immediately — they would occur on a scale of 1,000 to 10,000 years. But there is one other effect that humans would have to find solutions to very quickly if the moon were blown up or hit with some sort of explosive force: debris.” – If Aliens Ever Attack Earth, We Need To Protect The Moon Or Die Trying

… and also in the news