The Little Things(1): Do Not Delete

CouchDB takes data storage extremely seriously. This usually means we work hard to make sure that the CouchDB storage modules are as robust as we can make them. Sometimes though, we go all the way to the HTTP API to secure against accidental data loss, saving users from their mistakes, rather than dealing with hard drives and kernel caches that usually stand in the way of safe data storage.

The scenario:

To delete a document in CouchDB, you issue the following HTTP request:

DELETE /database/docid?rev=12345 HTTP/1.1

A common way to program this looks like this:

http.request('DELETE', db + '/' + docId + '?rev=' + docRev);

So far so innocent. Sometimes though, users came to us and complained that their whole database was deleted by that code.

Turns out the above code creates a request that deletes the whole database, if the docId variable isn’t set correctly. The request then looks like:

DELETE /database/?rev=12345 HTTP/1.1

It looks like an honest mistake, once you check the CouchDB log file, but good old CouchDB would just go ahead and delete the database, ignoring the ?rev= value.

We thought this is a good opportunity to help users not accidentally losing their data. So since late 2009 (yes, this is an oldie, but it came up in a recent discussion and we thought it is worth writing about :), CouchDB will not delete a database, if it sees that a ?rev= parameter is present and it looks like that this is just a malformed request, as database deletions have no business requiring a ?rev=.

One can make an easy argument that the code sample is fairly shoddy and we’d agree. But we are not here to argue how our users use our database beyond complying with the API and recommended use-cases. And if we can help them keep their data, that’s a win in our book

Continuing down this thought, we thought we could do one better. You know that to delete a document, you must pass the current rev value, like you see above. This is to ensure that we don’t delete the document accidentally without knowing that someone else may have added an update to it that we don’t actually want to delete. It’s CouchDB’s standard multi version currency control (MVCC) mechanism at work.

Databases don’t have revisions like documents, and deleting a database is a simple HTTP DELETE /database away. Databases, however, do have a sequence id, it’s the ID you get from the changes feed, it’s an number that starts at 0 when the database is created and increments by 1 each time a document is added, updated or deleted. Each state of the database has a single sequence ID associated with it.

Similar to a rev, we could require the latest sequence ID to delete a database, as in:

DELETE /database?seq_id=6789

And deny database deletes that don’t carry the latest seq_id. We think this is a decent idea, but unfortunately, this would break backwards compatibility with older versions of CouchDB and it would break a good amount of code in the field, so we are hesitant to add this feature. In addition, sequence IDs change a little when BigCouch finally gets merged, so we’d have to look at this again then.

In the meantime, we have the protection against simple coding errors and we are happy that our users keep their hard earned data more often now.

Cloudant and IBM: Our Commitment to Apache CouchDB

IBM have announced that they are acquiring Cloudant.

Adam Kocoloski is co-founder and CTO of Cloudant as well as serving on CouchDB’s Project Management Committee. And with his permission, I am re-posting an email he sent to the CouchDB developer list a few hours ago.

In his words:

“Apache CouchDB means a great deal to me, and to Cloudant as a company. Cloudant and CouchDB have grown alongside each other over the past several years in one of the more authentic vendor/community collaborations I can think of in Apache history. Today marks the next step in Cloudant’s growth as we enter into a definitive agreement to become part of IBM.

“What does this mean for CouchDB? I would not have agreed to this transaction if I had any concerns about Cloudant’s ability to continue its contributions and collaboration with Apache CouchDB. IBM has a strong track record in open source software and a productive relationship with Apache; in fact, IBM was instrumental in bringing CouchDB to the ASF many years ago. IBM is fully supportive of our efforts here, and I’m looking forward to bringing increased resources to bear in support of the project.

“CouchDB has the potential to shape the future of distributed data management and computing. 2013 was a year of tremendous progress, as we doubled our committer base and shipped no fewer than eight releases. Already in 2014 we’ve seen amazing progress on long-standing initiatives to enhance the core of the system. The timing is right — in the market and for the community — to take the next big step forward. With your help, that is exactly what we will do.

“Truly, the future of CouchDB is CouchDB.”

Thanks Adam!

I’d like to extend my hearty congratulations to everyone at Cloudant.

The future for CouchDB looks very interesting…