CouchDB Developer Profile: Geoffrey Cox

You might recognize the name Geoffrey, or Geoff Cox from appearing more than a few times this year in the CouchDB Weekly News. Geoff has authored posts like Don’t Just Relax; Slouch: A JS Client for CouchDB that Does the Heavy LiftingRunning a CouchDB 2.0 Cluster in Production on AWS with Docker, and more recently, Monitoring CouchDB with Prometheus, Grafana and Docker. Currently, Geoff is the co-founder of Quizster, where he daily deals with JavaScript, PouchDB, and CouchDB.

Do you want to talk about your background, or how you got involved in CouchDB?

Four years ago I sold my company, GoExpo, which produces event management software used by some of the largest events in the world to process hundreds of millions of dollars a year in booth sales. GoExpo is powered by PHP, JS and MySQL, which is a rock solid technology stack for processing financial transactions.

About 5 years ago I started experimenting with a prototype of an offline-first GoExpo mobile app that used a custom syncing layer to sync data between the app and database. Even though I got the prototype working, I quickly started to see that real-time syncing is not trivial and is something that you want baked into the core of your database. We ended up scrapping this mobile app because the company that acquired us had their own mobile app, but I still continued to tinker with offline mobile apps in my spare time.

I started experimenting with Firebase, Hoodie and PouchDB and realized that PouchDB gave me just the right level of flexibility and offline capabilities. And by extension, I started using CouchDB, which I found to have a steep learning curve as I was coming from a more traditional SQL world. Over the last few years and especially with the release of CouchDB 2, I’ve really begun to appreciate the power of CouchDB and it’s multi-master syncing.

Recently, I’ve moved on from my position at GoExpo and my wife and I have launched a digital dropbox and grading app called Quizster. Quizster uses CouchDB and with some supporting technologies, CouchDB has been handling our growing user base quite well. I am planning on open sourcing more of our stack so that others can focus on building their apps instead of spending time on some of the overhead incurred when interfacing with CouchDB.

What areas of the project do you work on?

Some people may recognize me as a plugin or peripheral developer. The majority of my CouchDB-related contributions have been to open source projects that use CouchDB and PouchDB.

I recently released Slouch, a JS client for CouchDB that does the heavy lifting. I consider nano to be a great library, but I was able to greatly simplify my backend code by using the patterns in Slouch.

I created replicate-couchdb-cluster as a fault-tolerant utility for replicating an entire CouchDB cluster. Even with CouchDB 2 and its redundant design, I find it essential to have a remote replication of my databases.

I’ve also created redgeoff-couchdb-docker and couchdb-docker-service, which can be used to run CouchDB with Docker in production. By running CouchDB via Docker, you can stay up to date with the latest CouchDB release without having to worry about managing dependencies.

A while back I created delta-pouch, which provides conflict-free collaborative editing for PouchDB. This plugin is useful for when you want a more Firebase-like feel when editing data, but is only really performant for small databases.

I’ve also written a few blog posts on CouchDB on topics that include running CouchDB 2 in production, monitoring CouchDB and using Slouch.

What’s a recent development/event/aspect of the project that you’re excited about?

I’m still very excited about the release of CouchDB 2.0! I was using CouchDB 1 for a while and wondering how I would actually be able to scale it once our user base became large enough to warrant adding more nodes. CouchDB 2’s native clustering solves this problem well and I’m ecstatic because it would be a significant effort to implement this level of clustering on your own. It’s also great that the team was able to preserve almost the entire CouchDB API, which makes it fairly easy to just swap CouchDB 1 with CouchDB 2. I truly appreciate all the hard work that the CouchDB team has put into releasing CouchDB 2!

I’m also excited about how quickly CouchDB 2.1 was released and in its ability to better schedule replications. Replication is essential in many database-per-user designs where different users must only have access to certain data. It can be difficult to scale these replications and CouchDB 2.1 appears to have a better handle on this scheduling. That being said, I still think there is a lot more that most developers need before they can scale their replications and I don’t think that all of it belongs in CouchDB. At Quizster, we have a fairly elaborate way of scaling our replications as we have many databases and some that need to be replicated in real-time. Very soon, we’ll be open sourcing this solution as Spiegel.

What do you think are the top three benefits of using CouchDB as a database solution?

  1. Multi-Master Sync. It is difficult to find another database that provides the same ability to distribute not just the read load, but also the write load. This is made possible by CouchDB’s conflict resolution policy, which is based on revision numbers. With CouchDB, I feel confident that as our user base grows we can horizontally scale just by adding a new node
  2. Offline Capable. CouchDB’s conflict resolution policy is great for offline-first applications. A lot of other databases use a timestamp or some form of a vector clock to implement a last-write-wins conflict resolution policy. These resolution policies are fine for online-only setups, but have serious drawbacks in offline applications. I’ve gained a deep appreciation for CouchDB’s revision numbers, especially after I wrote DeltaDB and suspended development due to only being able to achieve a “Reasonable Ordering.”
  3. Change Feeds. CouchDB’s feeds allow you to listen for changes. You can use these feeds in the frontend with PouchDB to sync data with your app, but you can also listen to these feeds in the backend and respond to certain changes in your data. This can allow you to process data in the background when it is modified directly via PouchDB.

What do you look forward to in the future of CouchDB?

I’m excited by the continual growth of the CouchDB community and that CouchDB now stacks up against some of the other modern databases. CouchDB has been a strong database for a while, but I suspect that more developers will incorporate it into their future designs now that CouchDB 2 supports clustering.

I look forward to the CouchDB team focusing more energy on compaction optimizations and other performance optimizations. A more performant CouchDB would make CouchDB easier to manage.

What advice do you have for someone who just discovered CouchDB?

Stick with it. Even with almost two decades of software development, I found that there was a steep learning curve adopting CouchDB. Most of my earlier work was with SQL databases, which are immediately consistent and therefore easy to use. Of course, this immediate consistency doesn’t typically scale well.

CouchDB, on the other hand, scales beautifully, but its data model often requires you to consider things from a new perspective and this can be frustrating and confusing. You need to consider this data model in every layer of your software and this is time consuming at first. The payoff is great though as once you become acquainted with CouchDB you’ll have an app that is truly capable of horizontally scaling.

 

For more about CouchDB visit couchdb.org or follow us on Twitter at @couchdb

Have a suggestion on what you’d like to hear about next on the CouchDB blog? Email us!

CouchDB takes Medic Mobile to the front lines of healthcare work

We were very excited for the chance to talk with Stefan du Fresne at Medic Mobile about how they have been employing CouchDB in applications.

Medic Mobile is a nonprofit technology company that was founded in 2010 to improve health in hard to reach communities. They serve their mission by designing, building, delivering, and supporting world-class software that helps community health workers, managers, and clinical teams work together to provide equitable care.

Their offices in Nairobi, Kathmandu, and San Francisco (and 75 staff) support more than 18,000 health workers in 23 countries, improving how health systems work for more than 11 million people.

Stefan was kind enough to share some specifics of their work with CouchDB.

How did you hear about CouchDB, and why did you choose to use it?

Many years ago CouchDB was picked for its master to master replication abilities, and resilience. We wanted to allow for multiple field deployments that worked well separately, and would push data upstream to an aggregator when network connectivity allowed.

Over time as our tools have evolved our use of CouchDB has grown. While many of our CHWs still send in data via SMS, CouchDB (with PouchDB) has allowed us to build a fully offline webapp that a growing number of our CHWs use.

More detail can be found in this blog post.

Did you have a specific problem that CouchDB solved?

We needed to be able to provide our application to our users in a consistent way, regardless of whether they have network connectivity. Seamless replication meant we didn’t need to worry about getting all data into one place and then relying on that place for functionality.

For the folks who are unsure of how they could use CouchDB–because there are a lot of databases out there—could you explain the use case?

Specifically for us, CouchDB (with PouchDB) allows us to have a fully functional offline first progressive web app that flawlessly synchronizes between a master datastore and any number of other datastores with almost no infrastructural effort. It is literally half a dozen lines of code to set up continuous replication.

More generally, CouchDB lets you push arbitrary JSON to a datastore that seamlessly replicates to other datastores whenever it is convenient to you. You can also combine it with PouchDB in the front-end as we have done, and you have a really low effort base for an offline-first web app.

What would you say is the top benefit of using CouchDB?

  • Simplicity. CouchDB can be accessed completely over REST. Being able to curl your way to database glory is really appealing. It makes it really easy to get started with a fresh DB, inspect an existing database, or quickly build up scripts/tooling around your data
  • Replication. This is a really hard problem, and CouchDB gives it you for free. It’s hard to understate how cool this is.
  • JSON. Because the database doesn’t enforce a schema, you can be very flexible in how you store and manage your data.

What tools are you using in addition for your infrastructure? Have you discovered anything that pairs well with CouchDB?

PouchDB is definitely a big one for us.

In general though, since CouchDB is exposed in a RESTian way whatever your favourite web service tool is (curl, httpie, insomnia, postman…) it will work great with CouchDB. And since the responses are all JSON, you can get surprisingly far debugging and mucking about with your databases with curl and jq (or even sed if you’re feeling adventurous).

We’ve been using Kanso, Gardener, Dashboard and Garden Market for deploying new versions of our software. These tools were great, but they have a lot going on within them, and a lot of their potential has not been realised over time. We’re currently building our cheekily named replacement Horticulturalist, which should reduce the amount of code we need to get deployments out the door, and generally give us more flexibility and stability in how we deploy our software.

We are also using couch2pg, a small utility to replicate data from a couchdb instance into PostgreSQL. This lets us build more complicated SQL based analytics where required.

What are your future plans with your project? Any cool plans or developments you want to promote?

We’re in a scaling phase, and expect to be supporting 200,000 health workers providing care for 100 million people by 2021. We believe community health workers have an important role to play in healthcare systems that reach everyone.

As we scale up the number of health workers using the software, we’ll also be adding to the support we provide to each health worker and family. Today, health workers use the app to diagnose and treat sick children, register and screen pregnancies and newborns, deliver modern contraceptives to advance family planning, help keep track of immunization visits, screen for malnutrition, and monitor for infectious disease outbreaks. In the near future, we’ll help address the burdens of HIV/AIDS, Tuberculosis, and non-communicable diseases.

We’re getting the offline-first app into the hands of more people on the community health team, including nurses and frontline managers. We’re also exploring ways for household caregivers to interact with and use the app as they provide care for their families, extending the health system even further.

Health systems and health workers want the newly-accessible data to guide their work – so, we’re planning to build risk profiles and targeting algorithms to answer questions like, “Which child is most likely to be sick?” Our goal is to improve coverage and equity, and we’ll release our code, processes, and learnings. Finally, the app will serve as an integration point for technologies that will help health workers deliver great care, including biometrics and low-cost diagnostics.

 

Use cases are a great avenue for sharing useful technical information. Please consider joining the fun! Additionally, if there’s something you’d like to see covered on the CouchDB blog, we would love to accommodate. Email us!

For more about CouchDB visit couchdb.org or follow us on Twitter at @couchdb