Introducing high availability for Scalingo MongoDB

In the last few months we worked really hard to improve our database addons offer. The InfluxDB database has been added to help people building applications around time series data (IoT, Analytics, etc.). The ergonomics of the database dashboard has been improved with some long-awaited features for PostgreSQL users like query statistics and monitoring. Lastly the MongoDB offers got improved with TLS encryption and Internet access to the database. The latest MongoDB 3.4 image is available as well.

All these small changes paved the way to a very important feature under the hood: MongoDB high availability using MongoDB replica sets.

What are replica sets?

Replica set is the way MongoDB implements high availability. According to the MongoDB documentation:

A replica set gathers different MongoDB servers into a cluster with different type of members:

• One primary instance which receives all the write operations. It also receives all the read operations by default.
• One or multiple secondary nodes which owns a copy of the data. These nodes can receive the read operations by modifying the read preference in your MongoDB driver (e.g. Mongoid). These secondary servers receives data modification in real time when the primary receives write operations.

Using a cluster provides redundancy and improved availability. Replica sets have been designed to resist server failures and netsplits. MongoDB 3.4 is even passing the Jepsen test. Know as one of the most rigorous challenge to pass for a distributed system.

Better Redundancy

In order to achieve a better redundancy, every mongod processes owns a copy of the data. This is made easy thanks to the oplog (operations log) feature.

The oplog is a special collection that keeps a record of all operations that modify the data stored in the database. MongoDB applies the write operation on the primary member of the replica set, then records the modifying operations on the primary's oplog. Finally, the secondary members copy and apply these operations.

Hence, the secondary members are always in sync with the primary.

Improved Availability

In order to achieve a better availability, MongoDB includes an automatic failover mechanism. At regular intervals, the different replica set members send a heartbeat to each other to ensure all members are alive. If a primary member is not reachable for 10 seconds, the secondary members hold an election in order to elect a new primary.

Hence, a replica set downtime is less than a minute, the time it takes for the secondaries to detect the failure and to elect a new primary!

For more information on how MongoDB handles replication, you can refer to the exhaustive official documentation.

How to take advantage of this feature?

This feature is available as part of new plans for your database. If you start a new database, you will be prompted to choose a plan. All the Business plans include replica set with three nodes holding the data, and one arbiter node.

Amongst the three nodes with a copy of the data: * one is used as a primary node, * one is used to make backups, * the last is a useful node in case of failover.

For an existing database, you can change the plan in the "Addons" section of your application and your database will migrate seamlessly from a single node to a replica set. Note that migration from a single node plan to a replica set plan causes a short downtime, due to the obligation to restart your database.

Necessary pricing change

As everybody doesn't need replica sets, nor can afford a high availability setup, Scalingo MongoDB pricing have been completely overhauled.

There's 3 pricing categories:

• Sandbox contains only 1 plan and should only be used for development purposes. No SLA is offered on this plan.
• Starter plans offers single node deployment (no replica set) but all of them includes automatic daily backups.
• Business plans are the ones you're looking for if you're doing production stuff: replica sets all the way and improved availability.

You'll find the new pricing grid below:

As of today, all old plans are obsolete and are not available for new add-ons provisioning. As a matter of fact, all Scalingo MongoDB add-ons running the old free tier plan will be discontinued on October 31st and automatically transformed into the new Sandbox plan.

If you have questions or remarks regarding the new pricing plans or the new replica set feature, don't hesitate to contact us. We'd love to hear your feedbacks!

What's coming up next?

We are really happy to strengthen our databases offer. With this first work on database high availability, you can expect some great news like cluster for Elasticsearch and hot standby for PostgreSQL in the next months.

In the same manner, you can expect encryption at rest for all our database add-ons.