Project Storky: Automate App And Data Migration Between Regions

While Scalingo is now available in multiple datacenters (called regions) and as we're planning to open more regions in the future, our customers felt the need to have the tools to migrate their data and apps from one region to another. This is why "Project Storky" was born. We are happy to announce today that this project has come to fruition: you have now the possibility to migrate apps from one region to another by yourself.

Storky in Action

Storky is the internal name of the service in charge of handling the migration of applications and their data between regions. It is a reference to the stork, the emblem animal of Scalingo's hometown region. This animal is known for its ability to deliver new babies at the new parents' doorstep (yes, this is how babies are made around here!). This task requires great care as the package a stork carries is delicate! This is how we see your applications' migration: a tough task which requires great care to handle a package made of many moving pieces.

With Storky's release, your application migrations are in safe hands (or should I say "beak"!). Follow us in this guide about how you can concretely migrate your applications from one region to another.

Migrating an Application

Migrating an application with our CLI from one region to another is completely automatic. The actual process is divided into 4 independent steps. Each of them can be cancelled at any time.

1. Creation of the migration: where we check that the migration is doable; this step returns a migration ID that will be required to run the next steps or to abort the migration.
2. Preparing the migration: where we copy the skeleton (the app, the collaborators, its configuration).
3. Migrating the data: this step is optionnal. All databases are supported except InfluxDB.
4. Finalizing the migration: where we start the new app and redirect the traffic from the old app.

During step 1 and 2 the source application will still work, will still receive traffic . Only its settings will be frozen to avoid platform interactions that could affect the migration.

It's only during step 3 and 4 that the source application is stopped. That means that the expected and unavoidable downtime for a migration is the duration of the data backup from the source database and data restoration into the target database.

You can find all the technical details about each step and how to run them in the “Application Migration Between Regions” documentation page.

What Can Be Migrated?

Our migration tool can handle the complete migration of an application from one region to another!

To be more specific, it will:

• replicate the container formation (number, type and size of your containers' fleet)
• copy the advanced settings (canonical domain, force HTTPS, sticky sessions)
• import the application soft limits (such as the image max size)
• import the build that is currently running on the source application (the latest successful deployment)
• import the environment variables
• import the collaborators that have accepted the invitation (pending collaborators are not imported)
• import the SCM integration (GitHub, GitLab, etc.)
• import the domains and their certificates (both from Let’s Encrypt and manually added ones)
• import the notifiers and the configured alerts
• import the log drains
• import the autoscalers

Storky can also migrate your data.

Here are the steps followed by Storky for each of your provisioned addons:

• provision a database with the same plan and version
• trigger a backup on the source database
• once those two operations are done, it will restore the backup on the target database

Shortcomings

While our initial goal with Project Storky was to provide an all-in-one, completely automatic migration tool, it has a few shortcomings.

Log drains on databases and custom MySQL configuration won't be migrated. If you're concerned, ask the support to enable them on your target application.

If you're using a git remote repository on Scalingo (i.e. if you're not using our GitHub or GitLab integration), this repo won't be migrated. That's not really a problem. You will be able to launch git push on the target repo.

Migration pre-flight checks will raise errors if you use any of the following: TCP add-on, VPN add-on, Docker add-on or InfluxDB add-on. In that case you can contact the support for further assistance.

Conclusion

Scalingo's goal is to automate as many things as possible to ease a developer's life. The availability of this tool is a new mandatory, step in this direction. With Scalingo's plans to expand in new regions during the year 2020, this tool will be more and more important for our customers.