26 June 2015

Data Residency Challenges

I recently ran into an issue with a data residency ruling and thought I would share my thoughts here.  I am working with a UK client for who we are building a brand new enterprise application.  The application is hosted on a combination of IBM's Bluemix and SoftLayer platforms.

We are using Bluemix Dedicated which allows us to choose a SoftLayer location for the system and associated data.  Our requirements were that client data must reside in the EU and nobody outside of the EU is to have access to the data.  Pretty standard.  Therefore we chose London as our prime data centre and Paris as our disaster recovery site.

During the latter stages of the solution design work the client came to us to ask if we could keep a small subset of the data resident in the UK - i.e. don't replicate it to Paris.

This presented us with an issue:

  • Bluemix is a cloud solution and only works in SoftLayer cloud data centres
  • There is only one UK SoftLayer data centre
  • Our solution was built around Bluemix and so moving away from Bluemix would be a very big deal


We considered a number of different scenarios, including approaching SoftLayer to see if they would build us a new data centre!  In the end, the simplest solution is what we went for.

Bluemix uses an IBM database technology called Cloudant to store data.  Cloudant is based on Apache CouchDB and retains its API and interfaces.  After some Googling I discovered a blog about Cloudant replication that explained that it is possible to replicate from Cloudant to any database that supports the API; CouchDB included.

As the data to retain in the UK was a very very small subset of the total data it became viable for us to standup what I call "Bluemix-on-a-box" standalone servers in one of our other UK data centres (the solution had additional non-cloud DCs for payment processing) that run WebSphere Liberty and CouchDB.  It is then a case of changing the replication of the instances of Cloudant that needed to be replicated to the UK.

Here is a pic:
Data Residency Solution Overview