In an effort to improve the overall stability of the platform, we have been working on separating database servers for each of the core components that make up the Spruce platform. This will help ensure that the entire platform does not become unavailable if the database server is having an issue, like it did during the incident on Jan 14 (https://status.sprucehealth.com/incidents/b9nhkvv3hrhk
). It will also help us scale the database for each component independent of the others, as well as scale the platform over time.
In preparation for this, we have already migrated three components that have no critical role in powering the Spruce inbox or the inbound communications flow. Now that those migrations have gone smoothly and we’re feeling well prepared, we’d like to focus on the core components that the Spruce inbox and inbound communications depend on.
Our most recent 14-day analysis of communications across our customer base indicates that the 3-5 am EST window is the best time period during which to conduct this migration. So, we will be conducting the migration across two time windows: Thursday, Feb 28, 3-5 am EST, and Sunday, March 3, 3-5 am EST. We will try our best to complete the migrations in a single window on Feb 28 to reduce the potential impact on your business. We will be updating this page on the progress of the migration, so feel free to subscribe to the page for updates. This page will persist after the incident is complete, so you can review them after the fact as well.
During the maintenance windows, the Spruce inbox and inbound communications (calls and SMS texting) may become unavailable for up to one minute at a time no more than four times during each window (we plan to migrate two databases in each window, but if all goes smoothly, we will try to migrate all four databases in a single window). We will update this page with exact periods of unavailability so that you can stay informed.
If any of your patients call during one of the brief outage periods, they will hear a message that informs them that the system is undergoing scheduled maintenance and that asks them to try again in a few minutes, or to dial 911 if it’s an emergency. If any of your patients sends you an SMS text during an outage period, the message will be delivered to your inbox once the system has been restored and we have identified the delayed inbound messages, which we project to take no more than 30 minutes after system restoration. Please note that informing you of the delayed calls and SMS texts will be a manual process, but we will do our absolute best to deliver them to you as soon as is feasible.
If we are unable to complete any of the database migrations, we will abort the migration attempt and continue using the current database server. In this worst-case scenario, the total period of unavailability will be less than five minutes.
I know this is a lot of information, so please let us know if you have any questions at all by emailing us at firstname.lastname@example.org
, and I’ll be happy to answer them.