Contact lists and search results failing to load intermittently
Incident Report for Spruce Health
As of 12:55pm PT on October 25 2022, we started to see recovery and found the system to return to being fully operational. There was a re-allocation of data across the nodes that store and process the search engine data. The re-allocation seems to have helped the overall recovery.

We will continue to investigate ways in which we can further optimize our search engine database to prevent such spikes in search latency.
Posted Oct 25, 2022 - 12:55 PDT
Note that while we are experiencing this performance issue, new contacts, updates to contacts and messages may not be reflected in search results. This is because there is a backup in indexing of new data into the search engine.

Also note that failures may be intermittent. Contact lists may fail to load but then if you refresh the page it may end up loading. This will also be the experience with conversation and contact search results.
Posted Oct 25, 2022 - 12:17 PDT
The performance issues seem to have started again as of 11:02am PT today, despite the infrastructure improvements. We are continuing to investigate to identify an performance bottlenecks.
Posted Oct 25, 2022 - 11:17 PDT
We have made some underlying infrastructure changes that should help alleviate the performance issues that we were experiencing. We will continue to monitor the overall health of the database that backs searches and contact lists during normal business operations today (October 25 2022) to ensure that the issue is fully resolved and the platform is back to being functional.
Posted Oct 25, 2022 - 00:49 PDT
We are currently investigating an issue where contact lists and search results fail to load intermittently.

There should be no impact to the Spruce inbox, inbound/outbound calls, video calls or exchanging of messages over SMS, secure messaging, email and fax.
Posted Oct 24, 2022 - 12:40 PDT
This incident affected: Web App and Mobile Apps.