Elation changed the error code being returned to indicate expired credentials. Spruce had to update the error code they were looking for to refresh credentials before making an API request. Spruce was notified of this issue due to a customer complaint, 10 hours after the issue started. Spruce identified that there was no code deployment on their end, so engaged Elation to help investigate.
Action items to mitigate the impact in the future
Spruce to put in a fix to resolve the issue (done)
On Elation end (not in our control so cannot say if will be done or not, but Spruce has communicated with Elation on these):
Elation to consider reverting the error code change
Elation to consider improved error messages if credentials have expired
Spruce to have a quicker escalation path with Elation (Elation shared email addresses that Spruce should reach out to, in addition to Slack)
Spruce to investigate how they can get proactively notified of elevated integration related error
Posted Oct 12, 2022 - 11:24 PDT
Resolved
We deployed a fix at 11:08 am PT Oct 12 2022 and can confirm as of 11:15am PT that the Spruce-Elation integration is now fully functional again.
Posted Oct 12, 2022 - 11:16 PDT
Identified
We have identified the cause to be a deployment by the Elation development team last night (around 9:40pm PT Oct 11 2022) where there was a change in assumptions for when an API partner (such as Spruce) should re-authenticate credentials for an integration. Consequently, most integrations were failing given that Spruce was not refreshing credentials when it should.
We are making a change to adapt to the updated assumptions, while the Elation team is evaluating if they should revert back to previous assumptions.
We should have a fix rolled out in the next 30 minutes.
Posted Oct 12, 2022 - 09:45 PDT
Investigating
We are currently experiencing an issue where the Spruce-Elation integration is erroring. Any patient a provider adds to Elation may not automatically show up in Spruce, if that's what you expect. Any demographic changes made to a patient in Spruce or Elation will not sync to Elation or Spruce. Any attempt to sync messages from Spruce to Elation will not work.
We are investigating this issue in partnership with Elation as we speak to root cause it.