CYPHER for Business portal issues

Incident Report for CYPHER Learning

Resolved

There were a very large number (1m+) orders associated with the "visitor" account that must have accumulated over the past year due to users not fulfilling orders. Today our caching code that caches orders for a user was triggered for the visitor account, resulting on heavy prolonged loading of those 1m accumulated orders. It took at around 45m for us to determine the root cause, since this was a novel edge case that we have never seen before. We then patched our code to ignore these specific calls for the visitor account, and then the site went back to normal again. The longer-term solution which will we will implement shortly is to periodically flush orders that accumulate for the visitor account.

Posted Aug 13, 2024 - 16:29 UTC

Investigating

Some CYPHER for Business users are experiencing issues in their portal. The team exploring now as a top priority.

Posted Aug 13, 2024 - 14:48 UTC

This incident affected: CYPHER for business.