Post incident report:

Start: 16-January-2022 0600 UTC
End: 19-January-2022 1452 UTC
Duration: 3 days

Summary:

At 0600 UTC on 16 January 2022, following maintenance a configuration issue resulted in issues for some queries.  This resulted in VIP spoofing for some users to not work consistently until the issue was caught and rectified.

Root Cause:

During the rollback of the initial attempt to transfer to a new database system an internal configuration was missed causing some of the queries to fail sporadically.

Customer Impact:

Incoming customer emails during this period may not have been properly checked against the VIP Spoofing list and therefore were not all flagged correctly.

Mitigation Action:

The missed configuration that resulted in the failure was corrected.

Follow-up Items and Preventative Measures:

1)      A full check of internal configurations will be made after any changes to those records are required for changes and/or rollbacks to changes.