The data center suffered a 120 minute outage today when a back-bone storage server underwent an unplanned restart. The incident was caused when one of our network administrators was troubleshooting an unrelated problem. We have put immediate safeguards to prevent an incident like this from re-occuring.
This type of restart affects systems throughout the farm, which will generally become available again within 20 minutes. However, during the reboot we discovered data inconsistencies on the storage system. As a measure to protect and safeguard your data, we immediately proceeded to run through consistency and integrity checks on all client data. That process takes a substantial amount of time, but is critical to ensuring ongoing up-time and data integrity. After the routines finished and all errors were cleared, we put the system back into production.
We apologize for the iconvenience this caused and the additional delay caused by the maintenance routines. We take every precaution possible to ensure your data is safe.