A few VMs are experiencing crashes
Incident Report for Flying Circus
Resolved
We haven't seen crashed VMs in about 24 hours after migrating workloads away. We're following up with investigation of the affected KVM host but are closing the incident on the status page now.
Posted Jan 31, 2019 - 19:57 CET
Monitoring
After evacuating production workloads from the affected servers no further VMs have crashed. We are still trying to recreate the issue with non-production workloads and are keeping a close eye on the situation.
Posted Jan 31, 2019 - 07:32 CET
Update
The rate of VM crashes remains very low (maybe 1 every few hours).

We have created additional instrumentation to help us in debugging the issue and are now evacuation production VMs from the affected virtualisation host. We will keep you updated once further crashes show up on our instrumentation.
Posted Jan 30, 2019 - 17:56 CET
Investigating
We have seen a small number of VMs crashing on a single over the last hours without a visible reason and are investigating the situation.

At the moment we are working to implement additional instrumentation and will evacuate the affected host from production workloads soon.
Posted Jan 30, 2019 - 15:59 CET
This incident affected: RZOB (production) (VM servers).