Flying Circus

All Systems Operational

About This Site

This is the global, public status page of the Flying Circus Internet Operations GmbH


Privacy Policy

RZOB (production) Operational
90 days ago
99.97 % uptime
Today
VM servers Operational
90 days ago
100.0 % uptime
Today
VM storage cluster Operational
90 days ago
99.9 % uptime
Today
VM backup ? Operational
90 days ago
100.0 % uptime
Today
Network and Internet uplink ? Operational
90 days ago
99.98 % uptime
Today
Central services ? Operational
90 days ago
100.0 % uptime
Today
my.flyingcircus.io ? Operational
90 days ago
100.0 % uptime
Today
gitlab.flyingcircus.io ? Operational
90 days ago
100.0 % uptime
Today
Emergency Channels Operational
90 days ago
100.0 % uptime
Today
Hotline ? Operational
90 days ago
100.0 % uptime
Today
Support emails ? Operational
90 days ago
100.0 % uptime
Today
RZHAL (backup and development data center) Operational
90 days ago
100.0 % uptime
Today
VM servers Operational
90 days ago
100.0 % uptime
Today
VM storage cluster Operational
90 days ago
100.0 % uptime
Today
VM backup Operational
90 days ago
100.0 % uptime
Today
Network and Internet uplink Operational
90 days ago
100.0 % uptime
Today
Related external services Operational
Bitbucket Git via HTTPS Operational
Bitbucket Mercurial via HTTPS Operational
Bitbucket SSH Operational
GitHub Operational
Fastly Europe (FRA) Operational
Fastly Europe (AMS) Operational
Python Infrastructure pypy.org Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.
Jul 9, 2025
Resolved - We have solved the issue which manifested sporadically in two ways:

1. DNS queries that required our border routers to retrieve data from the internet timed out
2. Traffic from VMs to the internet that required NAT timed out

The issue was caused by a new transfer network that was introduced in the maintenance earlier today. It was generally carrying traffic to our data center, but traffic destined for addresses on the network itself was lost.

Our routers dynamically used transfer net addresses when sending traffic to the outside world (i.e. for DNS and NAT as described above). To resolve the issue we have changed our router configs to use source addresses from our networks which are properly routed.

We've tested the configuration in the redundant configuration using fail-overs and have seen the issues on affected machines subside.

We're sorry for the outage.

Jul 9, 00:07 CEST
Update - We have achieved a semi-stable routing state again, where connectivity has mainly been restored.
DNS resolution within RZOB is currently not stable.

Jul 8, 23:16 CEST
Investigating - Unfortunately, our triaging attempt caused additional connectivity issues, now also via IPv6. These issues only affect some machines, but the underlying cause is not identified yet.
Jul 8, 23:10 CEST
Identified - Due to the changes in the announced RZOB network uplink maintenance, the RZOB data centre lost some of its IPv4 connectivity.

We are triaging this by reverting the routing changes to the state before the maintenance right now to restore IPv4 connectivity.

Note: The revert itself will also briefly interrupt the data centre connectivity again.

Jul 8, 23:05 CEST
Jul 8, 2025
Completed - We have completed this maintenance.
Jul 8, 21:46 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 8, 21:00 CEST
Scheduled - We will be performing maintenance on the internet uplink in RZOB, our production datacentre, in cooperation with our provider in order to complete a pending internal migration. We will be monitoring the situation for any unexpected impact to customer services, in order to respond promptly should problems arise.
Jul 1, 08:45 CEST
Jul 7, 2025
Completed - The scheduled maintenance has been completed.
Jul 7, 22:00 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 7, 21:00 CEST
Scheduled - Release 2025_023 is ready and will be rolled out during the specified timeframe.

See the Changelog for information about the specific changes and impact of the release.

Note: releases themselves do not cause downtimes. If a change within a release requires downtime then every VM that is affected will schedule a maintenance period according to your projects' preferences and notify you with specific details about the downtime.

Check the "Impact" section of the changelog (linked above) to see potential causes of downtimes in a release.

Jul 3, 18:45 CEST
Jul 6, 2025

No incidents reported.

Jul 5, 2025

No incidents reported.

Jul 4, 2025

No incidents reported.

Jul 3, 2025

No incidents reported.

Jul 2, 2025

No incidents reported.

Jul 1, 2025

No incidents reported.

Jun 30, 2025
Completed - The scheduled maintenance has been completed.
Jun 30, 22:00 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jun 30, 21:00 CEST
Scheduled - Release 2025_021 is ready and will be rolled out during the specified timeframe.

See the Changelog for information about the specific changes and impact of the release.

Note: releases themselves do not cause downtimes. If a change within a release requires downtime then every VM that is affected will schedule a maintenance period according to your projects' preferences and notify you with specific details about the downtime.

Check the "Impact" section of the changelog (linked above) to see potential causes of downtimes in a release.

Jun 26, 16:38 CEST
Jun 29, 2025

No incidents reported.

Jun 28, 2025

No incidents reported.

Jun 27, 2025

No incidents reported.

Jun 26, 2025
Completed - The scheduled maintenance has been completed.
Jun 26, 20:30 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jun 26, 19:30 CEST
Scheduled - We will apply software updates to our main router in Halle. Our office and backup data center will be down for maintenance for a few minutes. We will subdue offsite monitoring for the period of the maintenance. Some secondary service may be unreachable for a few minutes (e.g. Gitlab, Youtrack, ...).

Customer production services will not be affected.

Jun 25, 12:07 CEST
Jun 25, 2025

No incidents reported.