Routing issues impacting monitoring and availability
Incident Report for Flying Circus
Resolved
We touched base with HLkomm again and they communicated that everything is back to normal and they're wrapping up temporary changes. Things have been stable for for about 2 hours now and we're closing this incident now.
Posted Jul 20, 2020 - 21:17 CEST
Update
IPv4 and IPv6 from the HLkomm network are back again and appear to be stable. However, HLkomm informed us that they're still working on a last router, so we'll keep an eye on things and will try get another status update in about 1 hour.
Posted Jul 20, 2020 - 20:02 CEST
Identified
Just as we decided to update the statuspage IPv6 has was being dropped again and just came back. We consider the situation still volatile and will check back later with HLkomm. Unfortunately HLkomm does not provide any status updates.
Posted Jul 20, 2020 - 19:16 CEST
Monitoring
IPv4 has come back as well shortly after our last notice. Both protocols have been stable for around 20 minutes now, but we are keeping an eye on the situation.
Posted Jul 20, 2020 - 19:10 CEST
Update
We are seeing some progress. The connectivity via IPv6 has become available again. IPv4 is still affected.
Posted Jul 20, 2020 - 18:48 CEST
Identified
HLkomm has told us that they are experiencing a major routing issue.

Yesterday a fibre connection between Nürnberg and Bamberg was damaged and repaired around 16:30 today. According to HLkomm their routers did not cope properly with the traffic changes that happened when the fibre came back up. They are working on the issue but have no further details at the moment.
Posted Jul 20, 2020 - 17:58 CEST
Update
The issue has popped up again. We are still trying to reach the HLkomm NOC which appears to be overloaded at the moment.
Posted Jul 20, 2020 - 17:35 CEST
Update
We are continuing to investigate this issue.
Posted Jul 20, 2020 - 17:23 CEST
Investigating
Since 16:40 we are experiencing package loss between HLKomm and KAMP networks. It appears to be a routing issue in the HLKomm backbone, but we have also seen customers from KabelDeutschland to be affected.

Since 17:20 we see services resuming but still seeing odd effects here and there.

KAMP has indicated no known issues at the moment and we are investigating with HLKomm at this point.
Posted Jul 20, 2020 - 17:23 CEST
This incident affected: Network and Internet uplink.