External DNS lookup issues

Incident Report for Flying Circus

Resolved

The situation has stabilized since we switched to direct authoritative resolution. We are in touch with Quad9 and will implement further measures at a later time.

Posted Dec 17, 2024 - 11:43 CET

Monitoring

Starting at around 3:15 CET we have seen increased errors on external DNS lookups.

Queries to our preferred upstream DNS provider started timing out. The issue improved at around 5:30 CET but has still been latently showing errors.

We have switched external DNS lookups away from Quad9, using direct authoritative access as a temporary countermeasure.

Posted Dec 17, 2024 - 08:39 CET

This incident affected: Central services and RZOB (production) (Network and Internet uplink).