Infrastructure on The Coders Blog

When DNSSEC Goes Wrong: Responding to the .de TLD Outage

Wed, 06 May 2026 22:22:12 +0000

Millions of .de domains vanished from the internet on May 5, 2026, not due to a sophisticated attack, but a seemingly routine DNSSEC key rotation gone awry. DENIC, the registry for Germany’s country-code top-level domain, inadvertently published incorrect DNSSEC signatures, triggering widespread SERVFAIL errors on validating resolvers worldwide. For users of services like Cloudflare’s 1.1.1.1, this meant the .de TLD effectively ceased to exist for several agonizing hours.

The Core Problem: Broken Signatures, Broken Resolution

The incident stemmed from a faulty Zone Signing Key (ZSK) rotation. During this process, DENIC’s system introduced malformed RRSIG records for the .de zone. Specifically, the ZSK tag 33834 was found on an NSEC3 record, a configuration that, when combined with other factors in the validation chain, broke the cryptographic trust model. When a validating resolver queried for a .de domain, it received these flawed signatures, leading it to conclude the DNS data was untrustworthy and respond with SERVFAIL. This “fail-closed” nature of DNSSEC, while intended to prevent spoofing, directly translated operational errors into complete service unavailability.

AI's Thirsty Truth: Why Its Water Footprint Isn't What You Think [2026]

Fri, 01 May 2026 21:27:09 +0000

Forget the ‘gallons per ChatGPT query’ headlines; that’s not where AI’s real water challenge lies. As senior engineers, we need to talk about the system, the infrastructure, and the optimizations that truly define AI’s water footprint by 2026.

The Core Misconception: Why ‘Gallons Per Query’ is a Distraction

The media loves a catchy, easily digestible metric. “X gallons per ChatGPT query” is precisely that – and it’s fundamentally misleading. This pervasive, oversimplified narrative fails to capture the true water demands of modern AI. It’s akin to measuring the fuel efficiency of a car by the amount of gasoline used for a single brake press.

Ubuntu Infrastructure Down: A Critical Cross-Border Cyberattack Exposes Core Weaknesses

Fri, 01 May 2026 21:17:20 +0000

On May 1st, 2026, the digital heartbeat of Ubuntu.com, the Snap Store, and Launchpad faltered under a declared cyberattack, plunging essential services into darkness. This wasn’t merely a fleeting outage; it was a sustained, cross-border assault that brought into sharp relief the vulnerabilities inherent even in the foundational components of our digital world.

Canonical’s web infrastructure, including critical services like login.ubuntu.com and essential Ubuntu Security APIs for CVEs and notices, became largely unresponsive. While mirror sites and the main Ubuntu archive largely continued to serve apt update requests, the impact on developer workflows and trust was immediate and severe. This incident should serve as a critical wake-up call for every organization relying on open-source ecosystems.

[Security Breakdown]: Ubuntu's 15+ Hour DDoS - Lessons for Every Developer [2026]

Fri, 01 May 2026 11:21:29 +0000

April 30, 2026: 6 PM UK time. Ubuntu’s core services, the very bedrock for millions of developers, started crumbling under a sustained DDoS assault. This wasn’t just a hiccup; it was a 15+ hour security breakdown, a stark reminder that even the giants can be brought to their knees. This incident isn’t merely a cautionary tale for Canonical; it’s a blueprint for understanding and hardening your own defenses against the inevitable.

Cloudflare Outage Disrupts X, ChatGPT, Downdetector: What Happened and Resilience Lessons

Tue, 18 Nov 2025 00:00:00 +0000

Disclaimer: This post summarizes publicly available status-page and press report information as of publication. Root cause analysis (RCA) has not yet been published at the time of writing; therefore speculative explanations are avoided.

1. Human Context – Why This Felt Big

Early on November 18, 2025 (UTC morning, mid‑morning ET) users attempting to reach high‑traffic destinations such as X (formerly Twitter) and ChatGPT encountered challenge failures and generic connectivity / 5xx style errors. Even downtime tracking platform Downdetector briefly showed disruption, creating a recursive reliability moment: when the monitoring site is also impaired, user anxiety escalates.