Cloudflare CTO Dane Knecht took to social media to address the widespread outage that disrupted services across the internet.
Cloudflare, which describes itself as “one of the world’s largest networks,” acknowledged the disruption and issued an apology.
Knecht said, “I won’t mince words: earlier today we failed our customers and the broader Internet when a problem in @Cloudflare network impacted large amounts of traffic that rely on us. The sites, businesses, and organizations that rely on Cloudflare depend on us being available and I apologize for the impact that we caused.”
Explaining the cause, he added, “Transparency about what happened matters, and we plan to share a breakdown with more details in a few hours. In short, a latent bug in a service underpinning our bot mitigation capability started to crash after a routine configuration change we made. That cascaded into a broad degradation to our network and other services. This was not an attack.”
Knecht said the team is already working to prevent such an incident from recurring. “That issue, impact it caused, and time to resolution is unacceptable. Work is already underway to make sure it does not happen again, but I know it caused real pain today. The trust our customers place in us is what we value the most and we are going to do what it takes to earn that back.”
Update on the outage
Cloudflare later announced that the issue had been fixed. “A fix has been implemented and we believe the incident is now resolved. We are continuing to monitor for errors to ensure all services are back to normal,” the company said.
It added that some users may still experience lingering issues.
)