European V2 Cluster Outage

Incident Report for The Things Network

Resolved

This incident has been resolved.
Posted Apr 29, 2021 - 15:57 CEST

Monitoring

We implemented a fix for the issues in our European v2 cluster and re-enabled the MQTT API. We will keep an eye on it and will take additional action if needed.
Posted Apr 29, 2021 - 14:58 CEST

Identified

We have identified the handler component of our European v2 cluster (and specifically the MQTT integration) as the culprit in this outage.

While we continue to investigate the root cause, we have blocked MQTT connections to our European v2 cluster and restarted the other services. Device management and integrations such as the HTTP (webhook) integration, the storage integration and integrations to external platforms are now back online.
Posted Apr 29, 2021 - 13:18 CEST

Update

We are still trying to get our European v2 cluster back online.
Posted Apr 29, 2021 - 10:56 CEST

Investigating

We are investigating an outage in our European v2 cluster.
Posted Apr 29, 2021 - 09:42 CEST