The Things Network
Update - In order to increase stability of our routing services, we also had to disable publishing uplink fields to MQTT in our US-West region. Applications can still subscribe to [AppID]/devices/[DevID]/up and read the payload_fields field from the uplinks published on that topic.
Apr 2, 09:23 CEST
Identified - For performance reasons we unfortunately had to disable publishing Uplink Fields to MQTT in our EU region. This means that the values from payload decodes are no longer published to individual [AppID]/devices/[DevID]/up/[Field] topics. Until this is resolved, you can subscribe to [AppID]/devices/[DevID]/up and read the payload_fields field from the uplinks you receive there.
Nov 18, 11:00 CET
Identified - As a result of flaky data streams, the information in the NOC component may be unreliable. As a result, gateways may appear in the console as offline, while they are in fact online. Gateway traffic in the console may not work. Application traffic is not affected by this.
Jan 27, 13:56 CET

About This Site

This page shows the current status of The Things Network services that are operated by the core team of The Things Network Foundation.

The metrics on this page are updated continuously. Component statuses and incidents are still updated manually while we work on automating these too.

Operations and incidents can be discussed in the #ops channel on Slack.

Global Services Operational
Website (www.thethingsnetwork.org) Operational
Forum (www.thethingsnetwork.org/forum) Operational
Account Server (account.thethingsnetwork.org) Operational
The Things Stack Community Edition (a.k.a. V3) Operational
Europe 1 (eu1.cloud.thethings.network) Operational
North America 1 (nam1.cloud.thethings.network) Operational
Australia 1 (au1.cloud.thethings.network) Operational
V3 Integrations and Third Party Services Operational
Packet Broker Integration Operational
The Things Network v2 Operational
Cluster Discovery Operational
Console (console.thethingsnetwork.org) Operational
Europe ? Operational
United States - West ? Operational
Asia-Southeast ? Operational
Brazil ? Operational
Cross-Region Traffic Exchange Operational
NOC (noc.thethingsnetwork.org) ? Operational
V2 Integrations Operational
AllThingsTalk Operational
Cayenne Operational
Collos Operational
Data Storage Operational
EVRYTHNG Operational
HTTP Webhook Operational
IFTTT Operational
OpenSensors Operational
TTN Mapper Operational
TagoIO Operational
Ubidots Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Connected Gateways (v2) ?
Fetching
Received Uplink Messages (v2) ?
Fetching
Delivered Uplink Messages (v2) ?
Fetching
Connected Gateways (v3) ?
Fetching
Received Uplink Messages (v3, from gateways) ?
Fetching
Received Uplink Messages (v3, from Packet Broker) ?
Fetching
Delivered Uplink Messages (v3) ?
Fetching
Past Incidents
May 8, 2021

No incidents reported today.

May 7, 2021

No incidents reported.

May 6, 2021

No incidents reported.

May 5, 2021

No incidents reported.

May 4, 2021
Resolved - This incident has been resolved.
May 4, 10:22 CEST
Update - Our metrics show that traffic is flowing again. We have re-enabled the MQTT API and are monitoring the system.
May 4, 10:01 CEST
Monitoring - We discovered that the router is the failing component at the beginning of the investigation. We recovered the router quite easily, but the handler crashed (similar to a previous outage) and took a while to recover.
We have temporarily disabled MQTT ingress and are monitoring the situation.
May 4, 09:28 CEST
Investigating - We are investigating an outage in the TTN V2 Cluster
May 4, 08:41 CEST
May 3, 2021
Completed - The scheduled maintenance has been completed.
May 3, 15:00 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 3, 13:00 CEST
Scheduled - During this maintenance window we will upgrade The Things Network v3.12.2

We do not expect noticeable downtime during this deployment.

Here is the changelog since the current version v3.12.1:

### Added

- Contextual tooltips to form fields in the Console.
- C-Style uint32_t representation for end device address field.
- Gateway Configuration Server to the cluster package.
- This introduces a new config option `cluster.gateway-configuration-server` that needs to be set in multi-instance deployments.
- Uplink storage for integrations in the Application Server. The number of uplinks stored per end device may be configured via the config option `as.uplink-storage.limit`.
- LoRaCloud GLS multi frame request support.
- LoRaCloud GNSS request support.
- LoRaCloud WiFi request support.

### Changed

- Allow the LinkADRReq commands to lower the data rate used by the end devices.

### Fixed

- Occasional crashes in the ratelimit middleware.
- Handling of zero EUI CUPS update-info requests.
- Backend validation messages for some forms.
- Gateway downlink message previews not displaying correctly in the event view of the Console.
- Importing end devices from the Console would occasionally ignore some device MAC settings fields.
Apr 30, 13:38 CEST
May 2, 2021

No incidents reported.

May 1, 2021

No incidents reported.

Apr 30, 2021

No incidents reported.

Apr 29, 2021
Resolved - This incident has been resolved.
Apr 29, 15:57 CEST
Monitoring - We implemented a fix for the issues in our European v2 cluster and re-enabled the MQTT API. We will keep an eye on it and will take additional action if needed.
Apr 29, 14:58 CEST
Identified - We have identified the handler component of our European v2 cluster (and specifically the MQTT integration) as the culprit in this outage.

While we continue to investigate the root cause, we have blocked MQTT connections to our European v2 cluster and restarted the other services. Device management and integrations such as the HTTP (webhook) integration, the storage integration and integrations to external platforms are now back online.
Apr 29, 13:18 CEST
Update - We are still trying to get our European v2 cluster back online.
Apr 29, 10:56 CEST
Investigating - We are investigating an outage in our European v2 cluster.
Apr 29, 09:42 CEST
Apr 28, 2021

No incidents reported.

Apr 27, 2021

No incidents reported.

Apr 26, 2021

No incidents reported.

Apr 25, 2021

No incidents reported.

Apr 24, 2021

No incidents reported.