Airflow ORANGE Azure is not available due to "Internal Server Error"

Incident Report for Mayan

Resolved

All DAGs now available on Production-GREEN
Posted Jan 23, 2024 - 17:18 UTC

Update

Although the server is still down, we have been running workflows locally to avoid service disruptions.
Posted Jan 23, 2024 - 12:51 UTC

Identified

The issue lies with an internal database that supports the Airflow Instance login session credentials. Until Microsoft is able to address this, we will likely see a continuing failure of this instance.
We have asked them to address the issue but at their scale this takes days to resolve.
Posted Jan 22, 2024 - 17:42 UTC

Update

Microsoft appears to have some internal issues on Azure
Will continue to monitor and update as new information becomes available.
Posted Jan 22, 2024 - 16:31 UTC

Investigating

We are currently investigating this issue.
Posted Jan 22, 2024 - 16:29 UTC
This incident affected: Automations and Data Ingestion (Airflow (batch)).