Intermittent Bad Gateway Errors

Incident Report for Flatfile

Postmortem

Incident Overview

Date and Time of Incident: June 10, 2025 from approximately 1:50pm to 2:05pm PDT

Nature of Incident: Requests routing to an unavailable frontend pod

Services Affected: Flatfile Platform Dashboard and Spaces

Details of the Incident

At approximately 1:50pm PDT on June 10, the Flatfile Platform Dashboard reported intermittent 503 status codes. This was due to an infrastructure change made earlier in the day. As a consequence 1 in every 4 requests made to the Platform dashboard returned a HTTP 503 status code. The engineering team were alerted to the incident as soon as the 503’s started appearing and manually scaled up the infrastructure resources.

Impact Assessment

The incident was intermittent and affected 1 in 4 requests made by users of the Platform frontend applications for approximately 15 minutes. The API was unaffected.

Root Cause

The root cause was due to an infrastructure scaling configuration made in the early of the morning.

Resolution

Flatfile engineering manually provisioned extra capacity in order for the resources to get allocated appropriately. Following this, further improvements will be made with how we serve traffic to end users.

Security and Data Integrity

Please be assured that this incident did not compromise the security or integrity of your data. Our commitment to data protection remains a top priority.

Posted Jun 10, 2025 - 14:28 PDT

Resolved

This incident has been resolved.
Posted Jun 10, 2025 - 14:01 PDT

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Jun 10, 2025 - 13:57 PDT

Investigating

We are investigating an issue where some users are seeing bad gateway errors.
Posted Jun 10, 2025 - 13:52 PDT
This incident affected: Flatfile Platform (Spaces, Dashboard).