WundertreOS - Increased Errors on HTTP Endpoints – Incident details

All metrics have been nominal and stable for several hours now, and our upstream provider has resolved their incident.

We've received confirmation from our networking provider that there is recovery across all regions. We are able to corroborate this with our own metrics collections. We will continue to monitor for a bit, but we believe this to be resolved at this time.

We apologize for the trouble, and we appreciate your patience.

We've seen a marked improvement in error rates across all regions in the last few minutes. We are still waiting on confirmation from our provider on status, but we believe things to be improved at the moment. We'll keep working on this until we have for sure reached a resolution.

We recognize that some of our users, especially in parts of North and South America, are continuing to experience DNS and 5XX errors when trying to connect to their projects. We have escalated this issue with our network provider and will work with them continuously until the problems are resolved.

Some users continue to experience errors. We are raising the issue with our provider.

Our upstream provider has reported their incident resolved. We will continue to monitor for any errors our customers may be experiencing for another 30 minutes.

The fix that our upstream provider implemented has taken effect. Service has significantly improved across all previously impacted regions. There are still residual elevated error rates in a handful of regions and we will continue to work with our provider to eliminate them.

Our upstream provider has rolled the fix out to most regions. While we are seeing improvement, we are still seeing increased error rates in Mexico and Brazil and are continuing to work with them until this is fully resolved.

Upstream provider mitigations are still under way, and we are seeing improvements incrementally as routing regions have the mitigations applied. We are still seeing particular impact in South America, but some users in North America may still be affected as well. We anticipate continued incremental improvements and will update as we get more information.

Our upstream networking provider has identified a fix and is currently implementing a fix. We are seeing preliminary improvements to error rates for users in North America, but are still following this issue closely with our provider.

Our upstream networking partner is continuing their investigation. We will continue to post regular updates as we have more information.

Our upstream networking provider has discovered an issue and has declared an incident on their side. They are currently working toward resolution. We are actively working with them and will provide updates on progress as more information is available.

We continue to have active lines of investigation going with our upstream network provider partners, and are still working toward a resolution.

This issue affects the network-level access to projects. The projects themselves and the data in them are safe and unaffected.

Users are seeing two separate symptoms of these networking issues:

1. DNS Lookup failures for supabase project URLs
2. 530 responses to HTTP requests to supabase project API Endpoints

We continue to work with partners and investigate.

We have further clarified the scope and impact of the issues currently affecting users. A subset of users across North and South America are experiencing DNS resolution failures and HTTP 530 errors.
These symptoms indicate a networking-related issue impacting the availability of Supabase projects. We are actively working with both internal teams and external network providers to isolate the root cause and determine the most effective path to resolution.
We will continue to provide updates as more information becomes available.

We are actively investigating this issue and working with relevant partner organizations. At this time, we do not believe this issue is specific to any particular network provider. A subset of users continue to be impacted in both South and North America.

We are currently investigating reports of users experiencing login issues when using Supabase Auth.

At this time, the impact appears to be limited to a subset of users, primarily in North and South America.
The exact cause and scope are still being determined.

Our team is actively working to identify the root of the issue and will provide updates as more information becomes available.

We appreciate your patience while we investigate.