Grafana

Jun 19, 02:33 UTC
Update - Due to the linked GCP outage below, users located in India may have trouble loading parts of Grafana.

https://status.cloud.google.com/incidents/5fGQt4VbkDnr3Yp8PXPr

We are continuing to work with our CSP on this investigation.

Impacted users may receive intermittent error messages such as "Error Loading" or "Failed to load Assets". To be clear, it does not matter the region the stack is located, but the geography where the user is physically in.

Jun 18, 16:57 UTC
Monitoring - Due to the linked GCP outage below, users located in India may have trouble loading parts of Grafana.

https://status.cloud.google.com/incidents/5fGQt4VbkDnr3Yp8PXPr

Impacted users …

Jun 18, 18:50 UTC
Resolved - This incident has been resolved. Thank you for your patience.

Jun 18, 18:09 UTC
Monitoring - We've verified a fix in our staging environment to restore functionality to the mobile app. The fix is currently being deployed to production. Thanks for your patience as we continue to roll this out and monitor the resolution.

Jun 18, 17:01 UTC
Identified - We're noticing an uptick in users being unable to respond to actions on the mobile app (acknowledging and silencing alerts, for example). Users working in the web UI should …

Jun 18, 16:21 UTC
Resolved - This incident has been resolved. Thank you for your patience.

Jun 18, 14:04 UTC
Update - We are continuing to monitor for any further issues.

Jun 18, 13:15 UTC
Monitoring - The root cause of the issue has been identified and a fix has been successfully deployed. We are observing widespread improvements across all systems. Our team is currently monitoring the environment to ensure performance remains stable.

Jun 18, 11:45 UTC
Update - We are continuing to investigate this issue.

Jun 18, 11:25 UTC
Investigating …

Jun 18, 14:08 UTC
Resolved - This incident has been resolved.

Jun 18, 08:04 UTC
Monitoring - A fix has been implemented and we are monitoring the results.

Jun 17, 23:22 UTC
Update - We are continuing to deploy the fix and monitor recovery efforts. As part of the rollout, we identified an issue that required adjustments to our deployment plan, which has extended the timeline for mitigation. Work remains actively underway, and we will share additional updates as progress continues.

Jun 17, 21:43 UTC
Update - Deployment of the fix is still …

Jun 18, 10:13 UTC
Resolved - A fix has been deployed and the issue after monitoring as been fixed.

Jun 18, 08:48 UTC
Investigating - We’re currently investigating an issue affecting Frontend Observability product. The "Suspected commit" feature is not currently working as expected. Ingestion and querying is unaffected by this. Our team has identified the cause and is actively working on a fix. Thank you for your patience.

Jun 17, 09:58 UTC
Resolved - Following our ongoing communications regarding the complete outage in prod-me-central-1, we are now closing this incident.
As noted in the latest AWS update, the Middle East (UAE) region (ME-CENTRAL-1) has suffered significant damage and restoration is expected to take several months. We strongly recommend all affected customers migrate workloads to an alternate Grafana Cloud region as soon as possible.

If you have not already done so, please follow the migration steps outlined in our previous updates:
1. Create a Grafana Cloud stack in an alternate region
2. Update clients to send telemetry to the new region, if …

Jun 12, 19:52 UTC
Resolved - This incident has been resolved. Thank you for your patience.

Jun 8, 21:36 UTC
Update - The incident has been mitigated, and services are operating normally. We continue to monitor the service to ensure full stability.

Jun 7, 11:00 UTC
Monitoring - The incident has been mitigated, and services are operating normally. We are currently monitor the service to ensure full stability.

Jun 7, 06:00 UTC
Update - We’re making ongoing progress on the investigation alongside our upstream provider.

Jun 7, 03:50 UTC
Update - …

Jun 12, 18:30 UTC
Resolved - Our team had discovered a read issue around 19:35-20:08 UTC. Impact at the time would have provided errors similar to context deadline exceeded (DatasourceError response). This has since been resolved, and should not have caused any data loss, only a short query disruption.

Jun 11, 05:36 UTC
Resolved - This incident has been resolved.

Jun 10, 10:49 UTC
Investigating - We’re currently investigating an issue affecting The Grafana Dashboards page. When set to view by folders, is currently experiencing an issue where no dashboards are shown. Our team is working on fixing the problem. In the meantime, switching to ‘View as list’ allows access to dashboards as usual”.

Jun 10, 13:11 UTC
Resolved - We continue to observe a continued period of recovery. At this time, we are considering this issue resolved. No further updates.

Jun 10, 10:57 UTC
Monitoring - Our team has implemented a fix and we are currently monitoring the results of this.

Jun 9, 23:11 UTC
Investigating - We are currently investigating an issue affecting data source-managed alerting management functionality in Grafana Cloud.

Customers may experience problems viewing, creating, updating, or managing alerts through Grafana when using data source-managed alerting. This issue is limited to alert management functionality within Grafana.

Alert …

Jun 8, 20:30 UTC
Resolved - This incident has been resolved.

Jun 8, 17:11 UTC
Monitoring - We've released a fix to the IRM app that should restore service for affected customers with issues related to labels. Thanks for your patience while investigating. We're continuing to monitor as we confirm the resolution in place.

Jun 8, 15:21 UTC
Update - We are continuing to work on a fix for this. To further clarify, this issue is not about accessing IRM or alert ingestion/notification/delivery, but rather with handling labels.

Jun 8, 13:14 UTC
Update …

Jun 5, 21:35 UTC
Resolved - This incident has been resolved.

Jun 5, 20:10 UTC
Update - Continuing to monitor progress. Most customers affected should have all services restored, with a few remaining customers receiving updates as the rollout finishes out. Thanks again for your patience.

Jun 5, 19:34 UTC
Update - A fix has been released to prod and rolling out across the fleet for IRM, restoring access to affected customers. Thanks for your patience through this work. We're continuing to monitor to confirm we've returned to a steady state.

Jun 5, …

Jun 5, 15:01 UTC
Resolved - This incident has been resolved.

Jun 5, 14:02 UTC
Identified - We have identified an issue causing Silences to not work as expected in the Cloud (Mimir) Alertmanager. Grafana Alertmanager is working ok, this is only affecting Data source-managed alerts.

Jun 4, 18:28 UTC
Resolved - This incident has been resolved.

Jun 4, 17:00 UTC
Identified - The issue has been identified, and we are working on a fix.

Jun 4, 16:56 UTC
Update - We are continuing to investigate this issue.

Jun 4, 16:44 UTC
Investigating - We are currently investigating an issue affecting the Skills page of Grafana Assistant. Impacted deployments will encounter a blank screen when attempting to access this page. At this time, we have observed partial impact in the us-east-0 and us-central-0 regions, and will provide an update …

Jun 4, 05:47 UTC
Resolved - This incident has been resolved.

Jun 3, 21:23 UTC
Monitoring - We have applied a fix, and are monitoring the results.

Jun 3, 20:40 UTC
Investigating - We are currently investigating an issue causing k6 test runs to take longer than expected to complete, or to time out within Grafana Cloud.

Jun 3, 18:22 UTC
Resolved - This incident has been resolved.

Jun 3, 17:07 UTC
Identified - We are in the process of deploying a fix for this issue.

Jun 3, 11:38 UTC
Investigating - We’re currently investigating an issue affecting Synthetic Monitoring where updates for Scripted/Browser checks might fail. Our team is actively working to identify the cause. Thank you for your patience.

Jun 3, 07:46 UTC
Resolved - This incident has been resolved.

Jun 2, 23:15 UTC
Identified - Between 21:20 and 22:40 UTC, writes to tempo-prod-25 failed due to an outage. tempo-prod-24 was also affected during an overlapping window from 22:32 to 22:40 UTC."

Jun 1, 20:26 UTC
Resolved - This incident has been resolved.

Jun 1, 20:02 UTC
Monitoring - A fix has been implemented and we are monitoring the results.

Jun 1, 19:53 UTC
Identified - Starting at 18:30 UTC, we noticed alert manager unavailability limited to prod-us-central-0 which affects grafana-managed and datasource-managed alerting, causing disruption to updating alertmanager config and limited disruption to alert sending.

We have identified the cause and are in the process of remediation.

May 29, 10:59 UTC
Resolved - This incident has been resolved.

May 29, 10:28 UTC
Monitoring - We have identified the cause of this incident and a fix has been applied. Normal functions are returning. We are currently monitoring the recovery process.

May 29, 09:03 UTC
Investigating - We’re currently investigating an issue affecting Loki queries in Grafana. We have had reports from customers showing the logs are not loading or showing missing logs. Our team is actively working to identify the cause. Thank you for your patience.

May 28, 14:00 UTC
Completed - The scheduled maintenance has been completed.

May 28, 12:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

May 22, 14:12 UTC
Scheduled - As part of an upcoming status page improvement initiative, we will be updating component names to better align with Grafana Cloud product naming.

This maintenance is focused on improving clarity and consistency across status communications and is not expected to impact service availability. During this time, customers may notice component names and groupings change on the status page.

May 27, 22:59 UTC
Resolved - This incident has been resolved. Thank you for your patience.

May 27, 21:57 UTC
Update - We are seeing recovery across affected Prometheus datasources, and error rates have significantly improved.

The service is recovering without any required customer action, and our team continues to monitor stability while we investigate the underlying cause.

We’ll provide another update as we learn more.

May 27, 21:36 UTC
Update - We continue to investigate an issue affecting Prometheus datasources causing intermittent timeouts and unexpected errors, primarily impacting alert rule evaluations.

Our team is actively working …