Gitlab Status

<small>April 8, 2026 07:00 UTC</small><br /><b>Investigating</b> - We're currently investigating intermittent issues with GitLab Duo Agentic Chat and Code Suggestions. We'll provide updates as more information becomes available.<br /><br /><small>April 8, 2026 07:16 UTC</small><br /><b>Investigating</b> - We're continuing to investigate degraded performance with GitLab Duo features, including Agentic Chat and Code Suggestions. This is linked to an ongoing upstream provider incident affecting Claude models. Switching to an alternative model may help as a workaround. We'll provide further updates as more information becomes available.<br /><br /><small>April 8, 2026 07:29 UTC</small><br /><b>Identified</b> - We've identified the cause of the degraded GitLab Duo performance as an upstream provider outage affecting Claude models. Error rates are beginning to decrease as the upstream issue is being resolved. We'll provide further updates as the situation progresses.<br /><br /><small>April 8, 2026 08:03 UTC</small><br /><b>Monitoring</b> - GitLab Duo services, including Agentic Chat and Code Suggestions, have returned to normal operation. We're continuing to monitor to ensure stability.<br /><br />

<small>April 1, 2026 16:01 UTC</small><br /><b>Investigating</b> - We are investigating reports of Git clone failures and CI pipeline timeouts. Affected users may see "GitLab is currently unable to handle this request due to load" or "CI configuration fetch from Gitaly timed out." We are actively investigating and will provide updates as we learn more.<br /><br /><small>April 1, 2026 17:04 UTC</small><br /><b>Identified</b> - We have begun rolling out a feature flag to address the intermittent Git clone failures and timeouts within pipelines. More details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21689.<br /><br /><small>April 1, 2026 19:14 UTC</small><br /><b>Identified</b> - We're rolling back the feature flag and exploring alternative options for a long-term fix. We've observed that the scope of impact is limited to a small set of customers. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21689.<br /><br /><small>April 1, 2026 22:16 UTC</small><br /><b>Monitoring</b> - After rollback of the feature flag we've observed a low error rate for this issue, but are continuing to monitor to ensure it remains that way.<br /><br /><small>April 2, 2026 03:50 UTC</small><br /><b>Resolved</b> - After a period of monitoring we have not seen a recurrence of the issue and consider it resolved.<br /><br />

<small>April 1, 2026 07:18 UTC</small><br /><b>Investigating</b> - Multiple users are unable to log in when using two-factor authentication via email. Several customers and users raised the issue. We are actively investigating logs to identify patterns in sign-in endpoint requests and confirming the issue is limited to 2FA via email.<br /><br /><small>April 1, 2026 08:55 UTC</small><br /><b>Investigating</b> - The incident remains under investigation. The scope is confirmed as affecting only free-tier users attempting to log in with email-based 2FA. Details can be found in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21683<br /><br /><small>April 1, 2026 10:09 UTC</small><br /><b>Investigating</b> - No material updates on resolution. We continue investigating logs to identify the root cause. Details can be found in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21683<br /><br /><small>April 1, 2026 11:31 UTC</small><br /><b>Investigating</b> - Investigation is ongoing. We are narrowing down potential causes and will provide an update once we have more to share. Details can be found in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21683<br /><br /><small>April 1, 2026 12:45 UTC</small><br /><b>Monitoring</b> - The incident has been mitigated. Users should now be able to log in with email-based 2FA. We continue investigating the root cause and will provide further updates. Details can be found in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21683<br /><br /><small>April 1, 2026 19:38 UTC</small><br /><b>Resolved</b> - After a period of monitoring we have not seen a recurrence of the issue and consider it resolved.<br /><br />

<small>March 31, 2026 09:37 UTC</small><br /><b>Investigating</b> - We are currently investigating an issue with Duo Agent Platform and LLM proxy endpoints hitting rate limits. Users may experience errors using Claude models. We recommend switching to a different model as a workaround.<br /><br /><small>March 31, 2026 10:14 UTC</small><br /><b>Investigating</b> - We continue investigating the issue, no material updates to report. Additional information can be found in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21672<br /><br /><small>March 31, 2026 11:28 UTC</small><br /><b>Investigating</b> - As we continue investigating, we're working on immediate mitigation steps. Additional information can be found in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21672<br /><br /><small>March 31, 2026 12:33 UTC</small><br /><b>Identified</b> - We've applied traffic filtering rules to block the primary source of excess requests to our endpoints. We're seeing a significant drop in rate limit errors and are continuing work on permanent fixes. More details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21672<br /><br /><small>March 31, 2026 15:06 UTC</small><br /><b>Monitoring</b> - We've confirmed the traffic filtering rules are applied, with errors dropping to near zero. Claude models should be available now. A permanent fix has been merged and deployment is in progress. We continue monitoring and working on follow-up improvements. More details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21672<br /><br /><small>March 31, 2026 21:57 UTC</small><br /><b>Resolved</b> - After a period of monitoring we have not observed a recurrence of the issue and are considering it resolved.<br /><br />

<small>March 24, 2026 17:39 UTC</small><br /><b>Identified</b> - The gitlab-org group are seeing 500 errors due to a specific Gitaly node. We are working to bring things back online but anyone accessing that repository, such as through pipelines or other tasks, will experience errors accessing it.<br /><br /><small>March 24, 2026 17:51 UTC</small><br /><b>Monitoring</b> - We are seeing things recover now and users should now be able to access the gitlab-org namespace. We are doing integrity checks to understand if there is any lingering impact on the impacted system.<br /><br /><small>March 24, 2026 18:25 UTC</small><br /><b>Resolved</b> - Operations have returned to normal and the gitlab-org repository is accessible as normal.<br /><br />

<small>March 17, 2026 15:51 UTC</small><br /><b>Investigating</b> - We are investigating delay in background job processing. We are working on further improvements, more details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br /><small>March 17, 2026 18:12 UTC</small><br /><b>Identified</b> - Improvements to help address this issue are currently under review. We are still observing a delay in background job processing. Users may still see delays in CI job status updates. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br /><small>March 17, 2026 20:17 UTC</small><br /><b>Monitoring</b> - Background job processing has recovered as peak load has passed. Additional changes are under review to address this going forward. This incident will remain in monitoring as we deploy these changes and observe metrics through tomorrow's peak. Details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br /><small>March 18, 2026 15:13 UTC</small><br /><b>Investigating</b> - We are investigating delay in background job processing. After a monitoring period that followed yesterday fixes, and expanding infra resources, we are investigating further improvements, more details in gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br /><small>March 18, 2026 17:56 UTC</small><br /><b>Monitoring</b> - Background job processing has recovered. Additional changes have been implemented which helped to reduce impact during peak load. We will continue to monitor this as we move through peak load times tomorrow. Details: http://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br /><small>March 19, 2026 09:41 UTC</small><br /><b>Monitoring</b> - The team has continued to make improvements since yesterday's update to further reduce impact during peak load. We are actively monitoring the situation. Details: gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br /><small>March 19, 2026 15:36 UTC</small><br /><b>Investigating</b> - We are currently investigating performance degradation and our team is currently actively working on a mitigation. More details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br /><small>March 19, 2026 17:11 UTC</small><br /><b>Monitoring</b> - We observed an increased delay in background job processing and CI job queue times, both of which have now recovered. We're closely monitoring the situation as we work on additional improvements. Details: gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br /><small>March 20, 2026 01:00 UTC</small><br /><b>Monitoring</b> - The team has continued to make improvements to the underlying infrastructure to prevent further impact. We have not seen any recent occurrences of background job delays. We will keep this incident open through peak load times over the next 20 hours to monitor performance metrics.<br />Details: gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br /><small>March 20, 2026 15:38 UTC</small><br /><b>Resolved</b> - The combination of infrastructure and application improvements made has sustained stability during peak hours yesterday. This incident has been moved to resolved following our monitoring.<br /><br />Details: gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br />

<small>March 17, 2026 08:18 UTC</small><br /><b>Identified</b> - We have identified a code change that provokes 500 errors for API requests. We are currently reverting the change. Workaround in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21573<br /><br /><small>March 17, 2026 09:27 UTC</small><br /><b>Identified</b> - The revert change has been merged, users will still experience 500 errors until the fix reaches production. We estimate it will be fully deployed at the end of EMEA day. More details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21573<br /><br /><small>March 17, 2026 12:31 UTC</small><br /><b>Identified</b> - The fix has now reached Canary environment, users can enable it by clicking "Next" at https://next.gitlab.com/ until the fix reaches production. More details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21573<br /><br /><small>March 17, 2026 15:22 UTC</small><br /><b>Monitoring</b> - The fix has reached production and users should not experience 500 errors on API calls. We will be monitoring the fix before marking this incident as Resolved. More details https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21573<br /><br /><small>March 17, 2026 20:40 UTC</small><br /><b>Resolved</b> - Our metrics indicate this is now fixed. We will now be marking this as resolved. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21573<br /><br />

<small>March 16, 2026 16:23 UTC</small><br /><b>Investigating</b> - We are investigating a delay in background job processing. Users may observe a delay in CI job status updates in the UI. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br /><small>March 16, 2026 16:48 UTC</small><br /><b>Investigating</b> - Our investigation is ongoing. Users may still observe delays in CI job status updates. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br /><small>March 16, 2026 17:21 UTC</small><br /><b>Identified</b> - We've identified a potential fix and are working to deploy it. We'll update you once we have a production deployment timeline. Details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br /><small>March 16, 2026 17:54 UTC</small><br /><b>Identified</b> - The primary fix has been merged and deployment is underway. Full rollout typically takes several hours, so we're pursuing additional mitigations and implementation changes in parallel. Details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br /><small>March 16, 2026 20:30 UTC</small><br /><b>Monitoring</b> - We are observing improved system metrics and reduced user impact. Mitigations are pending deployment with the goal of preventing future recurrence. This incident will remain in monitoring status as we observe system behavior. Details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br /><small>March 17, 2026 10:58 UTC</small><br /><b>Monitoring</b> - Monitoring of CI pipelines shows metrics have returned to normal. We will keep monitoring the fix for the next hour. More details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br /><small>March 17, 2026 11:54 UTC</small><br /><b>Resolved</b> - After monitoring the CI pipelines, we are marking the incident as resolved. More details https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21562<br /><br />

<small>March 12, 2026 17:05 UTC</small><br /><b>Investigating</b> - Our team is investigating potential delays with background job processing. This could impact CI/CD job processing.<br /><br /><small>March 12, 2026 17:12 UTC</small><br /><b>Investigating</b> - We're seeing increased job queuing rates that are primarily affecting small CI runners. Our team is still investigating.<br /><br /><small>March 12, 2026 17:27 UTC</small><br /><b>Investigating</b> - No material updates at this time, our team is still investigating the issue.<br /><br /><small>March 12, 2026 17:35 UTC</small><br /><b>Investigating</b> - Our team is still investigating the issue.<br /><br /><small>March 12, 2026 17:59 UTC</small><br /><b>Investigating</b> - Our team is seeing some recovery but are still investigating the root cause. We believe that a previous Sidekiq backlog caused CI jobs to accumulate, leading to delays. More detail in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21532<br /><br /><small>March 12, 2026 18:06 UTC</small><br /><b>Monitoring</b> - The backlog has recovered and we are no longer seeing delays. Our investigation points to a Sidekiq backlog which lead to delays in CI job execution. Our team is currently monitoring. Additional details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21532<br /><br /><small>March 12, 2026 18:33 UTC</small><br /><b>Monitoring</b> - For clarification, it was noted earlier in the incident that we saw GitLab-hosted Runners were primarily impacted. However, the background job processing delay has impact on self-hosted runners as well. Our team has seen a recovery and we are still monitoring for stability.<br /><br /><small>March 12, 2026 20:07 UTC</small><br /><b>Resolved</b> - Our team has been monitoring and job processing has returned to normal. This incident is now resolved. Further details available in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21532<br /><br />

<small>March 12, 2026 15:34 UTC</small><br /><b>Investigating</b> - We are currently investigating an issue affecting Duo Workflow and AI Gateway for requests routed through GPT models. Users may experience errors using only gpt models on DAP. Engineering is currently investigating the issue.<br /><br /><small>March 12, 2026 15:40 UTC</small><br /><b>Investigating</b> - The cause for the intermittent errors is still under investigation.<br /><br /><small>March 12, 2026 16:26 UTC</small><br /><b>Identified</b> - Our team has identified the issue as related to a recent commit. Our team is discussing next steps.<br /><br />Users may also experience NotImplementedError with Code Review flows.<br /><br />More information will be available in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21529<br /><br /><small>March 12, 2026 16:52 UTC</small><br /><b>Identified</b> - Our team is currently reverting some recent changes to resolve the issue. <br /><br /><small>March 12, 2026 17:48 UTC</small><br /><b>Identified</b> - This incident is pending a deployment to resolve the issue. We'll provide an update once it is deployed.<br /><br /><small>March 12, 2026 19:22 UTC</small><br /><b>Resolved</b> - The fix has been deployed and we have not seen any more errors. This incident is now resolved.<br /><br />

<small>March 12, 2026 15:19 UTC</small><br /><b>Investigating</b> - We are seeing the CI job backlog building up, which causes CI job operations to be delayed. We are investigating the cause for this issue.<br /><br /><small>March 12, 2026 15:47 UTC</small><br /><b>Resolved</b> - The situation has now stabilized. We are closing this incident.<br /><br />

<small>March 11, 2026 15:56 UTC</small><br /><b>Investigating</b> - We are experiencing 5XX errors. The cause for the errors is under investigation.<br /><br /><small>March 11, 2026 16:11 UTC</small><br /><b>Identified</b> - Our team has taken steps to mitigate the issue and services are back to normal. Our team is monitoring and discussing next steps.<br /><br /><small>March 11, 2026 16:35 UTC</small><br /><b>Monitoring</b> - GitLab is currently operational and the incident has been mitigated. We're currently monitoring and will provide an update at a later time.<br /><br /><small>March 11, 2026 20:05 UTC</small><br /><b>Resolved</b> - Our team has mitigated this issue and has been monitoring for stability. This incident is now resolved.<br /><br />

<small>March 10, 2026 16:01 UTC</small><br /><b>Investigating</b> - Our team is investigating potential delays with background job processing. This could impact CI/CD job processing.<br /><br /><small>March 10, 2026 16:16 UTC</small><br /><b>Investigating</b> - No material updates at this time. Our team is still investigating this issue.<br /><br /><small>March 10, 2026 16:43 UTC</small><br /><b>Identified</b> - Our team has identified the issue as PgBouncer saturation and are looking at mitigation efforts. Users may see small delays in background processing. More information in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21498<br /><br /><small>March 10, 2026 17:28 UTC</small><br /><b>Monitoring</b> - We're seeing stabilization and recovery but are still monitoring and discussing next steps. More info in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21498<br /><br /><small>March 10, 2026 18:31 UTC</small><br /><b>Monitoring</b> - Background processing has recovered and CI wait times are back to normal. Our team is continuing to monitor for stability.<br /><br /><small>March 11, 2026 19:47 UTC</small><br /><b>Resolved</b> - Our team has been monitoring for stability and have taken additional steps to help reduce database saturation. This incident is now resolved. More details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21498<br /><br />

<small>March 11, 2026 03:29 UTC</small><br /><b>Identified</b> - Starting from approximately 08:00 UTC on 2026-03-10, multiple users were unable to download NuGet packages due to 403 Forbidden errors. We have identified the cause, and are going through the process of reverting the MR.<br /><br /><small>March 11, 2026 16:51 UTC</small><br /><b>Monitoring</b> - The MR in question has been reverted and error rates have decreased. We are currently monitoring this endpoint.<br /><br /><small>March 11, 2026 19:32 UTC</small><br /><b>Resolved</b> - The revert has deployed and we are no longer observing 403 Forbidden responses when downloading packages. We are resolving this incident.<br /><br />

<small>March 9, 2026 15:59 UTC</small><br /><b>Investigating</b> - We are seeing delays in CI/CD pipeline processing. The cause for this problem is under investigation. Please follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21484 for more details.<br /><br /><small>March 9, 2026 16:19 UTC</small><br /><b>Investigating</b> - Our team is still investigating this issue. Users might encounter some delays with processing CI/CD pipelines.<br /><br /><small>March 9, 2026 16:33 UTC</small><br /><b>Investigating</b> - Our team is considering increasing resources to address the issue with the Sidekiq backlog impacting CI/CD pipelines. We're working on a deployment fix to address this, though we are seeing some improvement overall with queue durations.<br /><br /><small>March 9, 2026 17:07 UTC</small><br /><b>Identified</b> - Our team has turned off a specific worker to help improve pipeline processing. There has been significant recovery, and our team has identified this as a resource issue. We are monitoring for stability in addition to investigating the root cause and determining a permanent fix.<br /><br /><small>March 9, 2026 17:45 UTC</small><br /><b>Identified</b> - We are continuing to monitor but the main backlog drivers are mitigated. Users should expect normal operations. Our team is currently deploying a configuration update to further address this. We'll provide an update once it is deployed.<br /><br /><small>March 9, 2026 18:15 UTC</small><br /><b>Resolved</b> - The configuration change has been deployed and we have seen stability with Sidekiq and pipeline processing. This incident is now resolved.<br /><br />

<small>March 6, 2026 17:48 UTC</small><br /><b>Identified</b> - Pipelines affected by a previous incident on 3/5 continue to be stuck in Running status. We are working on a process to identify and safely correct these stuck pipelines, and will update this notice once that remediation is in place. Follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for details.<br /><br /><small>March 6, 2026 20:30 UTC</small><br /><b>Identified</b> - After consideration, we have decided not to automatically change previously stuck pipelines. Doing so could cause unsafe or stale state changes, for example around deployments and environments.<br /><br /><small>March 6, 2026 20:31 UTC</small><br /><b>Resolved</b> - If a pipeline still appears stuck, please cancel and retry the affected job or pipeline. Protections deployed during this incident will prevent this in new pipelines. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for complete details.<br /><br />

<small>March 6, 2026 09:19 UTC</small><br /><b>Investigating</b> - We're noticing errors while pulling GitLab's main repository. This issue is currently under investigation.<br /><br /><small>March 6, 2026 09:53 UTC</small><br /><b>Monitoring</b> - We have now identified the root cause and took actions to mitigate the issue. Git pull operations for GitLab's main repo are now working as expected. We'll continue to monitor the situation.<br /><br /><small>March 6, 2026 11:03 UTC</small><br /><b>Resolved</b> - This incident is now resolved.<br /><br />

<small>March 6, 2026 03:58 UTC</small><br /><b>Investigating</b> - Users are intermittently seeing HTTP 50X errors when attempting to access to GitLab.com. We will post again when we have material updates.<br /><br /><small>March 6, 2026 04:06 UTC</small><br /><b>Monitoring</b> - We have identified the cause is due to excessive traffic against specific endpoints and have implemented mitigation measures. We're continuing to monitor the situation, and are seeing the error rates decrease.<br /><br /><small>March 6, 2026 04:14 UTC</small><br /><b>Monitoring</b> - We identified the cause of errors as excessive access by a specific user and implemented mitigation measures. We will continue to monitor the overall status of our services.<br /><br /><small>March 6, 2026 06:47 UTC</small><br /><b>Resolved</b> - This incident has now been mitigated and services are back to normal.<br /><br />

<small>March 5, 2026 21:25 UTC</small><br /><b>Investigating</b> - Some users are experiencing CI/CD pipelines where jobs and pipelines are stuck in a “running” state, even though earlier-stage jobs have completed successfully. Follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for details.<br /><br /><small>March 5, 2026 21:40 UTC</small><br /><b>Investigating</b> - No material change since the last update. The incident remains under active investigation. Follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for details.<br /><br /><small>March 5, 2026 21:55 UTC</small><br /><b>Investigating</b> - We’re still actively investigating and are currently collating data from affected pipelines to better understand the patterns and root cause. Please follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for details.<br /><br /><small>March 5, 2026 22:12 UTC</small><br /><b>Investigating</b> - We are currently reviewing logs from our production infrastructure and specifically the status and behavior of the pipeline workers for potential causes. Follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for details.<br /><br /><small>March 5, 2026 22:25 UTC</small><br /><b>Investigating</b> - We’ve identified a potential issue with the job deferral mechanism that may be affecting pipeline transitions between stages. For details, please see https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462.<br /><br /><small>March 5, 2026 23:19 UTC</small><br /><b>Identified</b> - We have identified the cause of this incident as the recurrence of an existing defect in the pipeline job deferral mechanism. In this case, it was triggered by a period of temporary high pipeline concurrency. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for details.<br /><br /><small>March 5, 2026 23:27 UTC</small><br /><b>Monitoring</b> - At this time, we are confident that current pipelines are not being affected by this behavior. We are continuing to monitor new reports closely until the top of the hour to ensure there is no ongoing impact. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for details.<br /><br /><small>March 6, 2026 00:04 UTC</small><br /><b>Resolved</b> - As there is no current impact, we are setting this incide to Resolved. We have identified the problematic behavior behind it and are working on a long-term fix. However, we know this defect can reoccur under certain high‑concurrency conditions. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21462 for details.<br /><br />

<small>March 4, 2026 20:36 UTC</small><br /><b>Investigating</b> - Agentic chat on GitLab.com intermittently fails to return responses to user messages; the root cause is under investigation. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for details.<br /><br /><small>March 4, 2026 20:49 UTC</small><br /><b>Investigating</b> - No material updates to share yet, please follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for any updates.<br /><br /><small>March 4, 2026 21:09 UTC</small><br /><b>Investigating</b> - We are continuing to investigate this incident. The team is currently reviewing the stability of the cloud connector as a potential contributing factor, but this has not yet been confirmed. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for details.<br /><br /><small>March 4, 2026 21:25 UTC</small><br /><b>Investigating</b> - We are observing transient authentication issues in our cloud connector service and are investigating this as a potential cause of the incident. Please follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for updates.<br /><br /><small>March 4, 2026 21:42 UTC</small><br /><b>Investigating</b> - We have identified a recently deployed merge request as a potential cause of this issue and have begun rolling out the previous version to observe if behavior improves. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for details.<br /><br /><small>March 4, 2026 21:56 UTC</small><br /><b>Investigating</b> - GitLab Duo Agentic Chat is showing promising improvement after rolling back to a previous version, with responses returning more consistently. We are continuing to review the identified merge request and its changes to confirm whether it is the root cause of the issue. Follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for details.<br /><br /><small>March 4, 2026 22:09 UTC</small><br /><b>Identified</b> - After further testing, the deployment without the suspected merge request has started exhibiting the errors again. We have identified an authentication failure as the cause of this issue and continue to troubleshoot it. Follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for details.<br /><br /><small>March 4, 2026 22:26 UTC</small><br /><b>Identified</b> - We have merged a patch that we believe will help restore GitLab Duo to a usable state, we are waiting for this change to be deployed to observe its impact. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for details.<br /><br /><small>March 4, 2026 23:03 UTC</small><br /><b>Monitoring</b> - After deploying the patch, we are no longer seeing related errors and the service appears stable. We will continue to monitor for a period before considering this incident resolved. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21448 for details.<br /><br /><small>March 4, 2026 23:53 UTC</small><br /><b>Resolved</b> - We are wrapping up our monitoring period and this incident is now resolved. <br /><br />

<small>March 4, 2026 06:18 UTC</small><br /><b>Investigating</b> - Users are intermittently seeing HTTP 50X errors when attempting to access to GitLab.com. We will post again when we have material updates.<br /><br /><small>March 4, 2026 06:54 UTC</small><br /><b>Monitoring</b> - We identified the cause of errors as excessive access by a specific user and implemented mitigation measures. We continue to monitor the overall status of our services.<br /><br /><small>March 4, 2026 09:01 UTC</small><br /><b>Resolved</b> - This incident has now been mitigated and services are back to normal.<br /><br />

<small>March 3, 2026 16:46 UTC</small><br /><b>Investigating</b> - Users are intermittently seeing 400 Bad Request errors when attempting to log into GitLab.com. This does not seem to be affecting all users at the moment and investigation is ongoing. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21436 for details.<br /><br /><small>March 3, 2026 16:56 UTC</small><br /><b>Investigating</b> - No material updates to report at this time. We continue tracing the errors to determine the root cause. Follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21436 for details.<br /><br /><small>March 3, 2026 17:08 UTC</small><br /><b>Investigating</b> - We’ve identified a recent deployment that may be contributing to this issue and are preparing a rollback to see if it improves the situation. Keep tuned for more updates on https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21436.<br /><br /><small>March 3, 2026 17:20 UTC</small><br /><b>Investigating</b> - Rollback is still in progress. We’ll update this status page as soon as we have new information to share. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21436 for details.<br /><br /><br /><br /><small>March 3, 2026 17:52 UTC</small><br /><b>Investigating</b> - The rollback has completed, but we are not yet seeing a reduction in the frequency of the error messages. Investigation continues, see https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21436 for details.<br /><br /><small>March 3, 2026 18:05 UTC</small><br /><b>Investigating</b> - We are now investigating whether an upstream component could be contributing to the behavior, though this has not yet been confirmed. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21436 for updates.<br /><br /><small>March 3, 2026 18:39 UTC</small><br /><b>Investigating</b> - We started reviewing a recent change to our HTTP router as a contributing factor, but have now ruled it out and continue investigating other potential causes. We’ll provide another update once we have new information to share.<br /><br /><small>March 3, 2026 19:04 UTC</small><br /><b>Investigating</b> - We are involving Cloudflare to investigate a potential upstream problem and have reached out for their assistance. Please follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21436 for details.<br /><br /><small>March 3, 2026 20:22 UTC</small><br /><b>Investigating</b> - We continue working with Cloudflare support to figure out the cause of these errors. We will update this incident as any findings come along. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21436 for details.<br /><br /><small>March 3, 2026 21:50 UTC</small><br /><b>Investigating</b> - Cloudflare has identified a potential lead on their side and is investigating further. We will update this status page once we have more information to share from their side. Follow https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21436 for details.<br /><br /><small>March 3, 2026 23:01 UTC</small><br /><b>Identified</b> - Cloudflare has declared a public incident to investigate this issue on their end. Please follow https://www.cloudflarestatus.com/incidents/t2skq2t69g5s and https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21436 for details.<br /><br /><small>March 4, 2026 00:13 UTC</small><br /><b>Monitoring</b> - The upstream Cloudlfare incident has been resolved and now we monitor to confirm the login error is no longer present in our logs before proceeding to resolve this incident. See https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21436 for details.<br /><br /><small>March 4, 2026 01:40 UTC</small><br /><b>Resolved</b> - Per further log inspection on our side, we're no longer seeing the problem occurring. The incident is now resolved. Refer to https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21436 for details.<br /><br />

<small>March 3, 2026 09:01 UTC</small><br /><b>Identified</b> - We've identified failures for remote flows, including Duo Developer, and other foundational workflows. Users are seeing failures such as "Session failed" when requesting code reviews through GitLab Duo. The root cause has been identified and a revert is ongoing. More details here: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21429<br /><br /><small>March 3, 2026 09:31 UTC</small><br /><b>Identified</b> - No material updates at this time, the revert is still in progress. More details here: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21429<br /><br /><small>March 3, 2026 10:01 UTC</small><br /><b>Identified</b> - Still no material updates at this time, the revert is in progress. More details here: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21429<br /><br /><small>March 3, 2026 10:46 UTC</small><br /><b>Identified</b> - Still no material updates at this time, the revert is in progress. More details here: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21429<br /><br /><small>March 3, 2026 11:03 UTC</small><br /><b>Monitoring</b> - The revert has been merged, and the flows are now operational. We are now monitoring the situation. More details here: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21429<br /><br /><small>March 3, 2026 11:46 UTC</small><br /><b>Resolved</b> - This incident is now resolved. More details here: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21429<br /><br />

<small>February 25, 2026 19:36 UTC</small><br /><b>Monitoring</b> - We observed increased 403 errors on Git HTTPS requests affecting some CI/CD pipelines on GitLab.com. The underlying issue has been corrected and we are monitoring to ensure stability. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21389<br /><br /><small>February 25, 2026 19:54 UTC</small><br /><b>Monitoring</b> - We observed increased 403 and 429 errors on Git HTTPS requests affecting some CI/CD pipelines on GitLab.com. We have adjusted our protection rules to alleviate these errors and are monitoring to ensure stability. Details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21389<br /><br /><small>February 25, 2026 21:14 UTC</small><br /><b>Resolved</b> - We have not seen any further reports of 403 or 429 errors on Git HTTPS requests, and customers should no longer be experiencing related CI/CD pipeline issues. As the service has remained stable, we will now resolve this incident. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21389<br /><br />

<small>February 24, 2026 17:11 UTC</small><br /><b>Investigating</b> - We’re investigating increased background processing delays on GitLab.com that are affecting pipeline execution and merge trains. Our engineering team is actively working to identify the cause and restore normal performance. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21361<br /><br /><small>February 24, 2026 17:28 UTC</small><br /><b>Investigating</b> - We do not have a material update to share at this time. Our engineering team continues to actively investigate and will provide further updates as soon as more information becomes available. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21361<br /><br /><small>February 24, 2026 17:47 UTC</small><br /><b>Investigating</b> - We continue to see delays in background job processing on GitLab.com. Our engineers continue to investigate the root cause and work to restore normal performance. More details: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/21361<br /><br /><small>February 24, 2026 18:19 UTC</small><br /><b>Identified</b> - Background processing on GitLab.com has largely recovered, and user experience has noticeably improved over the past 15 minutes. We’re continuing to investigate the root cause and will share additional updates if we see any recurrence. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21361<br /><br /><small>February 24, 2026 18:54 UTC</small><br /><b>Identified</b> - Background processing on GitLab.com continues to improve, but some users may still experience delays in pipeline execution and merge trains while the backlog is processed. Our engineers continue to investigate the root cause. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21361<br /><br /><small>February 24, 2026 19:36 UTC</small><br /><b>Monitoring</b> - Background processing on GitLab.com has stabilized. We will continue monitoring for the next 24 hours. If the issue does not recur, this incident will be resolved. Otherwise, we’ll provide updates and take additional action as needed. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21361<br /><br /><small>February 25, 2026 18:58 UTC</small><br /><b>Resolved</b> - After continued monitoring, performance has remained stable and within normal levels. As systems are operating as expected, this incident is now marked as resolved. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21361<br /><br />

<small>February 18, 2026 20:01 UTC</small><br /><b>Investigating</b> - GitLab is investigating ongoing issues with several Gitaly nodes that are causing intermittent 500 errors. We are working to identify the cause of instability on these nodes. Our investigation is ongoing; we will post again when we have material updates or in two hours with a status update.<br /><br /><small>February 18, 2026 22:37 UTC</small><br /><b>Investigating</b> - The Gitaly team continues to investigate the performance issues on these Gitaly nodes and are still working to identify a root cause. We will update again in about two hours.<br /><br /><small>February 19, 2026 00:38 UTC</small><br /><b>Investigating</b> - The Gitaly team continues to investigate and mitigate the performance issues on affected Gitaly nodes and are still working to identify a root cause. We will post an update within the next four hours.<br /><br /><small>February 19, 2026 04:38 UTC</small><br /><b>Investigating</b> - The Gitaly team continues to investigate and mitigate the performance issues on affected Gitaly nodes. We are still working to identify a root cause and will post an update within the next four hours if not sooner.<br /><br /><small>February 19, 2026 08:30 UTC</small><br /><b>Investigating</b> - The Gitaly team continues to investigate and mitigate the performance issues on affected Gitaly nodes. We are still working to identify a root cause and mitigation steps.<br /><br /><small>February 19, 2026 20:26 UTC</small><br /><b>Investigating</b> - The Gitaly team continues to investigate and mitigate the performance issues on affected Gitaly nodes. We are still working to identify a root cause and mitigation steps.<br /><br /><small>February 20, 2026 00:39 UTC</small><br /><b>Investigating</b> - The Gitaly team continues to investigate and mitigate the performance issues on affected Gitaly nodes. We are still working to identify a root cause and mitigation steps. A status update will be made within the next 8 hours.<br /><br /><small>February 20, 2026 08:25 UTC</small><br /><b>Investigating</b> - The Gitaly team continues to investigate and mitigate the performance issues on affected Gitaly nodes. We have not seen issues within the past 24 hours and continue to monitor closely. We are still working to identify a root cause. A status update will be made within the next 8 hours.<br /><br /><small>February 20, 2026 18:02 UTC</small><br /><b>Investigating</b> - The Gitaly team continues to investigate and mitigate the performance issues on affected Gitaly nodes. We have not seen issues within the past 24 hours and continue to monitor closely. We are still working to identify a root cause. A status update will be made within the next 8 hours.<br /><br /><small>February 21, 2026 02:11 UTC</small><br /><b>Monitoring</b> - The Gitaly team has applied mitigations to address performance issues and will monitor over the weekend. We have confirmed that current performance issues only affect internal GitLab operations. Customer activity should not be impacted at this time.<br />With the current impact and mitigation in place, we will resume status updates for this incident on Monday, February 23rd at 17:00 UTC<br /><br /><small>February 23, 2026 17:25 UTC</small><br /><b>Resolved</b> - After continued monitoring over the weekend, performance has remained stable and within normal levels. As systems are operating as expected, this incident is now marked as resolved.<br /><br />

<small>February 21, 2026 02:11 UTC</small><br /><b>Monitoring</b> - We are experiencing degraded performance with the Duo Agent Platform in Europe. We have identified the root cause and are monitoring the situation.<br />More information on the incident can be found here: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21352<br /><br /><small>February 21, 2026 03:22 UTC</small><br /><b>Resolved</b> - Performance for the Duo Agent Platform in Europe has been restored to normal.<br />For more information, please see: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21352<br /><br />

<small>February 20, 2026 04:26 UTC</small><br /><b>Identified</b> - We are experiencing some Anthropic service disruption in the asia-northeast1 region. We are currently investigating to identify the cause.<br /><br /><small>February 20, 2026 05:34 UTC</small><br /><b>Monitoring</b> - The Anthropic service in asia-northeast1 experienced partial disruption between 00:15 and 04:50 UTC, impacting the Duo Agentic Platform (DAP). The service has been stable now for the past 20 minutes and we are monitoring the situation for the next hour before resolving the incident.<br /><br /><small>February 20, 2026 06:38 UTC</small><br /><b>Resolved</b> - The service has been stable for the past 90 minutes. The issue has been resolved.<br /><br />https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21340<br /><br />

<small>February 17, 2026 11:54 UTC</small><br /><b>Identified</b> - We see a 1.5% degraded state on two of our Gitaly nodes. Some customer may experience delays while contributing to their projects. We have identified the problem and we are working through steps to resolve this.<br /><br />https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21303<br /><br /><small>February 17, 2026 11:57 UTC</small><br /><b>Identified</b> - Partial service disruption due to this issue. <br /><br /><small>February 17, 2026 12:27 UTC</small><br /><b>Monitoring</b> - We have recovered and are still monitoring.<br /><br />https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21303<br /><br /><small>February 17, 2026 19:55 UTC</small><br /><b>Investigating</b> - We have seen an unexpected increase in 500 errors on GitLab.com pages, often related to Merge Requests. We are investigating the root cause of these issues.<br /><br />Further details in the incident issue: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21303<br /><br /><small>February 17, 2026 20:29 UTC</small><br /><b>Investigating</b> - We are continuing to investigate the 500 errors on GitLab.com. We are investigating the root cause of the incident. We are preparing to apply mitigations to reduce error rates. <br /><br />Further details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21310<br /><br /><small>February 17, 2026 20:55 UTC</small><br /><b>Investigating</b> - We have rebooted the VMs that were generating 500 errors. Once the VMs have rebooted we will validate that Gitaly is responding normally and work to identify the root cause of these issues to prevent re-occurrence. <br /><br />Further details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21310<br /><br /><small>February 17, 2026 21:15 UTC</small><br /><b>Monitoring</b> - The nodes have been rebooted and Gitaly appears to be working as expected. We are moving this incident to Monitoring. Our team is working to identify the root cause of these errors while we monitor the node performance. <br /><br />Further details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21310<br /><br /><small>February 17, 2026 22:35 UTC</small><br /><b>Monitoring</b> - Gitaly appears to be working normally at this time. We are continuing to monitor the issue and have mitigation steps should the issue reoccur. We do not anticipate further updates unless we start seeing degraded Gitaly performance again.<br /><br />Further details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21310<br /><br /><small>February 18, 2026 04:28 UTC</small><br /><b>Investigating</b> - We're observing elevated 500 error rate on GitLab.com again and we're investigating the root cause.<br /><br />See detail issue: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21303<br /><br /><small>February 18, 2026 04:52 UTC</small><br /><b>Identified</b> - We've identified the problem related to git activities. We're looking into solution.<br /><br />See issue: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21303<br /><br /><small>February 18, 2026 06:07 UTC</small><br /><b>Identified</b> - We're still exploring the solution for the git problem. <br /><br />See details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21303<br /><br /><small>February 18, 2026 06:50 UTC</small><br /><b>Monitoring</b> - Restarting specific nodes improved the situation. We'll keep monitoring the situation for now.<br /><br />More information: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21303<br /><br /><small>February 18, 2026 07:00 UTC</small><br /><b>Resolved</b> - Restarting specific nodes improved the situation. We'll mark the incident as resolve while we keep monitoring the situation.<br /><br />More information: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21303<br /><br />

<small>February 11, 2026 20:23 UTC</small><br /><b>Identified</b> - Developer and Fix CI/CD Pipeline Duo Flows are currently failing due to a bug. The cause has been identified, and the fix is set to merge. For more information, please see: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21266<br /><br /><small>February 11, 2026 20:53 UTC</small><br /><b>Identified</b> - The fix has been merged. It should deploy to production in about an hour. For more information, please see: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21266<br /><br /><small>February 11, 2026 21:27 UTC</small><br /><b>Identified</b> - No updates at this time. The fix has been merged, and we're awaiting deployment to production. For more information, please see: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21266<br /><br /><small>February 11, 2026 22:01 UTC</small><br /><b>Monitoring</b> - The fix has been deployed, and we're monitoring the situation. We will resolve this incident if no further errors are observed. For more information, please see: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21266<br /><br /><small>February 11, 2026 22:30 UTC</small><br /><b>Resolved</b> - The fix has been deployed, and no further errors are apparent. This is now resolved. For more information, please see: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21266<br /><br />

<small>February 3, 2026 11:36 UTC</small><br /><b>Investigating</b> - We have received reports of delayed or stuck pipelines, and jobs waiting longer than expected to execute. We are investigating. See issue for more details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190<br /><br /><small>February 3, 2026 12:08 UTC</small><br /><b>Investigating</b> - Investigation is still underway. An MR to increase resource limits to mitigate impact is currently awaiting merge. See issue for details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190<br /><br /><small>February 3, 2026 12:39 UTC</small><br /><b>Investigating</b> - Resource limits have been increased and we are monitoring the impact. Investigation into the root cause is still in progress. See issue for details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190<br /><br /><small>February 3, 2026 13:51 UTC</small><br /><b>Investigating</b> - The resource limits increase did not sufficiently mitigate impact. Investigation into cause and mitigation options is still in progress. See issue for details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190<br /><br /><small>February 3, 2026 14:28 UTC</small><br /><b>Investigating</b> - Mitigation action has been taken and new pipelines can be triggered. Pipelines are being processed, but delays are still expected for up to 2 hours. Jobs may still be stuck in pending state during this time. See issue for details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190<br /><br /><small>February 3, 2026 15:19 UTC</small><br /><b>Identified</b> - We have identified the cause and taken action to mitigate re-occurance. We are seeing recovery on CI workloads and it should return to normal within the next hour. Some delays are still expected during this time.<br />See issue for details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190<br /><br /><small>February 3, 2026 15:31 UTC</small><br /><b>Monitoring</b> - CI workloads are recovering as expected and should return to normal within the next hour. Some pipelines may still be delayed during this time. We are currently monitoring.<br />See issue for details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190<br /><br /><small>February 3, 2026 17:21 UTC</small><br /><b>Identified</b> - We are continuing to see delays in CI job processing and are actively investigating mitigation steps. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190.<br /><br /><small>February 3, 2026 17:44 UTC</small><br /><b>Investigating</b> - In addition to CI job processing delays, users may also see "Failed to retrieve CI/CD variables from cache" when running new pipelines manually via the UI. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190.<br /><br /><small>February 3, 2026 18:09 UTC</small><br /><b>Investigating</b> - Issues with Hosted Runner CI job queue delays and cache errors related to CI/CD variables are still under investigation. Self-hosted Runners are not currently affected. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190.<br /><br /><small>February 3, 2026 18:43 UTC</small><br /><b>Investigating</b> - No material update available at this time. Investigation into CI job delays on Hosted Runners and CI variable retrieval is ongoing. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190.<br /><br /><small>February 3, 2026 20:33 UTC</small><br /><b>Investigating</b> - We have observed some improvement in queue times but are still investigating CI job delays on Hosted Runners and CI variable retrieval. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190.<br /><br /><small>February 3, 2026 23:29 UTC</small><br /><b>Investigating</b> - We have observed that jobs on Hosted Runners are processing without delay and are able to retrieve cached CI/CD variables. We are currently awaiting deployment of a fix to ensure the issue remains mitigated. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190.<br /><br /><small>February 4, 2026 01:29 UTC</small><br /><b>Investigating</b> - Hosted Runners continue to process without delay and are able to retrieve cached CI/CD variables. However some background processing is currently impacted and users may observe environments/review apps URLs not updating. We continue to wait for the fix to reach production. Details in gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190.<br /><br /><small>February 4, 2026 03:29 UTC</small><br /><b>Investigating</b> - Background processing remains degraded, affecting Environments & Review Apps. Hosted Runners and CI jobs are healthy. A revert MR is queued for production, with more changes needed to address background processing. See: gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190<br /><br /><small>February 4, 2026 06:31 UTC</small><br /><b>Investigating</b> - Background processing remains degraded, however the revert MR has been merged a short time ago. We are monitoring performance to see if the MR results in performance returning to optimal levels. <br />See: gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190<br /><br /><small>February 4, 2026 09:56 UTC</small><br /><b>Monitoring</b> - We've disabled the worker responsible for the issues. We've merged another fix for that specific worker and are in the process of deploying it to production before re-enabling it. The next update will be posted once that is completed.<br /><br /><small>February 4, 2026 10:33 UTC</small><br /><b>Identified</b> - Background processing remains degraded, affecting Environments and Review Apps, a fix is awaiting deployment. Hosted Runners and CI jobs are healthy. Details in gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190.<br /><br /><small>February 4, 2026 14:25 UTC</small><br /><b>Monitoring</b> - We have applied a fix and are now monitoring before marking as resolved. More details can be found in gitlab.com/gitlab-com/gl-infra/production/-/work_items/21190.<br /><br /><small>February 4, 2026 17:40 UTC</small><br /><b>Resolved</b> - This issue is now resolved. The CI job queue on Hosted Runners and Environments/Review Apps have remained stable during our monitoring period.<br /><br />

<small>February 3, 2026 13:11 UTC</small><br /><b>Investigating</b> - GitLab Duo is unavailable for all users in both the VSCode extension and the web interface. Attempting to use the agentic chat triggers an error: "GitLab Duo is already responding to this chat in another tab or location." See issue for details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21192<br /><br /><small>February 3, 2026 13:31 UTC</small><br /><b>Investigating</b> - GitLab Duo is still unavailable for all users in both the VSCode extension and the web interface. We are currently investigating.<br />See issue for details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21192<br /><br /><small>February 3, 2026 15:04 UTC</small><br /><b>Identified</b> - The issue has been identified to have been caused by a recent change. The change has been reverted and we expect Duo should be available for VSC and Web UI. See issue for details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21192<br /><br /><small>February 3, 2026 15:36 UTC</small><br /><b>Monitoring</b> - Duo availability is back to normal. We are currently monitoring. See issue for details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21192<br /><br /><small>February 3, 2026 17:52 UTC</small><br /><b>Resolved</b> - Duo is once again fully available from within the GitLab UI and Visual Studio Code.<br /><br />

<small>February 2, 2026 12:03 UTC</small><br /><b>Investigating</b> - Collapsing and expanding inline comment threads is broken for merge requests. We are investigating. Workaround is available, see issue for details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21184<br /><br /><small>February 2, 2026 12:39 UTC</small><br /><b>Identified</b> - We have identified the recent refactor of a Vue component as the cause. We are preparing a fix for the problematic MR.<br />Workaround is available, see issue for details: gitlab.com/gitlab-com/gl-infra/production/-/work_items/21184<br /><br /><small>February 2, 2026 15:52 UTC</small><br /><b>Identified</b> - The fix has been merged and is currently being deployed to production.<br />A workaround is available, see issue for details: gitlab.com/gitlab-com/gl-infra/production/-/work_items/21184<br /><br /><small>February 2, 2026 19:02 UTC</small><br /><b>Identified</b> - Deployment of the fix is progressing. We will provide another update once the fix has landed within the next ~4 hours.<br /><br /><small>February 2, 2026 23:23 UTC</small><br /><b>Resolved</b> - We've confirmed that the fix has been deployed and inline comment threads on merge requests can now be expanded and collapsed normally.<br /><br />

<small>February 2, 2026 22:40 UTC</small><br /><b>Identified</b> - We are experiencing elevated error rates affecting External Agents on the Duo Agent Platform caused by a recent deployment. We have reverted the offending merge request and are awaiting deployment of the fix. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21185.<br /><br /><small>February 2, 2026 23:10 UTC</small><br /><b>Resolved</b> - The fix has been deployed and we've observed the error rate return to 0%. This is now resolved.<br /><br />

<small>January 28, 2026 20:35 UTC</small><br /><b>Investigating</b> - We are currently seeing issues with starting pipelines on GitLab.com and are investigating the issue.<br /><br /><small>January 28, 2026 20:41 UTC</small><br /><b>Identified</b> - We identified that a recently enabled feature flag may have contributed to the pipeline failures on GitLab.com. The feature flag was just disabled, and we're now seeing a sharp drop in error rates.<br /><br />Initial checks confirm that pipelines can now be started again. We're monitoring to ensure the issue is resolved for all users.<br /><br />Details can be found in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21160<br /><br /><small>January 28, 2026 20:55 UTC</small><br /><b>Resolved</b> - We have identified and disabled a feature flag that was the cause. Error rates have returned to normal.<br /><br />Additional information can be found in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21160<br /><br />

<small>January 26, 2026 15:16 UTC</small><br /><b>Investigating</b> - Some GitLab users and automated monitoring tools may see 404 errors when accessing the infrastructure repository on gitlab.com.<br />Workaround: Clearing your browser cache and logging in again to start a new session may resolve the issue. You might need to clear the cache again after the incident if you use the workaround.<br />We're currently investigating and will provide updates as we learn more.<br /><br /><small>January 26, 2026 15:30 UTC</small><br /><b>Monitoring</b> - The access GitLab.com has been restored, with users now able to log in and access previously affected pages.<br />Both the sign-in page and repository URLs are working again for users, and monitoring alerts have resolved. The request rate for HTTP Router dropped sharply following the change, confirming the mitigation was effective.<br /><br /><small>January 26, 2026 16:19 UTC</small><br /><b>Monitoring</b> - No changes since the last update. Access to GitLab.com remains restored, and monitoring alerts continue to show normal operation. We're continuing to monitor before marking this issue as resolved.<br /><br /><small>January 26, 2026 20:15 UTC</small><br /><b>Resolved</b> - The issue has been resolved and systems have returned to normal. <br /><br />

<small>January 23, 2026 20:20 UTC</small><br /><b>Identified</b> - We are aware of an error occurring when attempting to update Duo settings for groups on GitLab.com. We have identified the problematic change and are working on a revert. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21138<br /><br /><small>January 23, 2026 21:27 UTC</small><br /><b>Identified</b> - A revert MR has been merged and we are working to get this deployed to production. This may take up to 5 hours or more. Details in: gitlab.com/gitlab-com/gl-infra/production/-/issues/21138<br /><br /><small>January 23, 2026 22:30 UTC</small><br /><b>Identified</b> - A partial workaround to this error is available: Duo availability for a group can be updated via the API. Please see the following comment in the production issue for important details and limitations: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/21138#note_3031631633<br /><br /><small>January 24, 2026 02:27 UTC</small><br /><b>Resolved</b> - A fix has now been deployed, and the issue is resolved. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/21138<br /><br />

<small>January 16, 2026 21:21 UTC</small><br /><b>Identified</b> - Foundational Flows are temporarily restricted to users with Beta/Experimental features enabled despite being a GA feature.<br /><br />Our team has identified a fix. As a workaround, you can enable Beta and Experimental features to enable Foundational Flows.<br /><br />- https://docs.gitlab.com/user/gitlab_duo/turn_on_off/#turn-on-beta-and-experimental-features<br />- https://docs.gitlab.com/user/duo_agent_platform/flows/foundational_flows/<br /><br /><small>January 20, 2026 06:10 UTC</small><br /><b>Identified</b> - Foundational flows now run correctly when experimental/beta features are disabled.<br /><br />Workaround: To enable foundational flows, temporarily turn on experimental/beta features, enable the flows, save, then disable experimental/beta features.<br /><br />A complete fix removing this workaround is in review and will be included in an upcoming patch release.<br /><br /><small>January 21, 2026 01:07 UTC</small><br /><b>Resolved</b> - The issue has been resolved. Foundational flows are now operating correctly as a GA feature. More details are available in https://gitlab.com/gitlab-org/gitlab/-/work_items/586912<br /><br />

<small>January 16, 2026 10:24 UTC</small><br /><b>Investigating</b> - We are currently investigating pipelines failing with "502 Bad Gateway" errors. More details available at https://gitlab.com/gitlab-com/gl-infra/production/-/issues/21096<br /><br /><small>January 16, 2026 10:48 UTC</small><br /><b>Investigating</b> - We are still investigating "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096<br /><br /><small>January 16, 2026 11:14 UTC</small><br /><b>Investigating</b> - We are investigating possible saturation. Users will still experience intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096<br /><br /><small>January 16, 2026 11:41 UTC</small><br /><b>Investigating</b> - We are still investigating "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096<br /><br /><small>January 16, 2026 12:16 UTC</small><br /><b>Investigating</b> - We are investigating a possible root cause, users still experience intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096<br /><br /><small>January 16, 2026 12:50 UTC</small><br /><b>Investigating</b> - We are investigating the rate limiting and related configuration changes is ongoing. Users might still experience intermittent 502 errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096<br /><br /><small>January 16, 2026 13:32 UTC</small><br /><b>Investigating</b> - We are still investigating the 502 intermittent errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096<br /><br /><small>January 16, 2026 14:16 UTC</small><br /><b>Investigating</b> - We are continuing to investigate possible backend performance bottlenecks and other upstream factors that could be causing the intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096<br /><br /><small>January 16, 2026 15:04 UTC</small><br /><b>Investigating</b> - We are continuing to investigate intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096<br /><br /><small>January 16, 2026 15:59 UTC</small><br /><b>Investigating</b> - Users still experience intermittent "502 Bad Gateway" errors when running pipelines. We are currently investigating previous Cloudflare configuration change. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096<br /><br /><small>January 16, 2026 16:52 UTC</small><br /><b>Investigating</b> - No material updates at this time. Users may still experience intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096<br /><br /><small>January 16, 2026 17:42 UTC</small><br /><b>Investigating</b> - Our team is currently working with Cloudflare to investigate the ongoing 502 errors. Users may still experience intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096<br /><br /><small>January 16, 2026 18:39 UTC</small><br /><b>Investigating</b> - No material updates at this time. Users may still experience intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096<br /><br /><small>January 16, 2026 19:38 UTC</small><br /><b>Investigating</b> - No material updates at this time. Users may still experience intermittent "502 Bad Gateway" errors when running pipelines. More details available at gitlab.com/gitlab-com/gl-infra/production/-/issues/21096<br /><br /><small>January 16, 2026 20:15 UTC</small><br /><b>Identified</b> - Our team has identified that the 502 Bad Gateway errors were related to a Cloudflare Worker deployment in the HTTP router. A rollback of the deployment is showing a decline in 502 errors for the artifact endpoints. Our team is continuing to monitor.<br /><br /><small>January 16, 2026 20:23 UTC</small><br /><b>Resolved</b> - This incident is now resolved. Our team has confirmed that the rollback fixed the issue and we are no longer seeing 502 Bad Gateway errors.<br /><br />

<small>January 8, 2026 03:15 UTC</small><br /><b>Investigating</b> - We are investigating degraded performance with classic Duo chat. Requests may fail intermittently at this time.<br /><br />For further details, see https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/21059<br /><br /><small>January 8, 2026 03:51 UTC</small><br /><b>Identified</b> - The cause of degraded performance has been identified. We are preparing a code change to revert the recent deployment that introduced the issue. <br /><br />A merge request has been opened to restore functionality: https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/merge_requests/4226<br /><br />For further details, see gitlab.com/gitlab-com/gl-infra/production/-/work_items/21059<br /><br /><small>January 8, 2026 04:22 UTC</small><br /><b>Identified</b> - We're deploying the revert change to production.<br /><br />For further details, see gitlab.com/gitlab-com/gl-infra/production/-/work_items/21059<br /><br /><small>January 8, 2026 04:55 UTC</small><br /><b>Monitoring</b> - We have deployed the changes to production. We're monitoring the situation.<br /><br />For further details, see gitlab.com/gitlab-com/gl-infra/production/-/work_items/21059<br /><br /><small>January 8, 2026 05:50 UTC</small><br /><b>Resolved</b> - We observe no more error on Classic Duo chat and confirm successful responses in the UI.<br /><br />For further details, see gitlab.com/gitlab-com/gl-infra/production/-/work_items/21059<br /><br />

<small>January 6, 2026 07:56 UTC</small><br /><b>Investigating</b> - We're investigating GitLab.com slow performance. <br /><br />

<small>December 22, 2025 15:38 UTC</small><br /><b>Investigating</b> - We are seeing increased errors for Duo Classic. Users are experiencing Error Code: A1004. For full details, see https://gitlab.com/gitlab-com/gl-infra/production/-/issues/21032<br /><br />

<small>December 15, 2025 20:36 UTC</small><br /><b>Investigating</b> - We're receiving reports that some CI/CD jobs are stuck in pending status. The cause is currently under investigation. Please see here for more information: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20996<br /><br /><small>December 15, 2025 20:37 UTC</small><br /><b>Investigating</b> - We're receiving reports that some CI/CD jobs are stuck in pending status. The cause is currently under investigation. Please see here for more information: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20996<br /><br />

<small>December 16, 2025 05:18 UTC</small><br /><b>Investigating</b> - We have identified a recent change that was causing GitLab Duo to fail to respond for some customers. We are working on a fix.<br /><br />

<small>December 10, 2025 15:01 UTC</small><br /><b>Identified</b> - We are seeing an increase in 500 errors from the SSH signature endpoints affecting commit-related pages for projects using SSH-signed commits. A possible workaround is to use `git pull` and work locally. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20989<br /><br />

<small>December 8, 2025 19:40 UTC</small><br /><b>Investigating</b> - We are investigating an issue in which requests to the Commits API is returning status code 400. Git operations remain operational. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20963.<br /><br />

<small>December 5, 2025 09:01 UTC</small><br /><b>Investigating</b> - GitLab.com is inaccessible at the moment due to a Cloudflare outage. We are currently investigating the cause. https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/20939<br /><br /><small>December 5, 2025 09:18 UTC</small><br /><b>Monitoring</b> - GitLab.com is becoming available for users. Cloudflare have applied a fix at their side at 09:12 UTC. We will continue to monitor the situation. https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/20939<br /><br />

<small>December 1, 2025 15:12 UTC</small><br /><b>Identified</b> - We are seeing a problem with the Merged tab of merge request homepage not loading. The workaround is to use the Search tab instead. We are working on rolling back the relevant MR. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/20914<br /><br />

<small>November 25, 2025 02:21 UTC</small><br /><b>Identified</b> - We've identified the root cause of pipeline creation failures affecting pipelines that use inputs since Nov 24, 11:00 UTC. A fix is in the process of being deployed and we're monitoring for resolution. More info here: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/213976<br /><br />

<small>November 21, 2025 11:56 UTC</small><br /><b>Investigating</b> - We've received reports that jobs that run on a specific macOS GitLab Runner (with the tag `saas-macos-large-m2pro`) have been failing with a “context deadline exceeded” error. We are in the process of investigating. As a workaround, you can switch to other runners. See https://gitlab.com/gitlab-com/gl-infra/production/-/issues/20878 for more details<br /><br />