Incidents | TeamRetro Incidents reported on status page for TeamRetro https://teamretro.betteruptime.com/ https://d1lppblt9t2x15.cloudfront.net/logos/ab924d071908a91a467240f6f387e209.png Incidents | TeamRetro https://teamretro.betteruptime.com/ en Issue creating first retrospective / health check https://teamretro.betteruptime.com/incident/539175 Thu, 03 Apr 2025 07:44:00 -0000 https://teamretro.betteruptime.com/incident/539175#f26e45684cb063b5e495dfaa4f86117fa8dd69c37ad6514de46ef96a84d6fa85 The issue has been resolved. Thank you for your patience. Issue creating first retrospective / health check https://teamretro.betteruptime.com/incident/539175 Thu, 03 Apr 2025 07:44:00 -0000 https://teamretro.betteruptime.com/incident/539175#f26e45684cb063b5e495dfaa4f86117fa8dd69c37ad6514de46ef96a84d6fa85 The issue has been resolved. Thank you for your patience. Issue creating first retrospective / health check https://teamretro.betteruptime.com/incident/539175 Thu, 03 Apr 2025 07:26:00 -0000 https://teamretro.betteruptime.com/incident/539175#0c653387ad481b98f61d775651822b13babd45a64afd39ca9c649ce182c28d33 Some customers have reported Issues creating the first retrospective or health check within a team. The issue has been identified and a fix is currently being deployed. Issue creating first retrospective / health check https://teamretro.betteruptime.com/incident/539175 Thu, 03 Apr 2025 07:26:00 -0000 https://teamretro.betteruptime.com/incident/539175#0c653387ad481b98f61d775651822b13babd45a64afd39ca9c649ce182c28d33 Some customers have reported Issues creating the first retrospective or health check within a team. The issue has been identified and a fix is currently being deployed. Email Delivery Issues https://teamretro.betteruptime.com/incident/537102 Mon, 31 Mar 2025 04:31:00 -0000 https://teamretro.betteruptime.com/incident/537102#0fe15205b700475d60f9fa25804e64e70b22527539e71bcf1b77c31fe5ceef6a Our email provider has resolved the delivery issues, and queued emails are now in the process of being delivered. Email Delivery Issues https://teamretro.betteruptime.com/incident/537102 Mon, 31 Mar 2025 04:31:00 -0000 https://teamretro.betteruptime.com/incident/537102#0fe15205b700475d60f9fa25804e64e70b22527539e71bcf1b77c31fe5ceef6a Our email provider has resolved the delivery issues, and queued emails are now in the process of being delivered. Email Delivery Issues https://teamretro.betteruptime.com/incident/537102 Sun, 30 Mar 2025 04:01:00 -0000 https://teamretro.betteruptime.com/incident/537102#9711ebfb4e7074142a31260c3e14037435ddd87c5ac97ac7d1affec1e3dd8670 We are currently investigating an issue affecting email delivery for our platform. This may impact notification emails, password resets, and other system-generated messages. Our team is actively working to identify the root cause and implement a fix. We will provide further updates as more information becomes available. Thank you for your patience. Email Delivery Issues https://teamretro.betteruptime.com/incident/537102 Sun, 30 Mar 2025 04:01:00 -0000 https://teamretro.betteruptime.com/incident/537102#9711ebfb4e7074142a31260c3e14037435ddd87c5ac97ac7d1affec1e3dd8670 We are currently investigating an issue affecting email delivery for our platform. This may impact notification emails, password resets, and other system-generated messages. Our team is actively working to identify the root cause and implement a fix. We will provide further updates as more information becomes available. Thank you for your patience. Scheduled Downtime for Database Upgrades https://teamretro.betteruptime.com/incident/464510 Sun, 24 Nov 2024 11:00:49 +0000 https://teamretro.betteruptime.com/incident/464510#43df7170e57fe0370fbae8e67dfd799f5be589047e1b80548cb3a83ea6eb5b6b Maintenance completed Scheduled Downtime for Database Upgrades https://teamretro.betteruptime.com/incident/464510 Sun, 24 Nov 2024 11:00:49 +0000 https://teamretro.betteruptime.com/incident/464510#43df7170e57fe0370fbae8e67dfd799f5be589047e1b80548cb3a83ea6eb5b6b Maintenance completed Scheduled Downtime for Database Upgrades https://teamretro.betteruptime.com/incident/464510 Sun, 24 Nov 2024 08:00:49 -0000 https://teamretro.betteruptime.com/incident/464510#add8577ebf8b8c15d548a320396efbaf5ae7dd4e6d20bdd5369ed1ed2a633069 During this time we'll be performing scheduled maintenance to upgrade our databases, this will likely involve up to 1 hour of downtime per environment in the maintenance window. Scheduled Downtime for Database Upgrades https://teamretro.betteruptime.com/incident/464510 Sun, 24 Nov 2024 08:00:49 -0000 https://teamretro.betteruptime.com/incident/464510#add8577ebf8b8c15d548a320396efbaf5ae7dd4e6d20bdd5369ed1ed2a633069 During this time we'll be performing scheduled maintenance to upgrade our databases, this will likely involve up to 1 hour of downtime per environment in the maintenance window. US site unavailable https://teamretro.betteruptime.com/incident/454286 Fri, 01 Nov 2024 05:42:00 -0000 https://teamretro.betteruptime.com/incident/454286#d57229e7717abd1ee0618fb0b6ba1345050b24443007d494bb0b41e9614fb988 We've identified the cause of the issue and deployed a fix, services are back online. Our team will continue to monitor closely. US site unavailable https://teamretro.betteruptime.com/incident/454286 Fri, 01 Nov 2024 05:21:00 -0000 https://teamretro.betteruptime.com/incident/454286#8c4878a9047beac03e960f2ae5643d6cad2bfd95d04bd4b21ed705452abf362b Our US environment is currently unavailable with a 503 error. The team is currently investigating and will update shortly. AI Grouping / Summarization Unavailable in US Environment https://teamretro.betteruptime.com/incident/420576 Wed, 28 Aug 2024 13:21:00 -0000 https://teamretro.betteruptime.com/incident/420576#624484e5e9fe2a7718911a7f161d15684b649d0b6c9ffe62d6ee41f83d0c1e72 We've restored use of the Claude 3.5 Sonnet model for AI summarization and grouping. AI Grouping / Summarization Unavailable in US Environment https://teamretro.betteruptime.com/incident/420576 Wed, 28 Aug 2024 08:59:00 -0000 https://teamretro.betteruptime.com/incident/420576#b4fa948963e320b53d13fddf53674a8821c037c8f0de6e5db5f1a7626f03da32 After confirming the AWS Bedrock issue is only affecting certain LLM models (Claude 3.5 Sonnet) we've switched to the previous version of the model, restoring AI grouping and summarization (although not quite as intelligent as the newer model). We'll continue to monitor the Claude 3.5 Sonnet availability and revert back once stability is restored. AI Grouping / Summarization Unavailable in US Environment https://teamretro.betteruptime.com/incident/420576 Wed, 28 Aug 2024 07:45:00 -0000 https://teamretro.betteruptime.com/incident/420576#1b78d2ffffe1e8c340e025e59ee0708e8e995dfeaba843660723c29736da3537 AWS Bedrock in us-east-1 region is currently experiencing service availability issues. AI grouping and summarization is currently unavailable in the TeamRetro US region. We are monitoring the situation and will update in 60 minutes. Voting is unavailable https://teamretro.betteruptime.com/incident/382053 Mon, 10 Jun 2024 04:36:00 -0000 https://teamretro.betteruptime.com/incident/382053#b671da1449249595947609fbbe1b3202f037bc2392cfde3cfc866cc2aa6dddc0 A fix has been deployed and new votes can be cast. Voting is unavailable https://teamretro.betteruptime.com/incident/382053 Mon, 10 Jun 2024 04:36:00 -0000 https://teamretro.betteruptime.com/incident/382053#b671da1449249595947609fbbe1b3202f037bc2392cfde3cfc866cc2aa6dddc0 A fix has been deployed and new votes can be cast. Voting is unavailable https://teamretro.betteruptime.com/incident/382053 Sun, 09 Jun 2024 14:35:00 -0000 https://teamretro.betteruptime.com/incident/382053#a8f95089dc23d5a0b413fe17a97eb3fa441b2c00ba6e0edb7fb51131534751f3 Due to a migration issue, users have been unable to cast new votes since deployment of the new Vote on Ideas funcitonality. A fix being prepared. Voting is unavailable https://teamretro.betteruptime.com/incident/382053 Sun, 09 Jun 2024 14:35:00 -0000 https://teamretro.betteruptime.com/incident/382053#a8f95089dc23d5a0b413fe17a97eb3fa441b2c00ba6e0edb7fb51131534751f3 Due to a migration issue, users have been unable to cast new votes since deployment of the new Vote on Ideas funcitonality. A fix being prepared. EU Database Upgrade https://teamretro.betteruptime.com/incident/305687 Thu, 28 Dec 2023 03:50:00 +0000 https://teamretro.betteruptime.com/incident/305687#d9cb24f65980f337dcc0cd325498041baf3d70ca3a0dac7686e9a5cb3b3711f4 Maintenance completed EU Database Upgrade https://teamretro.betteruptime.com/incident/305687 Thu, 28 Dec 2023 02:50:00 -0000 https://teamretro.betteruptime.com/incident/305687#467014b507e62604ea53497686f6f92fd2e409bb88a4baa1eee718c9168413cf We're upgrading our TeamRetro EU database instances for improved performance. CDN Issues https://teamretro.betteruptime.com/incident/294049 Thu, 30 Nov 2023 05:39:00 -0000 https://teamretro.betteruptime.com/incident/294049#f0e5304da9cf5994f17be7cd35758e85e52883643e91bbcd83250f3597eee8df Our logs indicate this issue has been resolved. We will continue to monitor closely. CDN Issues https://teamretro.betteruptime.com/incident/294049 Thu, 30 Nov 2023 05:39:00 -0000 https://teamretro.betteruptime.com/incident/294049#c3bb888f838f5a831218235c81c93564efe1ea9e7af00ba455e72a245e0964f3 Some users have reported issues loading TeamRetro application pages, due to CDN access issues from specific Points of Presence (CDN PoP). Our teams has cleared our CDN caches and redeployed to immediately resolve the issue; and continues to investigate the root cause. Degraded real-time synchronization (US environment) https://teamretro.betteruptime.com/incident/256059 Wed, 06 Sep 2023 20:00:00 -0000 https://teamretro.betteruptime.com/incident/256059#a131c9473691e83fddd318985b517aad25400ecb8c5a7f2d90c092a175cac31b Our real-time synchronization provider is reporting the issues have been resolved (https://status.pusher.com). Degraded real-time synchronization (US environment) https://teamretro.betteruptime.com/incident/256059 Wed, 06 Sep 2023 14:00:00 -0000 https://teamretro.betteruptime.com/incident/256059#2c0de7005dc6f724a92376bfb700af250361206437605504cda4b8cfc22c2bf8 Our real-time synchronization provider is experiencing degraded performance; resulting in delays synchronizing changes between participants (https://status.pusher.com) in the US environment. Our EU environment is unaffected. We continue to monitor the situation. EU Database Upgrade https://teamretro.betteruptime.com/incident/235213 Sun, 23 Jul 2023 03:00:00 +0000 https://teamretro.betteruptime.com/incident/235213#e6652c5dd9e2f11faf8eed8955447a0fac37743a6f6e51309d4a5559d335d8d0 Maintenance completed EU Database Upgrade https://teamretro.betteruptime.com/incident/235213 Sun, 23 Jul 2023 02:00:00 -0000 https://teamretro.betteruptime.com/incident/235213#5f9e5808b558e5f28d62bd178d5a5b01e5220e4691ac977a08ec22e8abe30a65 The TeamRetro EU environment will be taken offline temporarily as we migrate to newer larger database instances. Issue with SAML SSO first time users https://teamretro.betteruptime.com/incident/219925 Wed, 14 Jun 2023 04:54:00 -0000 https://teamretro.betteruptime.com/incident/219925#4024aec5e1285fed10297885e9ca03e7e1232fe6ea7c37234002b93a91189ccd A fix has been deployed and tested. The issue is now resolved. Issue with SAML SSO first time users https://teamretro.betteruptime.com/incident/219925 Wed, 14 Jun 2023 04:54:00 -0000 https://teamretro.betteruptime.com/incident/219925#4024aec5e1285fed10297885e9ca03e7e1232fe6ea7c37234002b93a91189ccd A fix has been deployed and tested. The issue is now resolved. Issue with SAML SSO first time users https://teamretro.betteruptime.com/incident/219925 Tue, 13 Jun 2023 04:00:00 -0000 https://teamretro.betteruptime.com/incident/219925#237c552465e60bd78291e4cb2f74e8c30ed35885785cc795ab52de04441d307b We've been seeing elevated error rates for first-time logins from SAML single sign on users. The issue has been identified and a fix is being prepared. Issue with SAML SSO first time users https://teamretro.betteruptime.com/incident/219925 Tue, 13 Jun 2023 04:00:00 -0000 https://teamretro.betteruptime.com/incident/219925#237c552465e60bd78291e4cb2f74e8c30ed35885785cc795ab52de04441d307b We've been seeing elevated error rates for first-time logins from SAML single sign on users. The issue has been identified and a fix is being prepared. Database Maintenance https://teamretro.betteruptime.com/incident/217879 Sun, 11 Jun 2023 02:00:00 +0000 https://teamretro.betteruptime.com/incident/217879#135d99255cdfc2b1b963a3f84d75d92d6592c253975bf3f862c586b765eacfce Maintenance completed Database Maintenance https://teamretro.betteruptime.com/incident/217879 Sun, 11 Jun 2023 02:00:00 -0000 https://teamretro.betteruptime.com/incident/217879#e326f4c59a5f212abba729dc30e67e3cd89e5bef12493276ac266274e16a3e22 We'll be performing database maintenance on our US environment. The EU environment will not be impacted. Database Upgrades https://teamretro.betteruptime.com/incident/213633 Sat, 03 Jun 2023 13:50:00 +0000 https://teamretro.betteruptime.com/incident/213633#4075a7d958f987603068b0f575dc2d29b4462f4ceb38d670ad2f60f9b88d0225 Maintenance completed Database Upgrades https://teamretro.betteruptime.com/incident/213633 Sat, 03 Jun 2023 13:50:00 +0000 https://teamretro.betteruptime.com/incident/213633#4075a7d958f987603068b0f575dc2d29b4462f4ceb38d670ad2f60f9b88d0225 Maintenance completed Database Upgrades https://teamretro.betteruptime.com/incident/213633 Sat, 03 Jun 2023 13:30:00 -0000 https://teamretro.betteruptime.com/incident/213633#522cafe95a6589aaf78ce3b1608b9159d50a47817673c671bfc9f93a4951dcb1 We'll be shifting our database to higher-performance AWS instance types during this scheduled downtime. Database Upgrades https://teamretro.betteruptime.com/incident/213633 Sat, 03 Jun 2023 13:30:00 -0000 https://teamretro.betteruptime.com/incident/213633#522cafe95a6589aaf78ce3b1608b9159d50a47817673c671bfc9f93a4951dcb1 We'll be shifting our database to higher-performance AWS instance types during this scheduled downtime. Database Upgrades https://teamretro.betteruptime.com/incident/196747 Sun, 16 Apr 2023 05:00:00 +0000 https://teamretro.betteruptime.com/incident/196747#61e4632afbdc0ee9295a372abb0c6fe641466f23c50ef2afb4b501adeca78599 Maintenance completed Database Upgrades https://teamretro.betteruptime.com/incident/196747 Sun, 16 Apr 2023 05:00:00 +0000 https://teamretro.betteruptime.com/incident/196747#61e4632afbdc0ee9295a372abb0c6fe641466f23c50ef2afb4b501adeca78599 Maintenance completed Database Upgrades https://teamretro.betteruptime.com/incident/196747 Sun, 16 Apr 2023 03:00:00 -0000 https://teamretro.betteruptime.com/incident/196747#9be3df1c3acfc1c12e9bdec1de54fe9b2ad15cb90f5a7a3a3a3877944a8c8a6d We are upgrading to larger database instances, will involve limited downtime (approximately 20 minutes) Database Upgrades https://teamretro.betteruptime.com/incident/196747 Sun, 16 Apr 2023 03:00:00 -0000 https://teamretro.betteruptime.com/incident/196747#9be3df1c3acfc1c12e9bdec1de54fe9b2ad15cb90f5a7a3a3a3877944a8c8a6d We are upgrading to larger database instances, will involve limited downtime (approximately 20 minutes)