tag:status.stellaconnect.net,2005:/historyStella Connect Status - Incident History2024-03-19T04:48:49-04:00Stella Connecttag:status.stellaconnect.net,2005:Incident/178683972023-07-16T09:31:54-04:002023-07-16T09:31:54-04:00Agent Connect Service Degradation<p><small>Jul <var data-var='date'>16</var>, <var data-var='time'>09:31</var> EDT</small><br><strong>Resolved</strong> - During this period, background job processing was unavailable for our web application for some users. This impacted areas of the application like fetching QA review transcripts and receiving email notifications when a QA review has been completed. This issue has been resolved and the affected components will no longer experience this issue.</p><p><small>Jul <var data-var='date'>16</var>, <var data-var='time'>07:00</var> EDT</small><br><strong>Update</strong> - We appreciate your patience as our team continues to investigate the issue causing increased error occurrence when conducting QA Reviews in the Agent Connect application. We'll be sure to provide updates here as soon as they become available!</p><p><small>Jul <var data-var='date'>16</var>, <var data-var='time'>05:29</var> EDT</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Jul <var data-var='date'>16</var>, <var data-var='time'>04:54</var> EDT</small><br><strong>Investigating</strong> - Our team is aware of and investigating a service degradation -- users may experience errors in the application, including when attempting to conduct a QA Review. We are assessing any other possible areas of impact and will provide updates here as soon as possible. Thank you for your patience and please do not hesitate to reach out to our support team with any questions!</p>tag:status.stellaconnect.net,2005:Incident/177327062023-07-05T09:31:35-04:002023-07-05T09:31:35-04:00Scheduled Maintenance<p><small>Jul <var data-var='date'> 5</var>, <var data-var='time'>09:31</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jul <var data-var='date'> 5</var>, <var data-var='time'>07:31</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'>30</var>, <var data-var='time'>16:12</var> EDT</small><br><strong>Scheduled</strong> - We will be going through a scheduled maintenance with our SFTP IPs and deprecation of insecure ciphers at this time. Customers who may be affected have already been notified and provided with steps to update their integration to prevent this maintenance from disrupting their integrations.<br /><br />If you think your integration has been affected by this maintenance, please reach out to our Support team at support@stellaconnect.net.</p>tag:status.stellaconnect.net,2005:Incident/175849402023-06-15T15:01:40-04:002023-06-15T15:01:40-04:00Outage in Europe environment<p><small>Jun <var data-var='date'>15</var>, <var data-var='time'>15:01</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'>15</var>, <var data-var='time'>14:02</var> EDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Jun <var data-var='date'>15</var>, <var data-var='time'>13:26</var> EDT</small><br><strong>Investigating</strong> - We are currently experiencing an outage for all customers deployed on our European environment.</p>tag:status.stellaconnect.net,2005:Incident/171462872023-05-08T17:32:20-04:002023-05-08T17:32:20-04:00Upcoming Database Maintenance for EMEA Customers Only<p><small>May <var data-var='date'> 8</var>, <var data-var='time'>17:32</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>May <var data-var='date'> 8</var>, <var data-var='time'>16:00</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>May <var data-var='date'> 4</var>, <var data-var='time'>14:39</var> EDT</small><br><strong>Scheduled</strong> - We will be undergoing maintenance at this time for EMEA customers. Users will not be able to send surveys nor respond to them during the scheduled maintenance. All systems will be functional for customers in other regions.</p>tag:status.stellaconnect.net,2005:Incident/162272762023-02-22T01:00:28-05:002023-02-22T01:00:29-05:00Database maintenance<p><small>Feb <var data-var='date'>22</var>, <var data-var='time'>01:00</var> EST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'>22</var>, <var data-var='time'>00:00</var> EST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>17:12</var> EST</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.stellaconnect.net,2005:Incident/160705602023-02-08T22:40:03-05:002023-02-08T22:40:03-05:00Stella Connect Service Degradation Notification<p><small>Feb <var data-var='date'> 8</var>, <var data-var='time'>22:40</var> EST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'> 8</var>, <var data-var='time'>22:32</var> EST</small><br><strong>Investigating</strong> - We are currently experiencing a service degradation on Stella Connect. We are aware of the incident and will provide an update within the hour.</p>tag:status.stellaconnect.net,2005:Incident/156948342022-12-22T13:05:49-05:002022-12-22T13:05:49-05:00Stella Connect Service Degradation Notification<p><small>Dec <var data-var='date'>22</var>, <var data-var='time'>13:05</var> EST</small><br><strong>Resolved</strong> - From approximately 4:35pm to 5:00pm UTC the Agent Connect API app was unavailable. This impacted features like exports, webhooks, and engagement emails. This was caused by an outage in our background queuing service. During this time, survey requests and survey responses were not processed and cannot be recovered. We apologize for the inconvenience this has caused.</p><p><small>Dec <var data-var='date'>22</var>, <var data-var='time'>12:05</var> EST</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Dec <var data-var='date'>22</var>, <var data-var='time'>11:34</var> EST</small><br><strong>Investigating</strong> - We are currently experiencing a service degradation on Stella Connect. We are aware of the incident and will provide an update within the hour.</p>tag:status.stellaconnect.net,2005:Incident/153515042022-12-16T14:40:37-05:002022-12-16T14:40:37-05:00Stella Connect Service Degradation Notification<p><small>Dec <var data-var='date'>16</var>, <var data-var='time'>14:40</var> EST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'>16</var>, <var data-var='time'>14:16</var> EST</small><br><strong>Investigating</strong> - We had previously experienced intermittent errors for a 5 minute period during a deployment. This may have caused error alerts in the Agent Connect application. We have mitigated the issue and customers should no longer be seeing alerts.</p>tag:status.stellaconnect.net,2005:Incident/146732432022-12-08T11:27:38-05:002022-12-08T11:27:38-05:00Agent Connect Monitoring Alert<p><small>Dec <var data-var='date'> 8</var>, <var data-var='time'>11:27</var> EST</small><br><strong>Resolved</strong> - Please note that this incident had no impact to our clients or their data and simply alerted for monitoring purposes. Thank you for your patience and understanding!</p><p><small>Dec <var data-var='date'> 8</var>, <var data-var='time'>10:57</var> EST</small><br><strong>Investigating</strong> - We are currently experiencing a service degradation on Stella Connect. We are aware of the incident and will provide an update within the hour.</p>tag:status.stellaconnect.net,2005:Incident/126475192022-10-28T13:01:28-04:002022-10-28T13:01:28-04:00Stella Connect Service Degradation Notification<p><small>Oct <var data-var='date'>28</var>, <var data-var='time'>13:01</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'>28</var>, <var data-var='time'>12:15</var> EDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Oct <var data-var='date'>28</var>, <var data-var='time'>11:59</var> EDT</small><br><strong>Investigating</strong> - We are currently experiencing a service degradation on Stella Connect. We are aware of the incident and will provide an update within the hour.</p>tag:status.stellaconnect.net,2005:Incident/126234372022-10-26T10:42:49-04:002022-10-26T10:42:49-04:00Agent Connect [fka Stella Connect] Application Outage<p><small>Oct <var data-var='date'>26</var>, <var data-var='time'>10:42</var> EDT</small><br><strong>Resolved</strong> - Our team has confirmed full resolution of this morning's incident which caused ~20 minutes of application inaccessibility for users. We sincerely apologize for any inconvenience this may have caused. If you have any questions or your team requires assistance, please reach out to our Support Team via the Help (?) icon in the top right-hand corner of the Agent Connect application and choose Contact Us from the dropdown menu to begin chatting with the team -- we're here to help!</p><p><small>Oct <var data-var='date'>26</var>, <var data-var='time'>09:34</var> EDT</small><br><strong>Monitoring</strong> - The issue causing lost access to the Agent Connect application has been identified and our team has found resolution -- we will continue monitoring for a period to ensure continued application stability. Thank you for your patience!</p><p><small>Oct <var data-var='date'>26</var>, <var data-var='time'>09:20</var> EDT</small><br><strong>Update</strong> - Our engineering team is currently investigating an application outage which is affecting end-user access to log into and interact the Agent Connect [fka Stella Connect] application. Our team will continue working to identify and resolve the issue to resume normal access to the application as soon as possible and we will keep you up to date here. Thank you for your patience!</p><p><small>Oct <var data-var='date'>26</var>, <var data-var='time'>09:00</var> EDT</small><br><strong>Investigating</strong> - We are currently experiencing a service degradation on Stella Connect. We are aware of the incident and will provide an update within the hour.</p>tag:status.stellaconnect.net,2005:Incident/108918612022-08-23T19:42:48-04:002022-08-23T19:42:48-04:00Images cannot be uploaded<p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>19:42</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>16:44</var> EDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>13:12</var> EDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>10:39</var> EDT</small><br><strong>Investigating</strong> - Images cannot be uploaded, preventing the update of profile pictures, company logos, etc.</p>tag:status.stellaconnect.net,2005:Incident/108612052022-08-19T13:56:16-04:002022-08-19T13:56:16-04:00QA dashboards not loading for several clients<p><small>Aug <var data-var='date'>19</var>, <var data-var='time'>13:56</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>19</var>, <var data-var='time'>12:05</var> EDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Aug <var data-var='date'>19</var>, <var data-var='time'>10:59</var> EDT</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.stellaconnect.net,2005:Incident/106637402022-07-21T12:04:00-04:002022-07-21T16:51:10-04:00Web application unavailable<p><small>Jul <var data-var='date'>21</var>, <var data-var='time'>12:04</var> EDT</small><br><strong>Resolved</strong> - The web UI was unavailable from 12:04:30 - 12:08:00 EST. There was no impact to survey requests and responses.</p>tag:status.stellaconnect.net,2005:Incident/105959762022-07-15T13:30:00-04:002022-07-15T16:31:42-04:00Non-admins unable to access QA dashboards<p><small>Jul <var data-var='date'>15</var>, <var data-var='time'>13:30</var> EDT</small><br><strong>Resolved</strong> - From approximately 17:30 UTC to 19:30 UTC non-admin users received an error while loading QA dashboards in the application.</p>tag:status.stellaconnect.net,2005:Incident/105030852022-07-08T13:12:56-04:002022-07-08T13:12:56-04:00Transactional email provider errors<p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>13:12</var> EDT</small><br><strong>Resolved</strong> - The issue has been resolved. We are continuing to monitor our transactional email provider for any new issues.</p><p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>11:20</var> EDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>11:19</var> EDT</small><br><strong>Update</strong> - We have noticed that the errors with the transactional email provider have caused some internal emails, like QA review notifications, to be delivered twice. Our team is working on mitigating this issue within our application. This issue has also caused some feedback updates to be delayed in the system. A fix for this issue is also actively in development.</p><p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>10:54</var> EDT</small><br><strong>Investigating</strong> - We are currently investigating an issue with our transactional email provider. This issue will delay surveys sent via email and all internal emails, such as trends summaries and new QA review alerts.</p>tag:status.stellaconnect.net,2005:Incident/98144612022-04-19T18:54:25-04:002022-04-19T18:54:25-04:00Issue with in progress exports<p><small>Apr <var data-var='date'>19</var>, <var data-var='time'>18:54</var> EDT</small><br><strong>Resolved</strong> - A fix has been deployed.</p><p><small>Apr <var data-var='date'>19</var>, <var data-var='time'>14:43</var> EDT</small><br><strong>Investigating</strong> - We are investing an issue starting yesterday where some users are unable to perform a new export due to a warning message that they already have an export in progress.</p>tag:status.stellaconnect.net,2005:Incident/94830682022-03-07T17:20:48-05:002022-03-07T17:20:48-05:00Stella Connect Service Degradation Notification<p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>17:20</var> EST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>14:06</var> EST</small><br><strong>Investigating</strong> - We are currently experiencing a service degradation on Stella Connect. We are aware of the incident and will provide an update within the hour.</p>tag:status.stellaconnect.net,2005:Incident/94313242022-03-01T10:34:07-05:002022-03-01T10:34:07-05:00Feedback exports error<p><small>Mar <var data-var='date'> 1</var>, <var data-var='time'>10:34</var> EST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>19:29</var> EST</small><br><strong>Identified</strong> - We have identified an issue preventing users from accessing feedback exports in the US data center. A fix is being worked on.</p>tag:status.stellaconnect.net,2005:Incident/92787292022-02-09T17:44:49-05:002022-02-09T17:44:49-05:00Degraded performance in US data center<p><small>Feb <var data-var='date'> 9</var>, <var data-var='time'>17:44</var> EST</small><br><strong>Resolved</strong> - From 5:09PM to 5:30PM EST the application in the US data center experienced intermitted errors and is now resolved.</p>tag:status.stellaconnect.net,2005:Incident/92599342022-02-07T19:08:00-05:002022-02-07T19:08:00-05:00Stella Connect Service Degradation Notification<p><small>Feb <var data-var='date'> 7</var>, <var data-var='time'>19:08</var> EST</small><br><strong>Resolved</strong> - From approximately 3:40-3:50 EST, the survey requests API experienced elevated error rates.</p><p><small>Feb <var data-var='date'> 7</var>, <var data-var='time'>15:53</var> EST</small><br><strong>Investigating</strong> - We are currently experiencing a service degradation on Stella Connect. We are aware of the incident and will provide an update within the hour.</p>tag:status.stellaconnect.net,2005:Incident/92317872022-02-03T15:46:11-05:002022-02-03T15:46:11-05:00Scheduled maintenance of German environment<p><small>Feb <var data-var='date'> 3</var>, <var data-var='time'>15:46</var> EST</small><br><strong>Resolved</strong> - Maintenance on the Germany environment is now complete.</p><p><small>Feb <var data-var='date'> 3</var>, <var data-var='time'>14:27</var> EST</small><br><strong>Investigating</strong> - The environment in our German data center is undergoing scheduled maintenance. The application will be unavailable during this window. The application in US East will be unaffected.</p>tag:status.stellaconnect.net,2005:Incident/90953772022-01-16T03:30:00-05:002022-01-16T05:24:20-05:00Scheduled maintenance<p><small>Jan <var data-var='date'>16</var>, <var data-var='time'>03:30</var> EST</small><br><strong>Resolved</strong> - The site underwent scheduled maintenance from 8:30AM through 9:00AM UTC. During that period, the application experienced increased error rates. All services are now back online.</p>tag:status.stellaconnect.net,2005:Incident/87822222021-12-07T20:50:38-05:002021-12-07T20:50:38-05:00Stella Connect Survey Intermittent Errors Due to 3rd Party Downtime<p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>20:50</var> EST</small><br><strong>Resolved</strong> - AWS has resolved their outage and our team has seen all error rates drop to normal rates. Thank you for your patience!</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>14:34</var> EST</small><br><strong>Update</strong> - Our team is continuing to monitor the ongoing AWS service degradation causing some intermittent API and application errors for Stella Connect. We apologize for the inconvenience and will keep posting updates as the AWS team gets services back up and running consistently.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>12:38</var> EST</small><br><strong>Update</strong> - Errors have dropped significantly, however, AWS is still experiencing downgraded service. For this reason, we will leave this incident open and our team will continue to monitor all queues until the downgraded performance status has been resolved for AWS.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>11:55</var> EST</small><br><strong>Monitoring</strong> - We are continuing to monitor and seeing error rates drop slowly and AWS is aware of the issue. We will update this page with any further information as it becomes available.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>11:50</var> EST</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>11:33</var> EST</small><br><strong>Investigating</strong> - It appears that AWS is currently experiencing a widespread outage, which may cause some errors for those attempting to fill out a Stella Connect survey during this time. We are still collecting responses at a high rate and will continue to monitor.</p>tag:status.stellaconnect.net,2005:Incident/87719042021-12-06T18:09:31-05:002021-12-06T18:14:27-05:00Survey Sending Issues in German Data Center<p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>18:09</var> EST</small><br><strong>Resolved</strong> - Our engineers have determined that survey sending was delayed for some clients in the German data center and this issue was resolved at 9:42am ET. All services are back to fully operational and all delayed surveys have been processed.</p><p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>12:32</var> EST</small><br><strong>Update</strong> - We have confirmed survey sending is back up and began recovering at 9:42AM ET. We are now assessing impact of the outage.</p><p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>09:35</var> EST</small><br><strong>Investigating</strong> - We are currently experiencing issues with survey sending for clients in our German data center. Our engineers are investigating and we will provide updates as they become available. This ONLY impacts clients in our German Data Center.</p>