Incidents reported on status page for Verosint https://status.verosint.com/ https://d1lppblt9t2x15.cloudfront.net/logos/009626154e05d7343cecda5e2e287531.jpeg https://status.verosint.com/ en https://status.verosint.com/incident/574309 Wed, 21 May 2025 14:32:00 -0000 https://status.verosint.com/incident/574309#2572d39ec3e5877dfb9bd3ddba5a4490b11cd590773943f24a2f10bf48db9c6d Cause of the latency has been identified and the delay is beginning to recover. At peak, signal data was about 10 minutes behind on average. Root cause was due to an abnormal combination of signal event volume and downstream signal enrichment workload. We expect full recovery in 15-20 minutes. We apologize for any inconvenience resulting from this issue and appreciate your patience while we work through the resolution. Please reach out to support@verosint.com with any questions. https://status.verosint.com/incident/574309 Wed, 21 May 2025 14:13:00 -0000 https://status.verosint.com/incident/574309#28d95ee35cf62308ddc0b074b93b957463c4a1927f7ae41cb47b9816d9dde725 We have observed elevated processing times of ingested signals starting at 8:35 AM CDT and are investigating the issue. While the issue is ongoing, signal data in the dashboard may be stale. The rules API is functioning normally and evaluations are not directly affected by this issue, but their results may also be impacted by stale data. We will update again when we have more information. https://status.verosint.com/incident/540027 Thu, 01 May 2025 04:51:39 -0000 https://status.verosint.com/incident/540027#077df9d4ce0e0995d73532bbf67c47b6df37403844c8c551b360fb482757a448 Verosint's back-end systems will undergo maintenance on **Wednesday, April 30th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) * Session management data back-end upgrade (minor service disruption) Potential scope of impact includes the `SignalPrint` and `Rules` APIs, as well as some functionality in the application front end. Most of this work will be highly-available, but the session management database operation will result in a brief period (estimated 45-90 seconds) where API token validation requests may time out. If that short period will be highly disruptive for your users and doing so is practical, please make any necessary preparations to prevent such disruptions, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/540027 Thu, 01 May 2025 04:51:39 -0000 https://status.verosint.com/incident/540027#077df9d4ce0e0995d73532bbf67c47b6df37403844c8c551b360fb482757a448 Verosint's back-end systems will undergo maintenance on **Wednesday, April 30th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) * Session management data back-end upgrade (minor service disruption) Potential scope of impact includes the `SignalPrint` and `Rules` APIs, as well as some functionality in the application front end. Most of this work will be highly-available, but the session management database operation will result in a brief period (estimated 45-90 seconds) where API token validation requests may time out. If that short period will be highly disruptive for your users and doing so is practical, please make any necessary preparations to prevent such disruptions, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/540027 Thu, 01 May 2025 04:51:39 -0000 https://status.verosint.com/incident/540027#077df9d4ce0e0995d73532bbf67c47b6df37403844c8c551b360fb482757a448 Verosint's back-end systems will undergo maintenance on **Wednesday, April 30th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) * Session management data back-end upgrade (minor service disruption) Potential scope of impact includes the `SignalPrint` and `Rules` APIs, as well as some functionality in the application front end. Most of this work will be highly-available, but the session management database operation will result in a brief period (estimated 45-90 seconds) where API token validation requests may time out. If that short period will be highly disruptive for your users and doing so is practical, please make any necessary preparations to prevent such disruptions, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/540027 Thu, 01 May 2025 04:00:39 +0000 https://status.verosint.com/incident/540027#fa17d1e0c3a43c555cff9dd6eddaf7068e8b8eb15fd0903081a3f86ce8d2c672 Maintenance completed https://status.verosint.com/incident/540027 Thu, 01 May 2025 04:00:39 +0000 https://status.verosint.com/incident/540027#fa17d1e0c3a43c555cff9dd6eddaf7068e8b8eb15fd0903081a3f86ce8d2c672 Maintenance completed https://status.verosint.com/incident/540027 Thu, 01 May 2025 04:00:39 +0000 https://status.verosint.com/incident/540027#fa17d1e0c3a43c555cff9dd6eddaf7068e8b8eb15fd0903081a3f86ce8d2c672 Maintenance completed https://status.verosint.com/incident/540027 Thu, 01 May 2025 01:00:39 -0000 https://status.verosint.com/incident/540027#077df9d4ce0e0995d73532bbf67c47b6df37403844c8c551b360fb482757a448 Verosint's back-end systems will undergo maintenance on **Wednesday, April 30th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) * Session management data back-end upgrade (minor service disruption) Potential scope of impact includes the `SignalPrint` and `Rules` APIs, as well as some functionality in the application front end. Most of this work will be highly-available, but the session management database operation will result in a brief period (estimated 45-90 seconds) where API token validation requests may time out. If that short period will be highly disruptive for your users and doing so is practical, please make any necessary preparations to prevent such disruptions, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/540027 Thu, 01 May 2025 01:00:39 -0000 https://status.verosint.com/incident/540027#077df9d4ce0e0995d73532bbf67c47b6df37403844c8c551b360fb482757a448 Verosint's back-end systems will undergo maintenance on **Wednesday, April 30th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) * Session management data back-end upgrade (minor service disruption) Potential scope of impact includes the `SignalPrint` and `Rules` APIs, as well as some functionality in the application front end. Most of this work will be highly-available, but the session management database operation will result in a brief period (estimated 45-90 seconds) where API token validation requests may time out. If that short period will be highly disruptive for your users and doing so is practical, please make any necessary preparations to prevent such disruptions, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/540027 Thu, 01 May 2025 01:00:39 -0000 https://status.verosint.com/incident/540027#077df9d4ce0e0995d73532bbf67c47b6df37403844c8c551b360fb482757a448 Verosint's back-end systems will undergo maintenance on **Wednesday, April 30th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) * Session management data back-end upgrade (minor service disruption) Potential scope of impact includes the `SignalPrint` and `Rules` APIs, as well as some functionality in the application front end. Most of this work will be highly-available, but the session management database operation will result in a brief period (estimated 45-90 seconds) where API token validation requests may time out. If that short period will be highly disruptive for your users and doing so is practical, please make any necessary preparations to prevent such disruptions, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/545250 Mon, 14 Apr 2025 22:50:00 -0000 https://status.verosint.com/incident/545250#20c7e942ed043935def5e29c863435f4fa42465fc76886db88215ea64c1ba1e2 Our authentication service provider has notified us that they identified and corrected the issue. We have verified that the Verosint front end is again operating normally. Thanks for your patience. https://status.verosint.com/incident/545250 Mon, 14 Apr 2025 22:27:00 -0000 https://status.verosint.com/incident/545250#220bb353cf63b7ed93fba7673f60e8d0e0311ec8f1d0881467dbdc4949a26070 We have received notification of an incident in progress with our authentication provider. At this time, the issue is only affecting Verosint's front-end application. Engineering staff are actively monitoring the situation and will update if the scope of impact expands. https://status.verosint.com/incident/545250 Mon, 14 Apr 2025 22:03:00 -0000 https://status.verosint.com/incident/545250#5e5b6e0e3a37711a46abd0c79cc4e34b42969358f3d80601b44b81f59713e469 We are investigating an intermittent availability issue with our web app at https://app.verosint.com that started at 16:55 CDT. Some users attempting to log into their tenant may experience errors, timeouts, or significant page load times but due to the intermittent nature of the issue, others may not be affected at all. We will update with any material findings when we have more information. https://status.verosint.com/ Wed, 26 Mar 2025 12:07:09 +0000 https://status.verosint.com/#67ff95db9386f1c54c809d6f552aed3b5a55c584bb94140b7805e3bfc7a95dc1 docs.verosint.com recovered https://status.verosint.com/ Wed, 26 Mar 2025 12:01:09 +0000 https://status.verosint.com/#67ff95db9386f1c54c809d6f552aed3b5a55c584bb94140b7805e3bfc7a95dc1 docs.verosint.com went down https://status.verosint.com/incident/504639 Thu, 13 Feb 2025 05:24:13 -0000 https://status.verosint.com/incident/504639#cf7462067a56845a90cddb91fa7ba0632dc9bdb8ec60e8cf428c710c02286d01 Verosint's back-end systems will undergo maintenance on **Wednesday, February 12th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) * Session management data back-end upgrade (minor service disruption) Potential scope of impact includes the `SignalPrint` and `Rules` APIs, as well as some functionality in the application front end. Most of this work will be highly-available, but the session management database operation will result in a brief period (estimated 45-90 seconds) where API token validation requests may time out. If that short period will be highly disruptive for your users and doing so is practical, please make any necessary preparations to prevent such disruptions, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/504639 Thu, 13 Feb 2025 05:24:13 -0000 https://status.verosint.com/incident/504639#cf7462067a56845a90cddb91fa7ba0632dc9bdb8ec60e8cf428c710c02286d01 Verosint's back-end systems will undergo maintenance on **Wednesday, February 12th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) * Session management data back-end upgrade (minor service disruption) Potential scope of impact includes the `SignalPrint` and `Rules` APIs, as well as some functionality in the application front end. Most of this work will be highly-available, but the session management database operation will result in a brief period (estimated 45-90 seconds) where API token validation requests may time out. If that short period will be highly disruptive for your users and doing so is practical, please make any necessary preparations to prevent such disruptions, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/504639 Thu, 13 Feb 2025 05:24:13 -0000 https://status.verosint.com/incident/504639#cf7462067a56845a90cddb91fa7ba0632dc9bdb8ec60e8cf428c710c02286d01 Verosint's back-end systems will undergo maintenance on **Wednesday, February 12th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) * Session management data back-end upgrade (minor service disruption) Potential scope of impact includes the `SignalPrint` and `Rules` APIs, as well as some functionality in the application front end. Most of this work will be highly-available, but the session management database operation will result in a brief period (estimated 45-90 seconds) where API token validation requests may time out. If that short period will be highly disruptive for your users and doing so is practical, please make any necessary preparations to prevent such disruptions, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/504639 Thu, 13 Feb 2025 05:00:01 +0000 https://status.verosint.com/incident/504639#719f4b6e9f72d56481470034580e5e39f2575eecfdc9157c24b3adcc277b265f Maintenance completed https://status.verosint.com/incident/504639 Thu, 13 Feb 2025 05:00:01 +0000 https://status.verosint.com/incident/504639#719f4b6e9f72d56481470034580e5e39f2575eecfdc9157c24b3adcc277b265f Maintenance completed https://status.verosint.com/incident/504639 Thu, 13 Feb 2025 05:00:01 +0000 https://status.verosint.com/incident/504639#719f4b6e9f72d56481470034580e5e39f2575eecfdc9157c24b3adcc277b265f Maintenance completed https://status.verosint.com/incident/504639 Thu, 13 Feb 2025 02:00:01 -0000 https://status.verosint.com/incident/504639#cf7462067a56845a90cddb91fa7ba0632dc9bdb8ec60e8cf428c710c02286d01 Verosint's back-end systems will undergo maintenance on **Wednesday, February 12th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) * Session management data back-end upgrade (minor service disruption) Potential scope of impact includes the `SignalPrint` and `Rules` APIs, as well as some functionality in the application front end. Most of this work will be highly-available, but the session management database operation will result in a brief period (estimated 45-90 seconds) where API token validation requests may time out. If that short period will be highly disruptive for your users and doing so is practical, please make any necessary preparations to prevent such disruptions, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/504639 Thu, 13 Feb 2025 02:00:01 -0000 https://status.verosint.com/incident/504639#cf7462067a56845a90cddb91fa7ba0632dc9bdb8ec60e8cf428c710c02286d01 Verosint's back-end systems will undergo maintenance on **Wednesday, February 12th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) * Session management data back-end upgrade (minor service disruption) Potential scope of impact includes the `SignalPrint` and `Rules` APIs, as well as some functionality in the application front end. Most of this work will be highly-available, but the session management database operation will result in a brief period (estimated 45-90 seconds) where API token validation requests may time out. If that short period will be highly disruptive for your users and doing so is practical, please make any necessary preparations to prevent such disruptions, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/504639 Thu, 13 Feb 2025 02:00:01 -0000 https://status.verosint.com/incident/504639#cf7462067a56845a90cddb91fa7ba0632dc9bdb8ec60e8cf428c710c02286d01 Verosint's back-end systems will undergo maintenance on **Wednesday, February 12th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) * Session management data back-end upgrade (minor service disruption) Potential scope of impact includes the `SignalPrint` and `Rules` APIs, as well as some functionality in the application front end. Most of this work will be highly-available, but the session management database operation will result in a brief period (estimated 45-90 seconds) where API token validation requests may time out. If that short period will be highly disruptive for your users and doing so is practical, please make any necessary preparations to prevent such disruptions, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/475453 Thu, 12 Dec 2024 02:51:24 +0000 https://status.verosint.com/incident/475453#80eb71ef87fdaf8499e2cf581fa90d2e21074afc10c7fbb07904870628282dbd Maintenance completed https://status.verosint.com/incident/475453 Thu, 12 Dec 2024 02:51:24 -0000 https://status.verosint.com/incident/475453#adc8342e1d5e28b0cc513d2c17feb688ed31ed480d92ff17e1697b99f3fd6d94 Verosint's back-end systems will undergo maintenance on **Wednesday, December 11th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) Potential scope of impact includes the `SignalPrint` and `Rules` APIs. This process maintains high-availability and we do not anticipate any service disruptions but still recommend, if practical, making any necessary preparations to prevent disruptions of your service for your users, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/475453 Thu, 12 Dec 2024 02:51:24 +0000 https://status.verosint.com/incident/475453#80eb71ef87fdaf8499e2cf581fa90d2e21074afc10c7fbb07904870628282dbd Maintenance completed https://status.verosint.com/incident/475453 Thu, 12 Dec 2024 02:51:24 -0000 https://status.verosint.com/incident/475453#adc8342e1d5e28b0cc513d2c17feb688ed31ed480d92ff17e1697b99f3fd6d94 Verosint's back-end systems will undergo maintenance on **Wednesday, December 11th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) Potential scope of impact includes the `SignalPrint` and `Rules` APIs. This process maintains high-availability and we do not anticipate any service disruptions but still recommend, if practical, making any necessary preparations to prevent disruptions of your service for your users, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/475453 Thu, 12 Dec 2024 02:00:42 -0000 https://status.verosint.com/incident/475453#adc8342e1d5e28b0cc513d2c17feb688ed31ed480d92ff17e1697b99f3fd6d94 Verosint's back-end systems will undergo maintenance on **Wednesday, December 11th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) Potential scope of impact includes the `SignalPrint` and `Rules` APIs. This process maintains high-availability and we do not anticipate any service disruptions but still recommend, if practical, making any necessary preparations to prevent disruptions of your service for your users, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/475453 Thu, 12 Dec 2024 02:00:42 -0000 https://status.verosint.com/incident/475453#adc8342e1d5e28b0cc513d2c17feb688ed31ed480d92ff17e1697b99f3fd6d94 Verosint's back-end systems will undergo maintenance on **Wednesday, December 11th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Signal processing back-end: rolling Redis cluster updates (high-availability) Potential scope of impact includes the `SignalPrint` and `Rules` APIs. This process maintains high-availability and we do not anticipate any service disruptions but still recommend, if practical, making any necessary preparations to prevent disruptions of your service for your users, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance event. https://status.verosint.com/incident/471228 Thu, 05 Dec 2024 06:00:00 +0000 https://status.verosint.com/incident/471228#8a1a7a45cfaca36008875b25425bd9211d5055dc13ddd25879a6961a10de73eb Maintenance completed https://status.verosint.com/incident/471228 Thu, 05 Dec 2024 06:00:00 +0000 https://status.verosint.com/incident/471228#8a1a7a45cfaca36008875b25425bd9211d5055dc13ddd25879a6961a10de73eb Maintenance completed https://status.verosint.com/incident/471228 Thu, 05 Dec 2024 03:00:00 -0000 https://status.verosint.com/incident/471228#c74b16cabd54d7ca484b5caa3c5fb734feaebed5c33b7c398c17e958b17ab16b Verosint will be performing routine maintenance on Wednesday, December 4th starting at 9:00 pm CST until 12:00 am CST to deploy system updates to application back-end infrastructure. No service downtime is expected during these operations. Primary objectives: - Kubernetes cluster updates - Database network performance optimization While we do not anticipate any service availability issues, please contact Verosint support (support@verosint.com) if any are encountered. https://status.verosint.com/incident/471228 Thu, 05 Dec 2024 03:00:00 -0000 https://status.verosint.com/incident/471228#c74b16cabd54d7ca484b5caa3c5fb734feaebed5c33b7c398c17e958b17ab16b Verosint will be performing routine maintenance on Wednesday, December 4th starting at 9:00 pm CST until 12:00 am CST to deploy system updates to application back-end infrastructure. No service downtime is expected during these operations. Primary objectives: - Kubernetes cluster updates - Database network performance optimization While we do not anticipate any service availability issues, please contact Verosint support (support@verosint.com) if any are encountered. https://status.verosint.com/incident/460220 Thu, 14 Nov 2024 02:56:45 +0000 https://status.verosint.com/incident/460220#9e7f0241800aae123e3b93451c78dbc5ddcf2645a0f92e92ee171ea312ad41f3 Maintenance completed https://status.verosint.com/incident/460220 Thu, 14 Nov 2024 02:56:45 -0000 https://status.verosint.com/incident/460220#688a445525c110fbfeea433a9e1b15d772e899b762cf2718d1478c55596f9648 A subset of Verosint's database systems will undergo maintenance on Wednesday, November 13th starting at 8:00 PM CDT until 11:00 PM CDT. Access to front-end tenants (app.verosint.com) will be temporarily disabled for the duration of operations, but all back-end API endpoints will remain available. Please contact us at `support@verosint.com` with any questions about this maintenance event. https://status.verosint.com/incident/460220 Thu, 14 Nov 2024 02:00:37 -0000 https://status.verosint.com/incident/460220#688a445525c110fbfeea433a9e1b15d772e899b762cf2718d1478c55596f9648 A subset of Verosint's database systems will undergo maintenance on Wednesday, November 13th starting at 8:00 PM CDT until 11:00 PM CDT. Access to front-end tenants (app.verosint.com) will be temporarily disabled for the duration of operations, but all back-end API endpoints will remain available. Please contact us at `support@verosint.com` with any questions about this maintenance event. https://status.verosint.com/incident/451960 Thu, 31 Oct 2024 03:26:49 +0000 https://status.verosint.com/incident/451960#bdae9308e9a225572620a5771946c9e8d2a6ec9992099534c217a778bb6e1d3a Maintenance completed https://status.verosint.com/incident/451960 Thu, 31 Oct 2024 03:26:49 -0000 https://status.verosint.com/incident/451960#2d8d82e97209723a3e8b5399121dcd8b8875665e6cad81a40efe4ea0b99421f9 Verosint's back-end systems will undergo maintenance on Wednesday, October 30th starting at 8:00 PM CDT until 11:00 PM CDT. This maintenance **will be disruptive** and all API endpoints will be unavailable for the duration. Front-end functionality will also be significantly affected and access to it will be temporarily disabled while the back-end is unavailable. We strongly recommend making any necessary preparations to prevent disruptions of your service for your users, such as configuring your app to fail open, until the maintenance is complete. As always, please reach out to our support team (support@verosint.com) with any questions about this event. https://status.verosint.com/incident/451960 Thu, 31 Oct 2024 03:26:49 +0000 https://status.verosint.com/incident/451960#bdae9308e9a225572620a5771946c9e8d2a6ec9992099534c217a778bb6e1d3a Maintenance completed https://status.verosint.com/incident/451960 Thu, 31 Oct 2024 03:26:49 -0000 https://status.verosint.com/incident/451960#2d8d82e97209723a3e8b5399121dcd8b8875665e6cad81a40efe4ea0b99421f9 Verosint's back-end systems will undergo maintenance on Wednesday, October 30th starting at 8:00 PM CDT until 11:00 PM CDT. This maintenance **will be disruptive** and all API endpoints will be unavailable for the duration. Front-end functionality will also be significantly affected and access to it will be temporarily disabled while the back-end is unavailable. We strongly recommend making any necessary preparations to prevent disruptions of your service for your users, such as configuring your app to fail open, until the maintenance is complete. As always, please reach out to our support team (support@verosint.com) with any questions about this event. https://status.verosint.com/incident/451960 Thu, 31 Oct 2024 03:26:49 +0000 https://status.verosint.com/incident/451960#bdae9308e9a225572620a5771946c9e8d2a6ec9992099534c217a778bb6e1d3a Maintenance completed https://status.verosint.com/incident/451960 Thu, 31 Oct 2024 03:26:49 -0000 https://status.verosint.com/incident/451960#2d8d82e97209723a3e8b5399121dcd8b8875665e6cad81a40efe4ea0b99421f9 Verosint's back-end systems will undergo maintenance on Wednesday, October 30th starting at 8:00 PM CDT until 11:00 PM CDT. This maintenance **will be disruptive** and all API endpoints will be unavailable for the duration. Front-end functionality will also be significantly affected and access to it will be temporarily disabled while the back-end is unavailable. We strongly recommend making any necessary preparations to prevent disruptions of your service for your users, such as configuring your app to fail open, until the maintenance is complete. As always, please reach out to our support team (support@verosint.com) with any questions about this event. https://status.verosint.com/incident/451960 Thu, 31 Oct 2024 01:00:24 -0000 https://status.verosint.com/incident/451960#2d8d82e97209723a3e8b5399121dcd8b8875665e6cad81a40efe4ea0b99421f9 Verosint's back-end systems will undergo maintenance on Wednesday, October 30th starting at 8:00 PM CDT until 11:00 PM CDT. This maintenance **will be disruptive** and all API endpoints will be unavailable for the duration. Front-end functionality will also be significantly affected and access to it will be temporarily disabled while the back-end is unavailable. We strongly recommend making any necessary preparations to prevent disruptions of your service for your users, such as configuring your app to fail open, until the maintenance is complete. As always, please reach out to our support team (support@verosint.com) with any questions about this event. https://status.verosint.com/incident/451960 Thu, 31 Oct 2024 01:00:24 -0000 https://status.verosint.com/incident/451960#2d8d82e97209723a3e8b5399121dcd8b8875665e6cad81a40efe4ea0b99421f9 Verosint's back-end systems will undergo maintenance on Wednesday, October 30th starting at 8:00 PM CDT until 11:00 PM CDT. This maintenance **will be disruptive** and all API endpoints will be unavailable for the duration. Front-end functionality will also be significantly affected and access to it will be temporarily disabled while the back-end is unavailable. We strongly recommend making any necessary preparations to prevent disruptions of your service for your users, such as configuring your app to fail open, until the maintenance is complete. As always, please reach out to our support team (support@verosint.com) with any questions about this event. https://status.verosint.com/incident/451960 Thu, 31 Oct 2024 01:00:24 -0000 https://status.verosint.com/incident/451960#2d8d82e97209723a3e8b5399121dcd8b8875665e6cad81a40efe4ea0b99421f9 Verosint's back-end systems will undergo maintenance on Wednesday, October 30th starting at 8:00 PM CDT until 11:00 PM CDT. This maintenance **will be disruptive** and all API endpoints will be unavailable for the duration. Front-end functionality will also be significantly affected and access to it will be temporarily disabled while the back-end is unavailable. We strongly recommend making any necessary preparations to prevent disruptions of your service for your users, such as configuring your app to fail open, until the maintenance is complete. As always, please reach out to our support team (support@verosint.com) with any questions about this event. https://status.verosint.com/incident/444439 Thu, 17 Oct 2024 04:00:26 +0000 https://status.verosint.com/incident/444439#27e5d87a7c67f678efe7723aa2b4ec006fb94dd6511406f783d99f3c0428a66e Maintenance completed https://status.verosint.com/incident/444439 Thu, 17 Oct 2024 01:00:26 -0000 https://status.verosint.com/incident/444439#29cd7f141ff63adc5c6236378a27b858350823c12ad07a197205e4ec2e28837e Verosint's back-end systems will undergo maintenance on **Wednesday, October 16th starting at 8:00 PM CDT until 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Back-end Redis cluster system updates This process maintains high-availability and we do not anticipate any service disruptions but still recommend making any necessary preparations to prevent disruptions of your service for your users, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance. https://status.verosint.com/incident/440751 Thu, 10 Oct 2024 04:00:40 +0000 https://status.verosint.com/incident/440751#66181e40bbf96f8b36dfeee2b324acbabd7cd13a20d6fa908be5454aaaa1c1fa Maintenance completed https://status.verosint.com/incident/440751 Thu, 10 Oct 2024 04:00:40 +0000 https://status.verosint.com/incident/440751#66181e40bbf96f8b36dfeee2b324acbabd7cd13a20d6fa908be5454aaaa1c1fa Maintenance completed https://status.verosint.com/incident/440751 Thu, 10 Oct 2024 01:00:40 -0000 https://status.verosint.com/incident/440751#8d090276552235f10027ea8702c888b25c021fc2034440c931c3fdaca84e3953 Verosint's back-end systems will undergo maintenance on Wednesday, October 9th starting at 8:00 PM CDT until 11:00 PM CDT. Maintenance goals include, but are not limited to: * SignalPrint event cache data back-end migration We do not anticipate any service disruptions during this maintenance but still recommend making any necessary preparations to prevent disruptions of your service for your users, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance. https://status.verosint.com/incident/440751 Thu, 10 Oct 2024 01:00:40 -0000 https://status.verosint.com/incident/440751#8d090276552235f10027ea8702c888b25c021fc2034440c931c3fdaca84e3953 Verosint's back-end systems will undergo maintenance on Wednesday, October 9th starting at 8:00 PM CDT until 11:00 PM CDT. Maintenance goals include, but are not limited to: * SignalPrint event cache data back-end migration We do not anticipate any service disruptions during this maintenance but still recommend making any necessary preparations to prevent disruptions of your service for your users, such as configuring your app to fail open. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance. https://status.verosint.com/incident/437777 Thu, 03 Oct 2024 04:00:32 +0000 https://status.verosint.com/incident/437777#076f1d6358a2fef0af3ee1c1feea50508ad3fb3621d8cb0452880cf3a6eb6ff9 Maintenance completed https://status.verosint.com/incident/437777 Thu, 03 Oct 2024 04:00:32 +0000 https://status.verosint.com/incident/437777#076f1d6358a2fef0af3ee1c1feea50508ad3fb3621d8cb0452880cf3a6eb6ff9 Maintenance completed https://status.verosint.com/incident/437777 Thu, 03 Oct 2024 04:00:32 +0000 https://status.verosint.com/incident/437777#076f1d6358a2fef0af3ee1c1feea50508ad3fb3621d8cb0452880cf3a6eb6ff9 Maintenance completed https://status.verosint.com/incident/437777 Thu, 03 Oct 2024 01:00:32 -0000 https://status.verosint.com/incident/437777#6a7e50653b61799932f50e3e6b3277ef1bfbb01e4216f00d6d7f318bf2c698a5 ## Verosint Scheduled Maintenance - October 02, 2024 Verosint's back-end systems will undergo minor maintenance on **Wednesday, October 2nd starting at 8:00 PM through 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Back-end system updates * Ingress load balancer updates We do not anticipate any service disruptions during this maintenance but still recommend making any necessary preparations to prevent disruptions of your service for your users. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance. https://status.verosint.com/incident/437777 Thu, 03 Oct 2024 01:00:32 -0000 https://status.verosint.com/incident/437777#6a7e50653b61799932f50e3e6b3277ef1bfbb01e4216f00d6d7f318bf2c698a5 ## Verosint Scheduled Maintenance - October 02, 2024 Verosint's back-end systems will undergo minor maintenance on **Wednesday, October 2nd starting at 8:00 PM through 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Back-end system updates * Ingress load balancer updates We do not anticipate any service disruptions during this maintenance but still recommend making any necessary preparations to prevent disruptions of your service for your users. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance. https://status.verosint.com/incident/437777 Thu, 03 Oct 2024 01:00:32 -0000 https://status.verosint.com/incident/437777#6a7e50653b61799932f50e3e6b3277ef1bfbb01e4216f00d6d7f318bf2c698a5 ## Verosint Scheduled Maintenance - October 02, 2024 Verosint's back-end systems will undergo minor maintenance on **Wednesday, October 2nd starting at 8:00 PM through 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Back-end system updates * Ingress load balancer updates We do not anticipate any service disruptions during this maintenance but still recommend making any necessary preparations to prevent disruptions of your service for your users. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance. https://status.verosint.com/incident/428225 Thu, 19 Sep 2024 04:00:14 +0000 https://status.verosint.com/incident/428225#9193570ecadc4091e6ff5f3b623181aae314c6741a58ac86d169e13245d2c460 Maintenance completed https://status.verosint.com/incident/428225 Thu, 19 Sep 2024 04:00:14 +0000 https://status.verosint.com/incident/428225#9193570ecadc4091e6ff5f3b623181aae314c6741a58ac86d169e13245d2c460 Maintenance completed https://status.verosint.com/incident/428225 Thu, 19 Sep 2024 04:00:14 +0000 https://status.verosint.com/incident/428225#9193570ecadc4091e6ff5f3b623181aae314c6741a58ac86d169e13245d2c460 Maintenance completed https://status.verosint.com/incident/428225 Thu, 19 Sep 2024 01:00:14 -0000 https://status.verosint.com/incident/428225#1a81f48fba4fe3dd7bbb3204005c39545697cb402d23f599b6baa60526f211dc ## Verosint Scheduled Maintenance - September 18, 2024 Verosint's back-end systems will undergo maintenance on **Wednesday, September 18th starting at 8:00 PM through 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Database version updates * Load balancer performance optimization * Data backend migration prep SignalPrint and Rules API endpoints will experience brief, intermittent service disruptions while certain maintenance operations are underway. During those disruptions, the associated functionality in front-end tenants will be affected. While we do not anticipate any prolonged service disruptions, the potential exists. We strongly suggest making appropriate preparations in applications utilizing the Rules API endpoints to minimize the impact to your users. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance. https://status.verosint.com/incident/428225 Thu, 19 Sep 2024 01:00:14 -0000 https://status.verosint.com/incident/428225#1a81f48fba4fe3dd7bbb3204005c39545697cb402d23f599b6baa60526f211dc ## Verosint Scheduled Maintenance - September 18, 2024 Verosint's back-end systems will undergo maintenance on **Wednesday, September 18th starting at 8:00 PM through 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Database version updates * Load balancer performance optimization * Data backend migration prep SignalPrint and Rules API endpoints will experience brief, intermittent service disruptions while certain maintenance operations are underway. During those disruptions, the associated functionality in front-end tenants will be affected. While we do not anticipate any prolonged service disruptions, the potential exists. We strongly suggest making appropriate preparations in applications utilizing the Rules API endpoints to minimize the impact to your users. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance. https://status.verosint.com/incident/428225 Thu, 19 Sep 2024 01:00:14 -0000 https://status.verosint.com/incident/428225#1a81f48fba4fe3dd7bbb3204005c39545697cb402d23f599b6baa60526f211dc ## Verosint Scheduled Maintenance - September 18, 2024 Verosint's back-end systems will undergo maintenance on **Wednesday, September 18th starting at 8:00 PM through 11:00 PM CDT**. Maintenance goals include, but are not limited to: * Database version updates * Load balancer performance optimization * Data backend migration prep SignalPrint and Rules API endpoints will experience brief, intermittent service disruptions while certain maintenance operations are underway. During those disruptions, the associated functionality in front-end tenants will be affected. While we do not anticipate any prolonged service disruptions, the potential exists. We strongly suggest making appropriate preparations in applications utilizing the Rules API endpoints to minimize the impact to your users. As always, please reach out to our support team (support@verosint.com) with any questions about this maintenance. https://status.verosint.com/incident/424119 Wed, 04 Sep 2024 14:23:00 -0000 https://status.verosint.com/incident/424119#d84f16e4858fd421d44eff887c2dcfd42e44aa01cca00f58501650e8d40d6599 Root cause of the monitor failure has been traced to a short-lived server-side error when calling our authentication provider. Further attempts to reproduce the issue continue to fail and the monitoring system has not flagged a subsequent failure. We consider this issue resolved, but please let support@verosint.com know if you experience authentication issues when logging into your tenant https://status.verosint.com/incident/424119 Wed, 04 Sep 2024 13:29:00 -0000 https://status.verosint.com/incident/424119#a2576ecd2f0bc6e92bf319d3d63dfc2b51116c8a724d3a8339ee39b9e7c6d483 We are receiving intermittent alerts from our front-end monitors indicating a potential issue with page rendering. Response teams have not been able to reproduce this issue thus far, but please let us know if you are encountering issues at support@verosint.com. https://status.verosint.com/incident/421048 Thu, 29 Aug 2024 02:39:00 -0000 https://status.verosint.com/incident/421048#af4119090f0544872d95adf36079efd96a23a497514ac26443f6705fdc2eacc6 Starting at 9:16 pm CDT, Rules API experienced a degradation of service for roughly 15 minutes following a back-end infrastructure update, during which a significant portion of rule evaluation requests timed out or responded with an error. The faulting service was recovered at 9:31 pm CDT and the platform is functioning normally, but root cause analysis is still ongoing. Any material discoveries will be posted as follow-up to this status. https://status.verosint.com/incident/405666 Mon, 29 Jul 2024 16:11:00 -0000 https://status.verosint.com/incident/405666#20383b934990e50184571b31f35a2ff63d126f4bc23895f4bbb00b87346c0380 A fix has been deployed and the affected services have been fully restored. The issue with the library that caused this issue is undergoing further investigation, as is the procedural gap in testing automation that allowed it to reach production. Our apologies for the disruption. https://status.verosint.com/incident/405666 Mon, 29 Jul 2024 16:11:00 -0000 https://status.verosint.com/incident/405666#20383b934990e50184571b31f35a2ff63d126f4bc23895f4bbb00b87346c0380 A fix has been deployed and the affected services have been fully restored. The issue with the library that caused this issue is undergoing further investigation, as is the procedural gap in testing automation that allowed it to reach production. Our apologies for the disruption. https://status.verosint.com/incident/405666 Mon, 29 Jul 2024 16:11:00 -0000 https://status.verosint.com/incident/405666#20383b934990e50184571b31f35a2ff63d126f4bc23895f4bbb00b87346c0380 A fix has been deployed and the affected services have been fully restored. The issue with the library that caused this issue is undergoing further investigation, as is the procedural gap in testing automation that allowed it to reach production. Our apologies for the disruption. https://status.verosint.com/incident/405666 Mon, 29 Jul 2024 15:57:00 -0000 https://status.verosint.com/incident/405666#d902d56ae0cc2bc64295dfcac7ce8f231045ca1c3a0798203f4a1cf67c228216 An issue with an underlying library introduced a fault in the back end of the application stack for both the SignalPrint API and the Rules API. The issue has been identified and a fix will be deployed momentarily. https://status.verosint.com/incident/405666 Mon, 29 Jul 2024 15:57:00 -0000 https://status.verosint.com/incident/405666#d902d56ae0cc2bc64295dfcac7ce8f231045ca1c3a0798203f4a1cf67c228216 An issue with an underlying library introduced a fault in the back end of the application stack for both the SignalPrint API and the Rules API. The issue has been identified and a fix will be deployed momentarily. https://status.verosint.com/incident/405666 Mon, 29 Jul 2024 15:57:00 -0000 https://status.verosint.com/incident/405666#d902d56ae0cc2bc64295dfcac7ce8f231045ca1c3a0798203f4a1cf67c228216 An issue with an underlying library introduced a fault in the back end of the application stack for both the SignalPrint API and the Rules API. The issue has been identified and a fix will be deployed momentarily. https://status.verosint.com/incident/399677 Wed, 17 Jul 2024 21:12:00 -0000 https://status.verosint.com/incident/399677#f3784462e4b611aaed0814c7273dacbd057a7b6e01be48ad62aa608bd062b513 We have confirmed the deployed fix has addressed the root cause and error rates have subsided. Thanks for your patience, and please contact us at support@verosint.com with any questions regarding this incident. https://status.verosint.com/incident/399677 Wed, 17 Jul 2024 21:03:00 -0000 https://status.verosint.com/incident/399677#beabb3645b64296f1893d414387cf5441d4da941497e80a47aba8ed38690d7d3 Root cause of the issue has been identified and a deployment to fix it is underway. We will update again when the rollout is complete and the issue is resolved. https://status.verosint.com/incident/399677 Wed, 17 Jul 2024 20:45:00 -0000 https://status.verosint.com/incident/399677#e145bc8933b27b8219f9452c7f0d9753fdbd3a0767e0d05ae5e07ce1b88e5119 Platform monitoring indicates a higher-than-usual error rate from the Rules API. Engineering teams are investigating and will update with any material developments. https://status.verosint.com/incident/390869 Fri, 28 Jun 2024 19:39:00 -0000 https://status.verosint.com/incident/390869#e09cb2693e34ab4207d539c6f3b7b58721c0222ec0cc28a971e743f853a8e3c6 The Signalprint and Rules APIs experienced a momentary downtime due to an order-of-operations race condition in back-end network management automation. The root cause was identified and remediated within moments of occurrence, and internal monitoring indicates full recovery. As a result of this incident, additional safeguards will be placed around the automated process to eliminate risk of recurrence. https://status.verosint.com/incident/390869 Fri, 28 Jun 2024 19:39:00 -0000 https://status.verosint.com/incident/390869#e09cb2693e34ab4207d539c6f3b7b58721c0222ec0cc28a971e743f853a8e3c6 The Signalprint and Rules APIs experienced a momentary downtime due to an order-of-operations race condition in back-end network management automation. The root cause was identified and remediated within moments of occurrence, and internal monitoring indicates full recovery. As a result of this incident, additional safeguards will be placed around the automated process to eliminate risk of recurrence. https://status.verosint.com/incident/390869 Fri, 28 Jun 2024 19:39:00 -0000 https://status.verosint.com/incident/390869#e09cb2693e34ab4207d539c6f3b7b58721c0222ec0cc28a971e743f853a8e3c6 The Signalprint and Rules APIs experienced a momentary downtime due to an order-of-operations race condition in back-end network management automation. The root cause was identified and remediated within moments of occurrence, and internal monitoring indicates full recovery. As a result of this incident, additional safeguards will be placed around the automated process to eliminate risk of recurrence. https://status.verosint.com/incident/380512 Tue, 11 Jun 2024 03:10:53 -0000 https://status.verosint.com/incident/380512#bf20052a60a88e059a97cddacbaf5910a28ee8d8259a5f298ad4150838401a76 We will be upgrading the back-end data store which handles initial processing and analysis of new SignalPrint events. While this is a highly-available upgrade procedure and we anticipate little to no service disruption during the upgrade operation, event processing may be slightly delayed for a brief period during the transition. Please reach us at support@verosint.com with any questions regarding this maintenance. https://status.verosint.com/incident/380512 Tue, 11 Jun 2024 03:00:00 +0000 https://status.verosint.com/incident/380512#7a0157c6f5da113b35beecf73b168e67dbd40b93c9dbec8f51e2d63964b16842 Maintenance completed https://status.verosint.com/incident/380512 Tue, 11 Jun 2024 02:00:00 -0000 https://status.verosint.com/incident/380512#bf20052a60a88e059a97cddacbaf5910a28ee8d8259a5f298ad4150838401a76 We will be upgrading the back-end data store which handles initial processing and analysis of new SignalPrint events. While this is a highly-available upgrade procedure and we anticipate little to no service disruption during the upgrade operation, event processing may be slightly delayed for a brief period during the transition. Please reach us at support@verosint.com with any questions regarding this maintenance. https://status.verosint.com/incident/370292 Thu, 23 May 2024 03:25:01 +0000 https://status.verosint.com/incident/370292#f4dacb79432f42754446fe2f63dd7985fa780c90d6af28752b30868a6ea915fc Maintenance completed https://status.verosint.com/incident/370292 Thu, 23 May 2024 03:25:01 -0000 https://status.verosint.com/incident/370292#ec39e16784ef0e57cab0922a5006c3619ee4a7c1626923aaeceb11e8c10aaa9f Starting at 9:30 pm CDT, the back-end databases supporting the Signalprint and Rules APIs will be taken offline for a major version update of the database engine. Both of the aforementioned will be unavailable for the duration, and the web application features which depend on them will also be affected. As a standard practice, we avoid disruptive changes like this unless there's no alternative. The downtime is not expected to be longer than 30 minutes, but we recommend planning for additional time as a precaution. Please reach out to support@verosint.com for additional information. https://status.verosint.com/incident/370292 Thu, 23 May 2024 03:25:01 +0000 https://status.verosint.com/incident/370292#f4dacb79432f42754446fe2f63dd7985fa780c90d6af28752b30868a6ea915fc Maintenance completed https://status.verosint.com/incident/370292 Thu, 23 May 2024 03:25:01 -0000 https://status.verosint.com/incident/370292#ec39e16784ef0e57cab0922a5006c3619ee4a7c1626923aaeceb11e8c10aaa9f Starting at 9:30 pm CDT, the back-end databases supporting the Signalprint and Rules APIs will be taken offline for a major version update of the database engine. Both of the aforementioned will be unavailable for the duration, and the web application features which depend on them will also be affected. As a standard practice, we avoid disruptive changes like this unless there's no alternative. The downtime is not expected to be longer than 30 minutes, but we recommend planning for additional time as a precaution. Please reach out to support@verosint.com for additional information. https://status.verosint.com/incident/370292 Thu, 23 May 2024 03:25:01 +0000 https://status.verosint.com/incident/370292#f4dacb79432f42754446fe2f63dd7985fa780c90d6af28752b30868a6ea915fc Maintenance completed https://status.verosint.com/incident/370292 Thu, 23 May 2024 03:25:01 -0000 https://status.verosint.com/incident/370292#ec39e16784ef0e57cab0922a5006c3619ee4a7c1626923aaeceb11e8c10aaa9f Starting at 9:30 pm CDT, the back-end databases supporting the Signalprint and Rules APIs will be taken offline for a major version update of the database engine. Both of the aforementioned will be unavailable for the duration, and the web application features which depend on them will also be affected. As a standard practice, we avoid disruptive changes like this unless there's no alternative. The downtime is not expected to be longer than 30 minutes, but we recommend planning for additional time as a precaution. Please reach out to support@verosint.com for additional information. https://status.verosint.com/incident/370292 Thu, 23 May 2024 02:34:15 -0000 https://status.verosint.com/incident/370292#ec39e16784ef0e57cab0922a5006c3619ee4a7c1626923aaeceb11e8c10aaa9f Starting at 9:30 pm CDT, the back-end databases supporting the Signalprint and Rules APIs will be taken offline for a major version update of the database engine. Both of the aforementioned will be unavailable for the duration, and the web application features which depend on them will also be affected. As a standard practice, we avoid disruptive changes like this unless there's no alternative. The downtime is not expected to be longer than 30 minutes, but we recommend planning for additional time as a precaution. Please reach out to support@verosint.com for additional information. https://status.verosint.com/incident/370292 Thu, 23 May 2024 02:34:15 -0000 https://status.verosint.com/incident/370292#ec39e16784ef0e57cab0922a5006c3619ee4a7c1626923aaeceb11e8c10aaa9f Starting at 9:30 pm CDT, the back-end databases supporting the Signalprint and Rules APIs will be taken offline for a major version update of the database engine. Both of the aforementioned will be unavailable for the duration, and the web application features which depend on them will also be affected. As a standard practice, we avoid disruptive changes like this unless there's no alternative. The downtime is not expected to be longer than 30 minutes, but we recommend planning for additional time as a precaution. Please reach out to support@verosint.com for additional information. https://status.verosint.com/incident/370292 Thu, 23 May 2024 02:34:15 -0000 https://status.verosint.com/incident/370292#ec39e16784ef0e57cab0922a5006c3619ee4a7c1626923aaeceb11e8c10aaa9f Starting at 9:30 pm CDT, the back-end databases supporting the Signalprint and Rules APIs will be taken offline for a major version update of the database engine. Both of the aforementioned will be unavailable for the duration, and the web application features which depend on them will also be affected. As a standard practice, we avoid disruptive changes like this unless there's no alternative. The downtime is not expected to be longer than 30 minutes, but we recommend planning for additional time as a precaution. Please reach out to support@verosint.com for additional information. https://status.verosint.com/incident/370292 Thu, 23 May 2024 02:30:00 -0000 https://status.verosint.com/incident/370292#ec39e16784ef0e57cab0922a5006c3619ee4a7c1626923aaeceb11e8c10aaa9f Starting at 9:30 pm CDT, the back-end databases supporting the Signalprint and Rules APIs will be taken offline for a major version update of the database engine. Both of the aforementioned will be unavailable for the duration, and the web application features which depend on them will also be affected. As a standard practice, we avoid disruptive changes like this unless there's no alternative. The downtime is not expected to be longer than 30 minutes, but we recommend planning for additional time as a precaution. Please reach out to support@verosint.com for additional information. https://status.verosint.com/incident/370292 Thu, 23 May 2024 02:30:00 -0000 https://status.verosint.com/incident/370292#ec39e16784ef0e57cab0922a5006c3619ee4a7c1626923aaeceb11e8c10aaa9f Starting at 9:30 pm CDT, the back-end databases supporting the Signalprint and Rules APIs will be taken offline for a major version update of the database engine. Both of the aforementioned will be unavailable for the duration, and the web application features which depend on them will also be affected. As a standard practice, we avoid disruptive changes like this unless there's no alternative. The downtime is not expected to be longer than 30 minutes, but we recommend planning for additional time as a precaution. Please reach out to support@verosint.com for additional information. https://status.verosint.com/incident/370292 Thu, 23 May 2024 02:30:00 -0000 https://status.verosint.com/incident/370292#ec39e16784ef0e57cab0922a5006c3619ee4a7c1626923aaeceb11e8c10aaa9f Starting at 9:30 pm CDT, the back-end databases supporting the Signalprint and Rules APIs will be taken offline for a major version update of the database engine. Both of the aforementioned will be unavailable for the duration, and the web application features which depend on them will also be affected. As a standard practice, we avoid disruptive changes like this unless there's no alternative. The downtime is not expected to be longer than 30 minutes, but we recommend planning for additional time as a precaution. Please reach out to support@verosint.com for additional information. https://status.verosint.com/incident/368950 Tue, 14 May 2024 03:12:00 -0000 https://status.verosint.com/incident/368950#e7314ec33a26c28d6c26b770bdab0cab863fbb2e0898d70e900dfbcbb7b04320 Backend database issue has been corrected and all affected services are against functioning normally. https://status.verosint.com/incident/368950 Tue, 14 May 2024 03:12:00 -0000 https://status.verosint.com/incident/368950#e7314ec33a26c28d6c26b770bdab0cab863fbb2e0898d70e900dfbcbb7b04320 Backend database issue has been corrected and all affected services are against functioning normally. https://status.verosint.com/incident/368950 Tue, 14 May 2024 03:06:00 -0000 https://status.verosint.com/incident/368950#cef893f48525344106cf04d36cc2551bcd4fc0988f3b4ddb3d85b9d28b4c0fce The rules API is currently in a degraded state resulting from an operational issue during routine backend maintenance operations. The root cause is known and the service degradation will be resolved shortly. https://status.verosint.com/incident/368950 Tue, 14 May 2024 03:06:00 -0000 https://status.verosint.com/incident/368950#cef893f48525344106cf04d36cc2551bcd4fc0988f3b4ddb3d85b9d28b4c0fce The rules API is currently in a degraded state resulting from an operational issue during routine backend maintenance operations. The root cause is known and the service degradation will be resolved shortly. https://status.verosint.com/incident/360763 Tue, 30 Apr 2024 04:00:00 +0000 https://status.verosint.com/incident/360763#3ac24d223be910cb101cf304bf03fe1d68b1ff68d1550cb399eb10b3af17b638 Maintenance completed https://status.verosint.com/incident/360763 Tue, 30 Apr 2024 04:00:00 +0000 https://status.verosint.com/incident/360763#3ac24d223be910cb101cf304bf03fe1d68b1ff68d1550cb399eb10b3af17b638 Maintenance completed https://status.verosint.com/incident/360763 Tue, 30 Apr 2024 02:00:00 -0000 https://status.verosint.com/incident/360763#d520f10001cf41fffbe1e442802413d5612e4b1ed11aea6be8de1b42010fb50c On Monday, April 29, at approximately 9 PM CDT, the Verosint API back-end platform systems will undergo necessary system updates. Operations such as this are handled in a rolling, highly-available manner and we expect no impact to service availability or performance while the maintenance proceeds. https://status.verosint.com/incident/360763 Tue, 30 Apr 2024 02:00:00 -0000 https://status.verosint.com/incident/360763#d520f10001cf41fffbe1e442802413d5612e4b1ed11aea6be8de1b42010fb50c On Monday, April 29, at approximately 9 PM CDT, the Verosint API back-end platform systems will undergo necessary system updates. Operations such as this are handled in a rolling, highly-available manner and we expect no impact to service availability or performance while the maintenance proceeds. https://status.verosint.com/incident/356849 Wed, 17 Apr 2024 15:31:00 -0000 https://status.verosint.com/incident/356849#c3ee795a07b4c8f43eb253484bb18b2f5ced94a862549c3d9f2061c0aa8cceea We are beginning to see recovery events from our auth provider and health monitors are beginning to recover, as well. We will continue monitoring the situation and will update if anything meaningfully changes. https://status.verosint.com/incident/356849 Wed, 17 Apr 2024 15:21:00 -0000 https://status.verosint.com/incident/356849#a583173dcbee567124ecc5c70672f910c8b2d32ed87f2b9c10776204aa51d9f6 Our authentication service provider has reported an issue under investigation which is affecting logins to app.verosint.com. Please see https://status.auth0.com/incidents/hq17dqrvgzxg for additional information. We will continue to provide updates as they become available. https://status.verosint.com/incident/356849 Wed, 17 Apr 2024 15:15:00 -0000 https://status.verosint.com/incident/356849#c56aad5e1c24b5470e0d127779cc41ecd80f5235820742e08beffc389a29c7ed Logins are currently failing in the application dashboard due to a failure in our authentication backend service. We are investigating the issue and will provide updates as progress is made. https://status.verosint.com/incident/346437 Tue, 26 Mar 2024 16:54:00 -0000 https://status.verosint.com/incident/346437#a2bf43db8dda356206ea28dc01e7b1bd738e7baf76299df4b470b41e4b886d50 Our documentation site host has identified and corrected the root cause of the issue and their services are recovering. docs.verosint.com is fully available at this time. https://status.verosint.com/incident/346437 Tue, 26 Mar 2024 16:29:00 -0000 https://status.verosint.com/incident/346437#7b3a04d5809650e8c4f34868caef78be4aca9559c6e1a6199ec8153df51dfab2 Our documentation site host is currently experiencing a significant outage. We apologize for the inconvenience and will provide any material updates as we receive them until the issue is resolved. https://status.verosint.com/incident/327424 Thu, 15 Feb 2024 22:42:00 -0000 https://status.verosint.com/incident/327424#c34a19764e93490362c06c4e98953c651ec6b910834bcebc8b902a45d3b1d2f2 Root cause has been identified and corrected. Analysis indicates that the impact was limited to a subset of internal Verosint systems, but we are continuing to investigate any potential user impact and will update accordingly with any material changes. https://status.verosint.com/incident/327424 Thu, 15 Feb 2024 22:10:00 -0000 https://status.verosint.com/incident/327424#881cacc2ccaa73477694b98ebc3294a8f3b96622e7f31fe3f0711ac08c3c07da Access to the Verosint management application is currently in a degraded state. The incident is currently under investigation and updates will be forthcoming shortly. https://status.verosint.com/incident/283808 Tue, 07 Nov 2023 18:13:00 -0000 https://status.verosint.com/incident/283808#557409035e3598432c067f145fdee824283b16c6f580e2b0cf4fb4dc2c074461 # Incident Overview At 10:19:06 am CST, Verosint's API backend cluster experienced a node failure which caused a brief downtime on an internal middleware service used by SignalPrint and Rules APIs. During the incident, requests to `https://api.verosint.com/v1/signalprint` or `https://api.verosint.com/v1/rules` endpoints would have returned a HTTP 401 error for an invalid API token. The issue was detected and remediated automatically, with full recovery of all services by 10:22:16 am CST. # Impact Requests sent to either API during the incident failed. Log streams sent via Verosint's Auth0 integration also initially failed, but succeeded on subsequent automatic retries once the incident had recovered, with no data loss. # Response and Root Cause Normally, a node loss should be a non-event, as Verosint's engineering standards require all services to be highly-available across multiple availability zones but, in this case, the middleware service was not properly configured and when the cluster node failed, it took the lone instance of the middleware service with it. Fortunately, the cluster detected and remediated the failure 2 minutes after it occurred, with full recovery validated within 1 minute of redeployment. Verosint engineering has corrected the misconfiguration and are taking additional steps to audit/test for improper configurations of every platform service. https://status.verosint.com/incident/283808 Tue, 07 Nov 2023 18:13:00 -0000 https://status.verosint.com/incident/283808#557409035e3598432c067f145fdee824283b16c6f580e2b0cf4fb4dc2c074461 # Incident Overview At 10:19:06 am CST, Verosint's API backend cluster experienced a node failure which caused a brief downtime on an internal middleware service used by SignalPrint and Rules APIs. During the incident, requests to `https://api.verosint.com/v1/signalprint` or `https://api.verosint.com/v1/rules` endpoints would have returned a HTTP 401 error for an invalid API token. The issue was detected and remediated automatically, with full recovery of all services by 10:22:16 am CST. # Impact Requests sent to either API during the incident failed. Log streams sent via Verosint's Auth0 integration also initially failed, but succeeded on subsequent automatic retries once the incident had recovered, with no data loss. # Response and Root Cause Normally, a node loss should be a non-event, as Verosint's engineering standards require all services to be highly-available across multiple availability zones but, in this case, the middleware service was not properly configured and when the cluster node failed, it took the lone instance of the middleware service with it. Fortunately, the cluster detected and remediated the failure 2 minutes after it occurred, with full recovery validated within 1 minute of redeployment. Verosint engineering has corrected the misconfiguration and are taking additional steps to audit/test for improper configurations of every platform service. https://status.verosint.com/incident/283808 Tue, 07 Nov 2023 18:13:00 -0000 https://status.verosint.com/incident/283808#557409035e3598432c067f145fdee824283b16c6f580e2b0cf4fb4dc2c074461 # Incident Overview At 10:19:06 am CST, Verosint's API backend cluster experienced a node failure which caused a brief downtime on an internal middleware service used by SignalPrint and Rules APIs. During the incident, requests to `https://api.verosint.com/v1/signalprint` or `https://api.verosint.com/v1/rules` endpoints would have returned a HTTP 401 error for an invalid API token. The issue was detected and remediated automatically, with full recovery of all services by 10:22:16 am CST. # Impact Requests sent to either API during the incident failed. Log streams sent via Verosint's Auth0 integration also initially failed, but succeeded on subsequent automatic retries once the incident had recovered, with no data loss. # Response and Root Cause Normally, a node loss should be a non-event, as Verosint's engineering standards require all services to be highly-available across multiple availability zones but, in this case, the middleware service was not properly configured and when the cluster node failed, it took the lone instance of the middleware service with it. Fortunately, the cluster detected and remediated the failure 2 minutes after it occurred, with full recovery validated within 1 minute of redeployment. Verosint engineering has corrected the misconfiguration and are taking additional steps to audit/test for improper configurations of every platform service. https://status.verosint.com/incident/280047 Mon, 30 Oct 2023 20:29:00 -0000 https://status.verosint.com/incident/280047#0cd28e131da82fc7aacbdd52e489c308ed561187ca6cece2d042e5f1f0c74d6c The issue at our CDN provider has been resolved and all Verosint services are available. Additional information regarding this incident can be found [here](https://www.cloudflarestatus.com/). Thank you for your patience. https://status.verosint.com/incident/280047 Mon, 30 Oct 2023 20:29:00 -0000 https://status.verosint.com/incident/280047#0cd28e131da82fc7aacbdd52e489c308ed561187ca6cece2d042e5f1f0c74d6c The issue at our CDN provider has been resolved and all Verosint services are available. Additional information regarding this incident can be found [here](https://www.cloudflarestatus.com/). Thank you for your patience. https://status.verosint.com/incident/280047 Mon, 30 Oct 2023 20:29:00 -0000 https://status.verosint.com/incident/280047#0cd28e131da82fc7aacbdd52e489c308ed561187ca6cece2d042e5f1f0c74d6c The issue at our CDN provider has been resolved and all Verosint services are available. Additional information regarding this incident can be found [here](https://www.cloudflarestatus.com/). Thank you for your patience. https://status.verosint.com/incident/280047 Mon, 30 Oct 2023 20:00:00 -0000 https://status.verosint.com/incident/280047#ac5156746423d9660c7e30395b50906aa7f19467c201b82de1aff6d2bbb10a96 Due to an ongoing issue with our CDN provider, app.verosint.com is temporarily unavailable. Existing sessions with api.verosint.com will continue to function until their session tokens expire, after which, the Verosint API will also become unavailable. We will continue to monitor the situation and update as we receive more information from the affected cloud provider. https://status.verosint.com/incident/280047 Mon, 30 Oct 2023 20:00:00 -0000 https://status.verosint.com/incident/280047#ac5156746423d9660c7e30395b50906aa7f19467c201b82de1aff6d2bbb10a96 Due to an ongoing issue with our CDN provider, app.verosint.com is temporarily unavailable. Existing sessions with api.verosint.com will continue to function until their session tokens expire, after which, the Verosint API will also become unavailable. We will continue to monitor the situation and update as we receive more information from the affected cloud provider. https://status.verosint.com/incident/280047 Mon, 30 Oct 2023 20:00:00 -0000 https://status.verosint.com/incident/280047#ac5156746423d9660c7e30395b50906aa7f19467c201b82de1aff6d2bbb10a96 Due to an ongoing issue with our CDN provider, app.verosint.com is temporarily unavailable. Existing sessions with api.verosint.com will continue to function until their session tokens expire, after which, the Verosint API will also become unavailable. We will continue to monitor the situation and update as we receive more information from the affected cloud provider. https://status.verosint.com/incident/267294 Mon, 02 Oct 2023 19:33:00 -0000 https://status.verosint.com/incident/267294#94ad6acac3cf7c70d0b01eaa070bc5b2d65b83bda6b4c04ac85470e0a70488f1 RESOLVED - We are investigating a redirection error in our front end dashboard application. The site at app.verosint.com is redirecting to our non-production endpoint environment, resulting in clients receiving our Cloudflare "Blocked" page. Resolution - Verosint engineering redeployed the frontend dashboard application and the redirection error has cleared. We are continuing to investigate root cause, but do not expect a recurrence. UPDATE: Root cause has been identified as an issue with the hosting provider for the dashboard web application. Additional details can be found at the following link: https://www.cloudflarestatus.com/incidents/t53r6tfsg1sj Again, to reiterate, this incident affected only the browser application. All Verosint API's, integrations, and back-end services remained fully operational during this incident. https://status.verosint.com/incident/267294 Mon, 02 Oct 2023 19:24:00 -0000 https://status.verosint.com/incident/267294#70af96a79523ed6d007e4197fdf09eee7122bbe48d80bd6f4d9c8ee08a2fb7d3 We are investigating a redirection error in our front end dashboard application. The site at app.verosint.com is redirecting to our non-production endpoint environment, resulting in clients receiving our Cloudflare "Blocked" page. https://status.verosint.com/incident/256387 Fri, 08 Sep 2023 04:00:00 +0000 https://status.verosint.com/incident/256387#87137650406f901b28e3208c0353ecee4968be18ea56c0cf639d42c521027e78 Maintenance completed https://status.verosint.com/incident/256387 Fri, 08 Sep 2023 04:00:00 +0000 https://status.verosint.com/incident/256387#87137650406f901b28e3208c0353ecee4968be18ea56c0cf639d42c521027e78 Maintenance completed https://status.verosint.com/incident/256387 Fri, 08 Sep 2023 02:00:00 -0000 https://status.verosint.com/incident/256387#cc8f50b68a6aea471748cf9d8abe655b60d4bbd1cf8a1ac3d86ea287218b9519 We will be performing routine system updates to our API backend application cluster. This is a high-availability operation and service disruption is not expected. https://status.verosint.com/incident/256387 Fri, 08 Sep 2023 02:00:00 -0000 https://status.verosint.com/incident/256387#cc8f50b68a6aea471748cf9d8abe655b60d4bbd1cf8a1ac3d86ea287218b9519 We will be performing routine system updates to our API backend application cluster. This is a high-availability operation and service disruption is not expected. https://status.verosint.com/incident/252465 Tue, 29 Aug 2023 18:07:00 -0000 https://status.verosint.com/incident/252465#5ebb0930acad534f2b84e4e10df426781b801fd2415538626d362bee31915f27 Rule evaluation requests were temporarily failing after underlying library code changed. We have reverted the version of said library and redeployed the previous version of the Rules API, which resolved the incident. https://status.verosint.com/incident/252465 Tue, 29 Aug 2023 18:07:00 -0000 https://status.verosint.com/incident/252465#5ebb0930acad534f2b84e4e10df426781b801fd2415538626d362bee31915f27 Rule evaluation requests were temporarily failing after underlying library code changed. We have reverted the version of said library and redeployed the previous version of the Rules API, which resolved the incident. https://status.verosint.com/incident/239717 Mon, 31 Jul 2023 15:26:00 -0000 https://status.verosint.com/incident/239717#0b485afba293cf0858d9e9cc2e359bb8ee049010574cf5d2bec87c3a619ea109 Between 9:34 am CDT and 9:46 am CDT, deployment of a dependent back-end service coincided with exceptionally high overall platform load, exposing an issue which resulted in a large number of 400 responses from the Rules API. The service degradation automatically resolved once the deployment completed, but Verosint engineering is aware of the issue, working on a permanent fix, and will take additional safety precautions during subsequent deployments of the dependent back-end service until the root cause is fully addressed. https://status.verosint.com/incident/225374 Tue, 27 Jun 2023 22:00:00 +0000 https://status.verosint.com/incident/225374#3b276b7cae627da14ec83dc1cec52aa9bb5d86b8c5ce3508485ceaff6c917913 Maintenance completed https://status.verosint.com/incident/225374 Tue, 27 Jun 2023 21:00:00 -0000 https://status.verosint.com/incident/225374#da5845e67b77c62656d4be9240811188886a17bc4622f45343b34d5ed80922c8 SignalPrint log ingestion will be temporarily paused to facilitate reconfiguration of backend stream resources. The SignalPrint API will continue to function normally during the operation and data sent to the API will buffer until the streams are resumed. Dashboards will not display newly-ingested data until the maintenance is complete, but otherwise we anticipate no impact to service availability. https://status.verosint.com/incident/223152 Wed, 21 Jun 2023 20:55:00 -0000 https://status.verosint.com/incident/223152#3f477cef18441246989a9bf78a3a314631742945d818f6f9025cb82504dc56d5 Following a standard application deployment, monitors for the Rules API and SignalPrint API backends notified Verosint engineering of significantly elevated error responses from those endpoints. Engineering staff determined that a faulty VCS repository configuration allowed the affected services configurations to be updated without deploying the application version for which said configuration was intended. Building and deploying an appropriate artifact restored both services, the associated monitors recovered shortly after the deployment completed, and the configuration which caused the incident has been addressed in the underlying version control system. https://status.verosint.com/incident/223152 Wed, 21 Jun 2023 20:55:00 -0000 https://status.verosint.com/incident/223152#3f477cef18441246989a9bf78a3a314631742945d818f6f9025cb82504dc56d5 Following a standard application deployment, monitors for the Rules API and SignalPrint API backends notified Verosint engineering of significantly elevated error responses from those endpoints. Engineering staff determined that a faulty VCS repository configuration allowed the affected services configurations to be updated without deploying the application version for which said configuration was intended. Building and deploying an appropriate artifact restored both services, the associated monitors recovered shortly after the deployment completed, and the configuration which caused the incident has been addressed in the underlying version control system. https://status.verosint.com/incident/216805 Mon, 05 Jun 2023 20:40:00 -0000 https://status.verosint.com/incident/216805#64965ad7792e50b4374ca6ea4affd5c00aaf5d549051a7b6dcfd81c746ecf530 The API backend experienced a brief service interruption on 05 June 2023, beginning at 20:33 UTC and ending at 20:39 UTC. Root cause was determined to be a stale network configuration on a subset of backend cluster nodes which prevented new ingress service instances from initializing. Rolling replacement of the affected cluster nodes ensured the correct network configuration was applied and API service monitors recovered shortly after the first few nodes joined the cluster. https://status.verosint.com/incident/211000 Tue, 23 May 2023 03:00:00 +0000 https://status.verosint.com/incident/211000#7d33398160f5e03ab99f60aa7f899043f5076f99cde0cb5870f6e34a2e0bc707 Maintenance completed https://status.verosint.com/incident/211000 Tue, 23 May 2023 03:00:00 +0000 https://status.verosint.com/incident/211000#7d33398160f5e03ab99f60aa7f899043f5076f99cde0cb5870f6e34a2e0bc707 Maintenance completed https://status.verosint.com/incident/211000 Tue, 23 May 2023 02:00:00 -0000 https://status.verosint.com/incident/211000#d92a49a92b3c0b89c04395a6684d79a08212a9f1d94896969ebbb4731ebe6291 Upgrading the core network driver in our backend microservice K8S cluster. Minimal service disruption is anticipated during the rollout. https://status.verosint.com/incident/211000 Tue, 23 May 2023 02:00:00 -0000 https://status.verosint.com/incident/211000#d92a49a92b3c0b89c04395a6684d79a08212a9f1d94896969ebbb4731ebe6291 Upgrading the core network driver in our backend microservice K8S cluster. Minimal service disruption is anticipated during the rollout.