We have completed the main recovery efforts related to this incident. The small amount of previously affected recurring orders are starting to flow through the placement process.
Posted Mar 08, 2025 - 21:12 EST
Update
All valid Shopify events have been replayed successfully. We are now entering the next and final phase of the recovery effort.
There is a small number of recurring orders that were impacted by this issue which will be retried over the next 24-48 hour period. We are taking extra precautionary measures to ensure that we only retry orders which were not properly processed by Shopify. No action needs to be taken and we will provide a final update once the task is completed by our team.
Posted Mar 08, 2025 - 05:42 EST
Update
All of the services have recovered and message delivery is back to normal. We are in the process of replaying any missed transactions. No action needs to be taken, our team will make sure all events will be reprocessed.
We will provide further updates as progress is made.
Posted Mar 07, 2025 - 17:53 EST
Monitoring
We have identified an issue related to internal systems that process Shopify events from 03/06/2025 4PM EST and until 03/07/2025 9AM EST. The underlying issue has been resolved and we are actively investigating all impacted areas that require manual intervention. No action needs to be taken, our team will make sure all events will be reprocessed.
Areas impacted: - Increase in recurring order placement retry attempts due to connection errors - Delayed delivery of events and updates tied to certain customer actions such as change of frequency or payment updates
Next Steps: - We are working diligently to ensure that all delayed events are being processed - We are reviewing all potentially impacted recurring orders to ensure proper placement
We will provide further updates throughout the day
Posted Mar 07, 2025 - 11:11 EST
This incident affected: Ordergroove Platform (e-commerce platform agnostic) (Order Placement) and Shopify (Ordergroove App - Subscription Creation & Webhooks).