Replays
Replays are at the core of the Batch platform. Replays allow you to replay events in your collections back to your event bus, helping with data loss and disaster recovery.
While Batch does transform and annotate your payloads with additional data for display and indexing purposes, your original payload will be replayed back exactly as it was received, in the format in which it was received.
- 1.Create a Destination
Kafka Destination
2. Goto collection and create a query or leave the default of all messages over the last day
3. Confirm collection is returning messages
Query is returning 1 message
3. Select Replay and fill in details selecting destination we created earlier
Replay Creation
- 1.Navigate to Replay Tab
- 2.Select the Replay name
3. View Replay log
Replay log
In some cases users may want to pause a replay. There are several possible reasons for this. A destination may need more resources to handle the data coming in. Another possibility maybe the need to make adjustments to production environments such as silencing events that are triggering notifications end users like an email event.
Pausing a Replay
There is a chance that paused events could repeat a few events after being resumed from a pause since messages are processed in batches.
During a replay, Batch will "push" events to your configured destination.
Replays take ~30 seconds to start up (and can increase from there, as the data is read from S3).
This list describes how the events are pushed and what their payloads look like.
- We will perform non-batched,
POSTrequests against the URL that you specified when creating the destination - We will pass any headers that were configured on the destination
- The payload will be included in the body of the
POSTrequest - The payload is an unmodified version of the original payload we received from you
- We will perform batched writes into the bootstrap servers specified when creating the destination
- We will replay into the topic configured when creating destination
- Supports SASL,TLS, and PLAIN protocol
- The payload is an unmodified version of the original payload we received from you
- We will perform non-batched, writes into servers specified in the
DSNwhen creating the destination - Will replay into the topic configured when creating destination
- The payload is an unmodified version of the original payload we received from you
- We will perform non-batched writes into the unique queue name specified when creating the destination
- Supports AWS IAM authentication
- The payload is an unmodified version of the original payload we received from you
Replays are counted in the number of GB's transferred; think bandwidth. Your plan comes with a specific number of GB's you can replay per month.
On average, Batch should be able to replay at:
- ~10,000 events per second if you are on the US West coast
- ~5,000 events per second if you are on the US East coast
- ~1,000 events per second if you are in EU
- ~500 events per second if you are in Asia
If you need higher throughput, you should try to relocate your destination to be closer to Batch's location.
If that is not possible, reach out to us and we'll see if it's possible for us to get closer to you.
Last modified 1yr ago