Replays

Overview
A replay is used to replay all or a specific subset of your events to a destination.
Replays are at the core of the Batch platform. Replays allow you to replay events in your collections back to your event bus, helping with data loss and disaster recovery.
For information on what destinations we can replay your data to, please see Destinations.
While Batch does transform and annotate your payloads with additional data for display and indexing purposes, your original payload will be replayed back exactly as it was received, in the format in which it was received.
Insights
Creating a Replay
1.Create a Destination
Kafka Destination
2. Goto collection and create a query or leave the default of all messages over the last day
3. Confirm collection is returning messages
Query is returning 1 message
3. Select Replay and fill in details selecting destination we created earlier
Replay Creation
​
Monitoring a Replay
1.Navigate to Replay Tab
2.Select the Replay name
3. View Replay log
Replay log
Pausing a Replay
In some cases users may want to pause a replay. There are several possible reasons for this. A destination may need more resources to handle the data coming in. Another possibility maybe the need to make adjustments to production environments such as silencing events that are triggering notifications end users like an email event.
Pausing a Replay
There is a chance that paused events could repeat a few events after being resumed from a pause since messages are processed in batches.
Replay Behavior and Payloads
During a replay, Batch will "push" events to your configured destination.
Replays take ~30 seconds to start up (and can increase from there, as the data is read from S3).
This list describes how the events are pushed and what their payloads look like.
Replays originate from these IP's.
HTTP
We will perform non-batched, POST requests against the URL that you specified when creating the destination
We will pass any headers that were configured on the destination
The payload will be included in the body of the POST request
The payload is an unmodified version of the original payload we received from you
Kafka
We will perform batched  writes into the bootstrap servers specified when creating the destination
We will replay into the topic configured when creating destination
Supports SASL,TLS, and PLAIN protocol
The payload is an unmodified version of the original payload we received from you
RabbitMQ
We will perform non-batched, writes into servers specified in the DSN when creating the destination
Will replay into the topic configured when creating destination
The payload is an unmodified version of the original payload we received from you
SQS
We will perform non-batched writes into the unique queue name specified when creating the destination
Supports AWS IAM authentication
The payload is an unmodified version of the original payload we received from you
Replay Metering
Replays are counted in the number of GB's transferred; think bandwidth. Your plan comes with a specific number of GB's you can replay per month. 
If you need additional replay, you can either upgrade your plan or purchase a replay add-on.
Throughput
Replay throughput will vary based on where the replay destination is located at.
On average, Batch should be able to replay at:
~10,000 events per second if you are on the US West coast
~5,000 events per second if you are on the US East coast
~1,000 events per second if you are in EU
~500 events per second if you are in Asia
If you need higher throughput, you should try to relocate your destination to be closer to Batch's location. 
If that is not possible, reach out to us and we'll see if it's possible for us to get closer to you.

Sources

Destinations

Last modified 4mo ago

Copy link

Contents

Replay Behavior and Payloads