Replays

Overview

A replay is used to replay all or a specific subset of your events to a destination.

Replays are at the core of the Batch platform. Replays allow you to replay events in your collections back to your event bus, helping with data loss and disaster recovery.

For information on what destinations we can replay your data to, please see Destinations.

While Batch does transform and annotate your payloads with additional data for display and indexing purposes, your original payload will be replayed back exactly as it was received, in the format in which it was received.

Insights

Creating a Replay

  1. Create a Destination

2. Goto collection and create a query or leave the default of all messages over the last day

3. Confirm collection is returning messages

3. Select Replay and fill in details selecting destination we created earlier

Monitoring a Replay

  1. Navigate to Replay Tab

  2. Select the Replay name

3. View Replay log

Pausing a Replay

In some cases users may want to pause a replay. There are several possible reasons for this. A destination may need more resources to handle the data coming in. Another possibility maybe the need to make adjustments to production environments such as silencing events that are triggering notifications end users like an email event.

There is a chance that paused events could repeat a few events after being resumed from a pause since messages are processed in batches.

Replay Behavior and Payloads

During a replay, Batch will "push" events to your configured destination.

Replays take ~30 seconds to start up (and can increase from there, as the data is read from S3).

This list describes how the events are pushed and what their payloads look like.

Replays originate from these IP's.

HTTP

  • We will perform non-batched, POST requests against the URL that you specified when creating the destination

  • We will pass any headers that were configured on the destination

  • The payload will be included in the body of the POST request

  • The payload is an unmodified version of the original payload we received from you

Kafka

  • We will perform batched writes into the bootstrap servers specified when creating the destination

  • We will replay into the topic configured when creating destination

  • Supports SASL,TLS, and PLAIN protocol

  • The payload is an unmodified version of the original payload we received from you

RabbitMQ

  • We will perform non-batched, writes into servers specified in the DSN when creating the destination

  • Will replay into the topic configured when creating destination

  • The payload is an unmodified version of the original payload we received from you

SQS

  • We will perform non-batched writes into the unique queue name specified when creating the destination

  • Supports AWS IAM authentication

  • The payload is an unmodified version of the original payload we received from you

Replay Metering

Replays are counted in the number of GB's transferred; think bandwidth. Your plan comes with a specific number of GB's you can replay per month.

If you need additional replay, you can either upgrade your plan or purchase a replay add-on.

Throughput

Replay throughput will vary based on where the replay destination is located at.

On average, Batch should be able to replay at:

  • ~10,000 events per second if you are on the US West coast

  • ~5,000 events per second if you are on the US East coast

  • ~1,000 events per second if you are in EU

  • ~500 events per second if you are in Asia

If you need higher throughput, you should try to relocate your destination to be closer to Batch's location.

If that is not possible, reach out to us and we'll see if it's possible for us to get closer to you.

PreviousSourcesNextDestinations

Last updated