Plumber Relay

This page describes the many ways you can get data into the Batch platform.

Plumber Relay

Plumber is our open source project for working with various messaging systems.

Besides offering read and write functionality, it can also be used for relaying data to Batch (which uses the gRPC API under the hood).

Relaying data using plumber is the most reliable and performant way to get data into Batch as plumber makes use of batching events which can increase your total throughput.

You can launch plumber relays in multiple ways:

  • Running plumber in single-relay mode via CLI

    • Best for quick, one-offs

  • Running plumber as a docker container

    • Best for ephemeral workloads

  • Running plumber in server mode

    • Best for production

The following examples show how to run plumber in single relay mode.

For production deployments, we suggest to deploy plumber running in server mode.

plumber relay kafka \
  --address "" \
  --topics orders \

In this example, all messages from kafka topic new_orders will be automatically sent to the collection with the specified relay token.

More examples of relaying from various systems can be found in

When should you use this API?

Plumber is the easiest way to relay throughput heavy workloads and should be used by anyone wanting to get up and running quickly.


plumber uses gRPC under the hood to communicate with Batch's collectors.

You should be able to comfortably reach 25K-50K messages/sec on a single plumber instance. To reach higher levels, you should run plumber in cluster server mode and launch 2+ replicas of plumber.

Make sure to use the same consumer group if relaying for backends such as Kafka or NATS.

Last updated