gRPC API

Our platform is also able to accept event data via a public gRPC API that uses protobuf.

To send data, call on the AddRecord(AddGenericRecordRequest) method and make sure to pass a gRPC metadata entry that contains a Batch-Token key with the value set to the appropriate collection token.

<Example code here>

Batching

The AddGenericRecordRequest argument accepts an array of messages - this facilitates batching which enables you to send more data in a single transaction.

How many messages you include in a single batch is entirely up to you and you will need to fine-tune this to find an optimal number.

As a starting point:

  • Batch 10 messages every 10s for low throughput

  • Batch 100 messages every 10s for medium throughput

  • Batch 1,000 messages every 10s for high throughput

When should you use this API?

<TODO>

Throughput

Given optimal latency (<50ms) and high batching (1,000+), you should be able to reach 50,000 event/sec ingestion rates.