Our platform is also able to accept event data via a public gRPC API that uses protobuf.
AddGenericRecordRequestargument accepts an array of messages - this facilitates batching which enables you to send more data in a single transaction.
How many messages you include in a single batch is entirely up to you and you will need to fine-tune this to find an optimal number.
As a starting point:
- Batch 10 messages every 10s for low throughput
- Batch 100 messages every 10s for medium throughput
- Batch 1,000 messages every 10s for high throughput
gRPC is intended for high throughput data collection. The goal of the gRPC api is to provide the ability to batch data and send heavy throughput workloads.
Given optimal latency (<50ms) and high batching (1,000+), you should be able to reach 50,000 event/sec ingestion rates.