POST /v1/collection
Description
Create a new collection.
Body Parameters
Responses
200 - Collection created successfully
{
"id": "566762f6-d06f-448f-911e-dea9f9315edb",
"name": "cli test",
"notes": "",
"token": "62d11890-f3c3-42a4-9e73-0877dfe16bfc",
"paused": false,
"inserted_at": "2021-02-16T15:46:30.897801Z",
"updated_at": "2021-02-16T15:46:30.897801Z",
"disable_archiving": false,
"team": {
"id": "7b66ed87-5ffb-494b-9f8e-aed519054045",
"name": "marktest-1",
"inserted_at": "2021-02-16T15:38:51.800964Z",
"updated_at": "2021-02-16T15:38:51.800964Z"
},
"schema": {
"id": "4459dae2-6b40-408c-8379-abb36f16108f",
"name": "Generic JSON",
"root_type": "",
"type": "json",
"shared": "false",
"archived": false,
"inserted_at": "2021-02-16T15:38:51.800964Z",
"updated_at": "2021-02-16T15:38:51.800964Z"
},
"datalake": {
"id": "a4d9397a-bdfb-4e22-b348-2f437e7643ca",
"type": "aws",
"name": "Default DataLake",
"status": "active",
"status_full": "",
"inserted_at": "2021-02-16T15:38:51.800964Z",
"updated_at": "2021-02-16T15:38:51.889373Z"
},
"author": {
"id": "e3ea07be-7565-4c25-9fb7-515d2b6eb582",
"name": "mark test",
"email": "mark-test@batch.sh"
},
"plan": {
"id": "ddc4915c-b0e0-4a6b-8911-bf93529d6019",
"stripe_plan_id": "prod_IlTxuV7gXfJic8",
"plan_attributes": {
"num_seats": 1,
"replay_gb": 1,
"storage_gb": 1,
"num_collections": 1,
"trial_available": true,
"trial_length_days": 14
},
"plan_usage": {
"seats": 1,
"collections": 0
},
"status": "active",
"status_reason": "account created",
"inserted_at": "2021-02-16T15:38:51.800964Z",
"updated_at": "2021-02-16T15:38:51.800964Z"
}
}
412 - Out of plan resources
{"error": "max collections reached"}
422 - Invalid data provided
{
"errors": [
{
"code": 422,
"domain": "collection",
"field": "schema_id",
"status": "invalid",
"raw_error": "",
"message": "must be a valid UUID v4"
}
]
}
GET /v1/collection
Description
Get all collections.
Example Response
{
"0dce7d85-8d61-41db-86bd-af005df755f4":{
"latency_ms":0,
"status":"error",
"message":"Failed to connect",
"last_checked_at":1624548189
}
}
GET /v1/collection/{id}
Description
Get one collection.
Path Parameters
Example Response
{
"0dce7d85-8d61-41db-86bd-af005df755f4":{
"latency_ms":0,
"status":"error",
"message":"Failed to connect",
"last_checked_at":1624548189
}
}
PUT /v1/collection
Description
You can archive a collection by passing "archived": true in the payload.
Archived collections CANNOT be deleted - archive is functionally equivalent to delete.
Body Parameters
Example Response
{
"0dce7d85-8d61-41db-86bd-af005df755f4":{
"latency_ms":0,
"status":"error",
"message":"Failed to connect",
"last_checked_at":1624548189
}
}
PUT /v1/collection/{id}/toggle-pause
Description
Pause or resume a collection.
Path Parameters
Body Parameters
Example Response
{
"0dce7d85-8d61-41db-86bd-af005df755f4":{
"latency_ms":0,
"status":"error",
"message":"Failed to connect",
"last_checked_at":1624548189
}
}
POST /v1/collection/{id}/search
Description
Search a collection.
Path Parameters
Body Parameters
Example Lucene Query
client.payload.hello: "world" AND batch.info.date_human: [2020-06-25T22:40:44.638Z TO 2021-06-25T22:40:44.639Z]
Example Response
{
"total": 4,
"is_replayable": true,
"data": [
{
"@timestamp": "2021-06-17T23:46:29Z",
"batch": {
"info": {
"date_human": "2021-06-17T23:46:29Z",
"date_unix": "1623973589784716776",
"request_id": "4df0f08e-0cc4-4e73-bda9-d82bf83858e2",
"source": "grpc-collector_grpc-collector-deployment-65676d4f86-rk6s9"
}
},
"client": {
"metadata": {
"header.test": "123",
"key": "",
"offset": "17",
"partition": "0",
"timestamp": "1623973589608730000",
"topic": "foo",
"transport": "kafka"
},
"payload": {
"hello": "world"
}
}
},
{
"@timestamp": "2021-06-17T23:42:29Z",
"batch": {
"info": {
"date_human": "2021-06-17T23:42:29Z",
"date_unix": "1623973349778615096",
"request_id": "2d94b699-7ecd-490a-92ae-c3537c828de4",
"source": "grpc-collector_grpc-collector-deployment-65676d4f86-8xdnh"
}
},
"client": {
"metadata": {
"header.test": "123",
"key": "",
"offset": "16",
"partition": "0",
"timestamp": "1623973349591253000",
"topic": "foo",
"transport": "kafka"
},
"payload": {
"hello": "world"
}
}
},
{
"@timestamp": "2021-06-17T23:41:19Z",
"batch": {
"info": {
"date_human": "2021-06-17T23:41:19Z",
"date_unix": "1623973279784094110",
"request_id": "c50cb190-a7b2-4da9-8729-e4962a2d56d1",
"source": "grpc-collector_grpc-collector-deployment-65676d4f86-zpwwj"
}
},
"client": {
"metadata": {
"header.test": "123",
"key": "",
"offset": "15",
"partition": "0",
"timestamp": "1623973279590935000",
"topic": "foo",
"transport": "kafka"
},
"payload": {
"hello": "world"
}
}
},
{
"@timestamp": "2021-06-17T23:39:49Z",
"batch": {
"info": {
"date_human": "2021-06-17T23:39:49Z",
"date_unix": "1623973189782415716",
"request_id": "0115f7d5-20fa-400a-bc14-e59dd6ac3c77",
"source": "grpc-collector_grpc-collector-deployment-65676d4f86-rk6s9"
}
},
"client": {
"metadata": {
"header.test": "123",
"key": "",
"offset": "14",
"partition": "0",
"timestamp": "1623973189591469000",
"topic": "foo",
"transport": "kafka"
},
"payload": {
"hello": "world"
}
}
}
]
}
POST /v1/collection/{id}/extended-search
Description
Initiate an extended search.
Path Parameters
Body Parameters
Example response
{
"task_id": "0c2afdf6-3e08-477e-9770-f1aa976a898e",
"status": "processing"
}
GET /v1/collection/{id}/extended-search-results
Description
Get extended search results.
Path Parameters
Example response
{
"next_page": "fdsgfg079sdgfz9g87fd90gfd===",
"query_id": "84d97fed-9610-4b4b-ab6c-ef9279758758",
"total_results": 300,
"data": [
{"batch": {...}, "client": {...}},
{"batch": {...}, "client": {...}},
{"batch": {...}, "client": {...}},
{"batch": {...}, "client": {...}},
{"batch": {...}, "client": {...}},
]
}
Omitting it will always get the first page of results.
Specifying it with the value of "next_page" will get the next page of 20 results.
If it is empty, there are no more pages of results
POST /v1/collection/{id}/initial
Description
Create initial event task
Path Parameters
Body Parameters
Example Response
{
"0dce7d85-8d61-41db-86bd-af005df755f4":{
"latency_ms":0,
"status":"error",
"message":"Failed to connect",
"last_checked_at":1624548189
}
}
POST /v1/collection/{id}/zip
Description
Request a ZIP archive of search results.
Path Parameters
Body Parameters
Example Response
{
"0dce7d85-8d61-41db-86bd-af005df755f4":{
"latency_ms":0,
"status":"error",
"message":"Failed to connect",
"last_checked_at":1624548189
}
}
