Ingest

The Events Ingest API provides an interface for on-premise MTA users to quickly, easily, and safely take advantage of SparkPost's cloud-specific services. The Events Ingest API accepts email event data, normalizes it, and sends it through SparkPost's data pipeline until it is ultimately consumable by various analytical services.

Event Format

Events sent to the Ingest API must adhere to a specific format in order to be correctly processed by SparkPost. Only certain aspects of incoming API requests are validated during the REST transaction. Requests that do not pass validation at this point are rejected and a HTTP error code will be returned. The body of the request must contain gzip-encoded, new-line-delimited JSON along with the appropriate headers. The size of the request is limited to 5 MB and the age of events must be less than 24 hours.

Additional validation occurs out of band of the request. This additional validation ensures that events specified as part of the request body are properly formatted and consumable by SparkPost analytical services. Events that are not proper JSON, lack the correct wrapper, or are missing required fields will be dropped. Please see Processing Out of Band Errors for additional information.

Processing Out of Band Errors

Every uploaded batch of events can be tracked via the id returned from the API after a successful POST. This batch ID can be used in conjunction with the Events API to view the outcome of SparkPost's pre-processing.

Note: the ingested email events may still take a few moments before they are viewable

For more on viewing ingested status events, see the events documentation

Ingest

Additional Ingest Notes:

For GETs, If using cURL, make sure the flags -O & -J are used. The -O flag will allow the file to be saved to the current location. The -J indicates to cURL to grab the file as a gzip.

Request

POST /api/v1/ingest/events

Response

{
  "results": {
    "id": "b2013586-c13f-4da4-baf4-11b9d8e40c50"
  }
}

Send Ingest Events

POST/api/v1/ingest/events

Post a new-line delimited list of JSON events.

Required Headers

Header	Value
`Content-Encoding`	`gzip`
`Content-Type`	`application/x-ndjson`

Example

{"msys":{"message_event":{"msg_batchname":"1A/50-00001-B03B45C5","template_version":"0","msg_spoolname":"2A/50-00001-B03B45C5","msg_size":"315","bounce_class":"25","template_id":"template_12994410358768097","rcpt_tags":[ ],"recv_method":"rest","binding_group":"default","routing_domain":"example.com","binding":"default","rcpt_to":"recipient@example.com","open_tracking":"1","msg_from":"sender@example.com","type":"inband","raw_reason":"551 5.7.0 [internal] recipient blackholed","conn_name":"00/00-00000-00000000","rcpt_meta":{ },"error_code":"551","message_id":"00000bb3545c0100a205","conn_stage":"22","reason":"551 5.7.0 [internal] recipient blackholed","node_name":"8c9f45e70999","queue_time":"2","click_tracking":"1","num_retries":"0","event_id":"12994410358769058","subject":"Summer deals are here!","timestamp":"1549054731","snippet_count":"0","transmission_id":"12994410358768097"}}}
{"msys":{"message_event":{"template_id":"template_12994410358768098","open_tracking":"1","msg_spoolname":"5A/50-00001-B03B45C5","binding_group":"default","message_id":"00000bb3545c0100a505","msg_size":"315","timestamp":"1549054731","template_version":"0","subject":"Summer deals are here!","pathway_group":"default","rcpt_meta":{ },"snippet_count":"0","pathway":"default","msg_batchname":"4A/50-00001-B03B45C5","event_id":"12994410358769060","click_tracking":"1","conn_stage":"22","type":"reception","conn_name":"00/00-00000-00000000","node_name":"8c9f45e70999","transmission_id":"12994410358768098","rcpt_tags":[ ],"recv_method":"rest","rcpt_to":"recipient@example.com","msg_from":"sender@example.com","routing_domain":"example.com","binding":"default"}}}

Request Body

msys object required

The event wrapper

Request

GET /api/v1/ingest/events/e038e809-1308-461c-83b7-fa73c9f50500

Response

  Saved to filename "b2013586-c13f-4da4-baf4-11b9d8e40c50".gz

Get Ingest Batch

GET/api/v1/ingest/events/{id}

Parameters

id number required

Ingest batch ID.

Downloads a gzip file of the ingest batch events. The binary of the gzip will be displayed in the response body.

Request

GET /api/v1/ingest/events/failures/e038e809-1308-461c-83b7-fa73c9f50500

Response

  Saved to filename "b2013586-c13f-4da4-baf4-11b9d8e40c50".gz

Get Failure Ingest Batch

GET/api/v1/ingest/events/failures/{id}

Parameters

id number required

Ingest batch ID.

Downloads a gzip file of the ingest batch with failures. The binary of the gzip will be displayed in the response body.

Documentation

These endpoints are used to generate the samples and descriptions for the Send Events section.

Additional Event Notes:

For Sending IP related reporting, processing will fallback to the binding field if the sending_ip field is not present.

Request

GET /api/v1/ingest/events/documentation

Response

{
  "results": [
    {
      "type": {
        "description": "Type of event this record describes",
        "sampleValue": "delivery",
        "required": true,
        "reporting": true
      },
      ...
    },
    ...
  ]
}

Ingest Events Documentation

GET/api/v1/ingest/events/documentation

Returns a list of descriptions of the event fields that could be included in a POST to the ingest events API. Fields will vary by event type. Event fields including "required": true must be present in the event and fields including "reporting": true are recommended as they are used to drive various reporting services.

Request

GET /api/v1/ingest/events/samples?events=delivery

Response

{
  "results": [
    {
      "type": "delivery",
      ...
    }
  ]
}

Ingest Events Samples

GET/api/v1/ingest/events/samples{?events}

Parameters

events string

Event types for which to get a sample payload. Defaults to all event types. See Events Types for the list of available event types.

Returns an example message event for each event type listed in the events parameter.