Webhooks

Use webhooks to be notified about changes that happen to objects connected to a certain service. Roaring can send webhook events that notify your application any time a change of information happens on an object connected to a dataset that you are interested in.

Common use-cases:

• When you have a customer database with information that you don't want to become outdated.
• When you have legal obligations to keep track of your customers (KYC/AML)

The webhook flow explained:

Although webhooks might sound complex at first, it really is a simple solution. Think of it as the world’s shortest game of table tennis where we (Roaring) start the game by serving you a notification of a new data change.

You can either return a ”no I’m not interested in that data” or a ”yes I would like to collect the data”, depending on if the data is related to your customer or not. If you return a ”NO”, it’s game over. However, if you return a ”YES”, the game continues and we smash the data change for you to collect and use.

Datasets available for monitoring

We are constantly adding more event types to the webhook monitoring service to be able to offer monitoring of as many of our available datasets as possible.

In order to see the full list of datasets that can be monitored using our webhook service please log in to our web portal developer.roaring.io/webhooks and navigate to the webhooks page in the developer section.

Apply for Production Data

Get started

Setting up Webhooks

You can register webhook URLs for Roaring to notify any time an event happens in the datasets of your interest. When the event occurs Roaring creates an Event object.

This Event object contains all the relevant information about what just happened, including the type of event, the data associated with that event and data on what has changed. Roaring then sends an Event "continue" request, via an HTTP POST request, to any endpoint URLs that you have defined in your account’s Webhooks settings. In order to then recieve the complete Event payload you need to respond the notification with a HTTP status code of 200 (OK), after which Roaring will HTTP POST the full Event object. You can have Roaring send events concerning a specific dataset to multiple webhook's.

To set up an endpoint, you need to define a route on your server for receiving events, configure webhook settings so Roaring knows where to POST events to, verify your endpoint is valid, and acknowledge your endpoint is receiving events successfully.

Step 1 - Create a webhook endpoint

Webhook data is sent as JSON in the POST request body. The full event details are included and can be used directly, after parsing the JSON into an Event object.

Creating a webhook endpoint on your server is no different from creating any page on your website accepting a HTTP POST payload request.

Step 2 - Configure webhook settings

With your endpoint created, you need to tell Roaring about where to send events to. Webhook endpoints are configured in the Webhooks tab on the Developer portal at developer.roaring.io/webhooks.

Add webhook endpoints

In the Webhooks tab, click the green plus button to reveal a form to add a new endpoint for receiving events. You can enter any URL as the destination for events. However, this should be a dedicated resource context on your server that is set up to receive webhook events. You can choose to be notified of all event types, or only specific ones. You can start off with setting the endpoint to sandbox mode which is a totally free testing environment for webhooks where you can test them by sending single event messages on your own for testing purposes.

Update webhook endpoints

If you need to update or delete a webhook endpoint, you can do so in the webhooks developer tab. You also have the option of disabling a webhook endpoint temporarily. Roaring does not retry any notifications that are generated while the endpoint is disabled.

Step 3 - Send webhook test event

Next, test that your endpoint is working properly. To do this, have your webhook set to sandbox mode, open the menu on the sandbox webhook (three dots to the right) and navigate to the "Make test calls" page. Here you can choose to send test events for the event types you have chosen for the webhook. Choose a request and press send.

The test event will then be sent to your webhook url and you will be able to see the results from the communication attempts in the log which is available under "Log overview" from the menu on your webhook.

Step 4 - Respond to webhook events

Roaring webhook events send notifications for all updates to objects connected to the event types that you choose to monitor.

1. Check the received notifications identifier against your database and decide whether you want the payload event information connected to the notification sent to your webhook url or not.
2. If you have no interest in the notification payload you respond with a HTTP status code of 417 (Expectation Failed) and no more information on the object will be sent.
3. If you want the notification payload you respond with a HTTP status code of 200 (OK) and Roaring will make another POST request to your webhook url with a payload containing the data that initiated the notification.

The webhook events notifications will be sent from the following IP addresses: 52.214.24.45, 52.31.182.3, 52.18.183.55.

To acknowledge receipt of the payload event, your endpoint must return a 200 HTTP status code. Acknowledge events prior to any logic that needs to take place to prevent timeouts.

All response codes outside this range, including 3xx codes, indicate to Roaring that you did not receive the event. This does mean that a URL redirection or a "Not Modified" response is treated as a failure. Roaring ignores any other information returned in the request headers or request body.

Step 5 - Go live!

Once you’ve verified your endpoint is receiving, acknowledging, and handling events correctly, switch the webhook from sandbox mode to production on the Developer pages and go through the configuration again to configure an endpoint for your live integration. If you’re using the same endpoint for both test and live modes, the signing secret is unique to each mode. Make sure to add the new signing secret to your endpoint code if you’re checking webhook signatures.

Events

Events are our way of letting you know when something interesting happens in the datasets we provide. When something interesting occurs or changes, we create a new Event object. This Event object is sent as a notification through our webhook service.

Continue Request

{
  "changeType": "ADD",
  "countryCodes": [
    "se"
  ],
  "keyMap": {
    "personId": 55000,
    "personalNumber": "193103249078"
  },
  "messageId":"1f875023-3ff8-4f2a-98c5-1aa66afefcbb",
  "requestType": "continue",
  "timestamp": 1568203715,
  "type": "pep",
  "version": "1.0"
}

Payload Request

{
  "changeType": "ADD",
  "countryCodes": [
    "se"
  ],
  "keyMap": {
    "personalNumber": "193103249078"
  },
  "messageId":"1f875023-3ff8-4f2a-98c5-1aa66afefcbb",
  "object": {
    "birthDate": "1931-03-24",
    "gender": "Male",
    "imageLink": "https://image.freepik.com/free-icon/protected-person-of-secret-service_318-64589.jpg",
    "names": [
      {
        "firstName": "Jan",
        "lastName": "Bananberg",
        "nameType": "Primary name",
        "nameTypeId": "3"
      },
      {
        "firstName": "Jan Erik",
        "lastName": "Bananberg",
        "nameType": "National registry name",
        "nameTypeId": "4"
      }
    ],
    "pep": true,
    "pepCountries": [
      "Sweden"
    ],
    "personId": 55000,
    "rca": false,
    "relations": [
      {
        "relationPersonId": 55001,
        "relationType": "Partner",
        "relationTypeId": 6
      },
      {
        "relationPersonId": 55002,
        "relationType": "Daughter-in-law",
        "relationTypeId": 9
      },
      {
        "relationPersonId": 55003,
        "relationType": "Son",
        "relationTypeId": 7
      }
    ],
    "roles": [
      {
        "active": false,
        "baseRoleCategoryId": 9,
        "baseRoleCategoryName": "Members of parliament",
        "countryOfJurisdiction": "SWE",
        "detailedRoleCategoryId": 54,
        "detailedRoleCategoryName": "Member of national parliament",
        "fromDateDay": 3,
        "fromDateMonth": 10,
        "fromDateYear": 1992,
        "roleDescription": "Riksdagsledamot"
      },
      {
        "active": false,
        "baseRoleCategoryId": 9,
        "baseRoleCategoryName": "Members of parliament",
        "countryOfJurisdiction": "SWE",
        "detailedRoleCategoryId": 54,
        "detailedRoleCategoryName": "Member of national parliament",
        "fromDateDay": 2,
        "fromDateMonth": 10,
        "fromDateYear": 2006,
        "roleDescription": "Riksdagsledamot"
      }
    ],
    "ssns": [
      {
        "currentSsn": "193103249078",
        "ssnType": "SwePersonalNumber",
        "ssnTypeId": 1
      }
    ]
  },
  "requestType": "payload",
  "timestamp": 1568203715,
  "type": "pep",
  "version": "1.0"
}

Handling webhook events correctly is crucial to making sure your integration’s business logic works as expected.

The event object

An event object is sent two times, once where the actual object initiating the notification is omitted and only the object identifier is sent. We call this a "continue" request and you as a customer use this object identifier to decide if you want Roaring to send you the full payload or not. If you decide you want the payload and answer the request with HTTP status code 200 Roaring will attempt to send over the full payload to the registered webhook url in what we call the "payload" request.

Attribute	Description
timestamp long	Unix timestamp for when the event was created
changeType string	Type of event change, ADD, UPDATE or DELETE
messageId string	Identifier for message, same identifier for continue and payload message
countryCodes array	ISO 3166-1 alpha-2 country codes for country(ies) that the object belong to
keyMap object	Identifier that uniqly identifies the object
object object	The actual object that caused the event to be created. The object will follow the model of the types api model of the version associated with the used version of the webhook event model
previousAttributes object	Previous version of the changed attribute. Only appears for update changeType
newAttributes object	New version of the changed attribute. Only appears for update changeType
requestType string	The webhook request event type, continue or payload
type string	The notification data type
version string	The notification event version

Change Type

The "changeType" field can have one of three different values explaining if the change is indicating an ADD of a new object the UPDATE of an existing object or for some datasets the DELETE of a previously existing object.

Value	Description
ADD	New object has been added to the monitored dataset
UPDATE	There has been an update to an object in the monitored dataset
DELETE	An entity in the monitored dataset has ben removed from it (this changeType is not applicable for all datasets)

Acknowledge payload events immediately

If your webhook script performs complex logic, or makes network calls when handling the payload, it’s possible that the script would time out before Roaring sees its complete execution. Ideally, your webhook payload message handler code (acknowledging receipt of an event by returning a 200 status code) is separate of any other logic you do for that payload event.

Handle duplicate events

Webhook endpoints might occasionally receive the same event more than once. We advise you to guard against duplicated event receipts by making your event processing idempotent. One way of doing this is logging the events you’ve processed, and then not processing already-logged events.

Monitoring lists

Some event types requires monitoring lists in order to receive webhook events. Instead of filtering continue requests, you apply a list of entities you want to monitor. In result, you will only receive payload requests to your webhook service.

An updated monitoring list must be uploaded through the Monitoring list API no later than 18:00 the day before the day of delivery.

CSV File example content

personalNumbers
193103249078

JSON File example content

{
  "personalNumbers": [
    "193103249078"
  ]
}

Upload a monitoring list file

You upload a file which either has the extension .csv or .json. Please notice you can only upload one file. It is possible to do it from the web (Roaring app) or using the api.

Example using the api with cURL:

#!/bin/bash
bundleId="123"
endpoint="https://api.roaring.io/se/person/monitoring/1.0/bundle"
# generate a token from clientId and secret
token="xyz"
# entities file lists personal numbers as described above
bundleFile="entities.json"
contentType="application/json"
curl -k -X PUT $endpoint/$bundleId -H "Authorization: Bearer $token" -H "Content-Type: multipart/form-data" -F "file=@$bundleFile;type=$contentType"

Error handling

If an error occurs when Roaring is trying to send notifications to a webhook url the webhook service will retry the sending of that notification every hour for 96 hours before dropping the notification which then will be lost.

In order to support your development process and make the setup of the webhook's as painless as possible the Roaring webhook Developer page holds the possibility to view logs connected to your webhooks. This makes it easy to locate the retry attempts connected to notifications and troubleshoot your endpoint.

Preventing replay attacks

A replay attack is when an attacker intercepts a valid payload and its signature, then re-transmits them. To mitigate such attacks, Roaring includes a timestamp in the Roaring-Signature header. Because this timestamp is part of the signed payload, it is also verified by the signature, so an attacker cannot change the timestamp without invalidating the signature. If the signature is valid but the timestamp is too old, you can have your application reject the payload.

We recommend that you use Network Time Protocol (NTP) to ensure that your server's clock is accurate and synchronizes with the time on Roaring's servers.

Roaring generates the timestamp and signature each time we send an event to your endpoint. If Roaring retries an event (e.g., your endpoint previously replied with an error), then we generate a new signature and timestamp for the new delivery attempt.

Verifying signatures

The Roaring-Signature header contains a timestamp and one or more signatures. The timestamp is prefixed by t=, and each signature is prefixed by a scheme. Schemes start with v, followed by an integer. Currently, the only valid signature scheme is v1.

Roaring-Signature: t=1568900674, v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd Note that newlines have been added in the example signature header for clarity, but a real Roaring-Signature header will be all one line. Roaring generates signatures using a hash-based message authentication code (HMAC) with SHA-256. To prevent downgrade attacks, you should ignore all schemes that are not v1.

Step 1 - Extract the timestamp and signatures from the header

Split the header, using the , character as the separator, to get a list of elements. Then split each element, using the = character as the separator, to get a prefix and value pair.

The value for the prefix t corresponds to the timestamp, and v1 corresponds to the signature(s). You can discard all other elements.

Step 2 - Prepare the signed_payload string

You achieve this by concatenating:

The timestamp (as a string) The character . The actual JSON payload (i.e., the request’s body)

Step 3: Determine the expected signature

Compute an HMAC with the SHA256 hash function. Use the endpoint’s signing secret as the key, and use the signed_payload string as the message.

Step 4 - Compare signatures

Compare the signature(s) in the header to the expected signature. If a signature matches, compute the difference between the current timestamp and the received timestamp, then decide if the difference is within your tolerance.

To protect against timing attacks, use a constant-time string comparison to compare the expected signature to each of the received signatures.

Authorization header

If you want authentication for your webhook endpoint we provide the possibility to add an Authorization header in your webhook configuration.

As Authorization header you have the possibility to add either Basic authorization credentials i.e. Base64 encoded username and password or a static Bearer token of your choosing.

Basic authorization example

Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=

Bearer token example

Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuYW1lIjoiSm9obiBEb2UifQ.DjwRE2jZhren2Wt37t5hlVru6Myq4AhpGLiiefF69u8

This Authorization header will then be sent with every call to your endpoint adding another shared secret to further secure your endpoint.

Webhook personal number encryption

How to decide what information is interesting to you

In order to protect our customers from handling personal information for persons that are not their customers the parameter "personId" in the continue request is one way HMAC encrypted using the signature secret for their specific webhook. This means that in order to control if the sent information is interesting for them they need to use the signature and encrypt the personal number of their customers and compare to the encrypted record sent by us.

If this proves a match it means that you want to fetch the payload and therefore should answer with an http 200 on our continue request.

How to encrypt a personal number

This is how you encrypt a personal number using the signature secret in order to be able to compare it to the encrypted field sent in the keyMap: Compute an HMAC with the SHA256 hash function. Use the webhook endpoint’s signing secret as the key (available on the webhook page on developer.roaring.io), and use the personal number including century digits and without the - separator as the message (format YYYYMMDDXXXX).