WebHook Integration

The WebHook Integration App allows the DataEngine to receive WebHooks from external systems and react to events in real time. This enables automatic integration of data from various platforms into the MARINI platform without manual intervention. It facilitates process automation and seamless connection of third-party tools with the platform. The app is highly configurable to support a variety of WebHooks and integration options.

Installation and Setup of the WebHook Integration

To use the WebHook Integration app, follow these steps:

1. Navigate to the admin area of your DataEngine, and under Developer Tools, select Application Loader.
2. Search for the app in the list of installable extensions from the Marini Store and download it.
3. Install the app by clicking the Install button in the list of ready-to-install extensions.
4. Create a module (or use an existing one) in the DataEngine to receive data from the WebHook integration. Ensure the following fields are present in the module for seamless data processing:
   • rawjson: Contains the request’s content as a JSON string.
   • path: Contains the path segment (e.g., xxx/xxx) from the POST request.

If you know in advance how the JSON request fields will map to your module, you can define the field names in the module accordingly. The JSON file will then be automatically parsed, and its contents mapped to the respective fields following these rules:

• Example: JSON {"EventType":"aaa"} is written to the EventType field if it exists.
• Nested objects/arrays can be addressed using an underscore in the field name. For instance, {"Properties":{"ClientId":"aaa"}} can be assigned to properties_clientid as aaa.
• A field named properties can store the entire JSON string, e.g., {"ClientId":"aaa"}.
• Field names in the dataset can be case-insensitive. Custom fields are also checked, so mappings like properties_ClientId_c will work.
• For longer field names, use a custom configuration via the Custom Config mapping. Detailed instructions are available here: Custom Config Documentation.
• If a POST request contains multiple records (comma-separated JSON), automatic field mapping isn’t possible. Use the Workflow Action JSON Split to separate the data into individual records

Setting Up Credentials

To generate a bearer token for access, follow these steps:

1. 1. Use the User Management to create a new user for authorization.
   2. Log in with this user once and set a new password.
   3. Copy the User ID from the URL when viewing the user record in the DataEngine. For example, in a URL like
      https://<your-dataengine-instance>/index.php?module=Users&return_module=Users&action=DetailView&record=aff3e2d5-f063-025c-c7d3-6735d88b2aeb,
      the User ID is the part at the end: aff3e2d5-f063-025c-c7d3-6735d88b2aeb.
   4. Find the name of the module that will receive the WebHook data by checking the URL when navigating to the module, e.g.,
      https://<your-dataengine-instance>/index.php?module=Webhook_Integration. The module name is Webhook_Integration.
   5. Visit: https://<your-dataengine-instance>/index.php?entryPoint=WebhookToken, or access Webhook Access Token in the admin area under External Services.
   6. Enter the saved User ID and module name, then click Generate.
   7. Save the displayed token and signature (if needed) in a password manager, as they are visible only once.

How Can the Module Receive Data via WebHook?

The sender (source) of the WebHook can send POST requests to one of these URLs:

• https://<your-dataengine-instance>/entrypoint/Webhook
• https://<your-dataengine-instance>/entrypoint/Webhook/xxx/xxx (use xxx/xxx for variables, e.g., to identify different sources).

Required Headers for POST Requests:

• Content-Type: application/json
• Authorization: Bearer <Access Token> (replace <Access Token> with your access token)
• X-Signature (optional): <HMAC hash of the content>

The WebHook will always create a new record in the DataEngine. The record is created under the user defined during token generation and in the module specified for the token. Fields are populated according to the mapping rules described earlier.

The WebHook Integration app allows seamless integration of data from external services into the DataEngine, enhancing data processing efficiency.

Recommendations:

1. Data Cleanup: Consider creating a data deletion policy for the module receiving WebHook data, as these records are often only needed temporarily. This improves database management efficiency.
2. Preferred Integrations: For most integrations, use the HubEngine. As an iPaaS solution, it offers seamless integrations. A full list of integrations can be found here. The WebHook Integration serves as an alternative for scenarios where other integration methods are unavailable