Lobster CEP Connector

The CEP Connector offers one unified integration to book shipments, fetch labels, and track parcels across various Courier, Express, and Parcel service providers. No need to juggle individual carrier endpoints or payload formats. The customer only needs to integrate the tailored, unified data format LobsterCEP and the CEP Connector does the rest.

Key use cases

• Book shipments and generate shipping labels in PDF, ZPL, or PNG formats

• Retrieve near real-time tracking updates throughout the delivery process

• Obtain Proof-of-Delivery (POD) documents once shipments are delivered

Prerequisites

Before you begin, ensure the following requirements are met:

• Supported Lobster Data Platform version (minimum 25.1)

• CEP API access and credentials for each carrier (username/password or client ID/client secret)

How it works

The CEP Connector includes a package of profiles of two different types:

• CEP_CONNECTOR profiles are fixed implementations containing all the actual    functionality of the CEP Connector. They contain mappings between the API formats of the    particular CEP providers and the abstracted unified format LobsterCEP.    

•  WARNING You must not change these to ensure the functionality and maintainability of the connector.

• TEMPLATE_LobsterCEP profiles are predefined templates to easily integrate    the connector profiles through the LobsterCEP format.

Shipment booking

The shipment booking process books carrier shipments via their public APIs. The CEP Connector calls a mapper profile to convert the LobsterCEP shipment transport order into the requested carrier format. It then sends this data to the CEP system, which returns a synchronous response. The connector converts this response back to the LobsterCEP structure using a specific CEP mapper profile. Finally, it forwards the processed data to your customer profile as a shipment transport order acknowledgement — either a confirmation including the shipment label or an error response with specific feedback from the carrier's endpoint.

Tracking

The tracking process retrieves shipment status updates from the CEPs via their public APIs. It reads Lobster Integration configuration files under ./conf/Connectors/CEP/Tracking containing tracking details in CSV format. The polling logic only fetches new tracking events, preventing redundant data retrieval. When you use the shipment booking process, tracking configuration files are automatically created and updated.

You can also fill these files manually for shipments booked elsewhere. The tracking files are structured as follows:

carrierTrackingNumber;customerShipmentId;latestEventTimestamp;shipmentCreationTimestamp

The latestEventTimestamp will be filled by the connector and should be left blank in the initial setup.

Example: to automatically track shipment test123 with DHL tracking number 1091847043 created at 2025-09-08 10:00:00, append to ./conf/Connectors/CEP/Tracking/List_ShipmentTrackingNumbers_DHL.csv:

1091847043;test123;;1757318400000

For FedEx, UPS, or PostNL, append to FEDEX.csv, UPS.csv, or PostNL.csv respectively instead of DHL.csv.

Installation

For Lobster Data Platform customers, preconfigured dataflows are available that simplify the integration with your CEP providers. These dataflows contain:

• CEP Connector profiles that integrate various CEP APIs

• Template profiles for your mappings from or to the LobsterCEP shipment structure

• Partner channels for your CEP Providers

• System constants to customize your CEP Connector

1. Click Home → Start from template.

2. Click Install on the CEP Connector template tile.

   You are automatically redirected to Integration → DataFlow → Designer.

3. The Import package dialog opens. It is organized into four tabs:

   • Contents — Lists all included profiles and DataFlows with type, name, status, and version.

   • Constants — Displays constants included in the package (in this package: 0).

   • Partners/Channels — Displays partners and partner channels included in the package with import options.

   • Information — Contains supplementary information about the package.

Contents

The Contents tab displays all components included in the package:

The package contains the following components::

Note

All profiles in this package have the status Active and are ready for use immediately after import. The tab badges indicate: Table of Contents (17 entries), Constants (5), Partners/Channels (1/3).

Group selection

Select the appropriate local groups for the profiles or create new ones by clicking each folder icon in the column Group (local).

• In the Constants tab you can find all predefined constants. Existing constants with different or identical values will not be overwritten by default.

• In the Partner channel tab, you can select if you want to import or overwrite the preconfigured partner channels. If overwrite is not toggled, an update will not change the partner channel, so you can safely update the connector.

Constants

• In the Constants tab you can find all predefined constants. Existing constants with different or identical values will not be overwritten by default.

If these constants already exist with other or the same values, they will not be overwritten by default.

Partner/Channels

The Partners/Channels tab displays the partners and partner channels included in the package. This package contains one partner with three partner channels. You can select if you want to import or overwrite the preconfigured partner channels. If overwrite is not toggled, an update will not change the partner channel, so you can safely update the connector.

• Import partners — Imports the partner included in the package (CEP Connector) along with the associated HTTPS channel. This option is enabled by default.

• Overwrite partners/channels — Overwrites existing partners and channels with the same name. Only enable this option if you intentionally want to replace an existing configuration.

Important

If the partner CEP connector already exists in your environment and you have made custom modifications, leave the Overwrite partners/channels option disabled to preserve your configuration.

Complete the import

Review the displayed components across all tabs, click Import, and confirm to start the import.    

2. Configure credentials

Required credentials per carrier

Partner channel configuration

Navigate to Administration → Partner → Partners/Channels → CEP Connector.

DHL_Express_HTTPS

• Fill in the DHL Express username as “Own ID”

• Fill in the DHL Express password as “Own Password”

FedEx_HTTPS

1. Click “Configure OAuth 2.0…”

2. Fill in your Client ID and Client Secret

3. Select Grant Type “Client Credentials”

4. Fill in the “Endpoint URL”:    

   • Test: https://apis-sandbox.fedex.com/oauth/token

   • Prod: https://apis.fedex.com/oauth/token

5. Click “Fetch Access Token”

6. Fill in the “OAuth2 Refresh URL”:    

   • Test: https://apis-sandbox.fedex.com/oauth/token?grant_type=client_credentials&client_id=<clientId>&client_secret=<clientSecret>

   • Prod: https://apis.fedex.com/oauth/token?grant_type=client_credentials&client_id=<clientId>&client_secret=<clientSecret>

UPS_HTTPS

1. Click “Configure OAuth 2.0…”

2. Fill in your Client ID and Client Secret

3. Select “Send ‘client credentials’ in header”

4. Select Grant Type “Authorization Code”

5. Fill in the “Endpoint URL”:    

   • Test: https://wwwcie.ups.com/security/v1/oauth/token

   • Prod: https://onlinetools.ups.com/security/v1/oauth/token

6. Fill in your “Redirect URL” (your internal Lobster Data Platform address):
       https://<your-server>:<your-port>/dw/oauth2/<your-Application-ID>
       The Application ID can be found in your OAuth 2.0 settings and starts with _data.
       This URL must also be registered in the UPS developer portal application settings (required for both Test and Prod in the same application).

7. Fill in the “OAuth URL”:    

   • Test: https://wwwcie.ups.com/security/v1/oauth/authorize

   • Prod: https://onlinetools.ups.com/security/v1/oauth/authorize

8. Click “Fetch Access Token”. You will be redirected to UPS to enter your developer account credentials. A successful redirect shows a blank page with ok. After reopening the channel, you should see SYS_HTTP_OAUTH2 and SYS_HTTP_OAUTH2_REFRESH tokens in “Additional IDs”.

9. Fill in the “OAuth2 Refresh URL”:    

   • Test: https://wwwcie.ups.com/security/v1/oauth/refresh?grant_type=refresh_token&refresh_token=<refreshToken>&client_id=<clientId>&client_secret=<clientSecret>&useCredentialsInHeader

   • Prod: https://onlinetools.ups.com/security/v1/oauth/refresh?grant_type=refresh_token&refresh_token=<refreshToken>&client_id=<clientId>&client_secret=<clientSecret>&useCredentialsInHeader

PostNL_HTTPS

• Add your API key as “SYS_HTTP_apikey” in the “Additional IDs” tab

3. Use profile templates for custom mappings

Warning

Modifying CEP_CONNECTOR profiles can break functionality and maintainability.

Constants for Configuration

The following system constants can be modified to customize the connector. (M) = mandatory, (D) = dependent, (O) = optional.

Input data structure: Shipment booking

Pass the following LobsterCEP data structure to the CEP_CONNECTOR_ShipmentBooking profile. The connector maps it to the carrier’s format, calls the carrier’s API internally, and forwards the result to your acknowledgement profile.

Output data structure

Output data structure example: Confirmed booking

{
  "LobsterCEP": {
    "header": {
      "carrierId": "DHL_EXPRESS",
      "processingStatus": {
        "code": "CONFIRMED",
        "realizationDateTime": "2025-08-13T09:33:59+02:00"
      }
    },
    "shipment": {
      "shipmentId": "ABC123456789",
      "carrierReference": "987654321",
      "estimatedDeliveryDate": "2025-08-18T23:59:00Z",
      "documents": [
        {
          "type": "LABEL",
          "name": "LABEL_987654321.pdf",
          "format": "PDF",
          "content": "<base64-encoded content>"
        }
      ],
      "trackingUrl": "https://express.api.dhl.com/mydhlapi/test/shipments/987654321/tracking"
    }
  }
}

Output data structure example: Error

{
  "LobsterCEP": {
    "header": {
      "carrierId": "DHL_EXPRESS",
      "processingStatus": {
        "code": "ERROR",
        "reason": "1001: The requested product(s) (P) not available based on your search criteria.",
        "realizationDateTime": "2025-08-13T09:33:59+02:00"
      }
    },
    "shipment": {
      "shipmentId": "ABC123456789"
    }
  }
}

Output data structure: Tracking

The CEP_CONNECTOR_Tracking profile polls the carrier automatically for new events and delivers the following LobsterCEP structure to your tracking target profile. No call is required from your side. Configure the tracking CSV files as described in the Tracking section, and the connector handles polling automatically.

Output data structure tracking example

{
  "LobsterCEP": {
    "header": { "carrierId": "DHL_EXPRESS" },
    "shipment": [
      {
        "shipmentId": "ABC123456789",
        "carrierReference": "987654321",
        "estimatedDeliveryDate": "2025-08-18T23:59:00Z",
        "trackingEvents": [
          {
            "owner": "DHL_EXPRESS",
            "eventDateTime": "2025-09-01T01:43:03Z",
            "eventLocation": "CPH (Copenhagen-DK)",
            "statusDetails": [{ "code": "OK", "statusDescription": "Delivered" }]
          }
        ],
        "documents": [
          {
            "type": "POD",
            "name": "POD_987654321.pdf",
            "format": "PDF",
            "content": "<base64-encoded content>"
          }
        ],
        "trackingUrl": "https://express.api.dhl.com/mydhlapi/test/shipments/987654321/tracking"
      }
    ]
  }
}

LobsterCEP data format

The LobsterCEP object is the unified data format for all CEP Connector operations. Below is a complete field reference.

Header

Contains carrier selection, action type, and document requirements.

Shipment

Contains all shipment details.

serviceType values by carrier

Parties and addresses

All parties involved in the shipment. shipper and recipient are required.

Address fields

Payment information

Packages and line items

Package fields

Dimensions

All measurement values have a value (number) and unit field.

Line item fields

Customs information

Available at shipment level, package level, and line item level.

Documents

Documents are returned in the booking acknowledgement and tracking responses.

Tracking response

Tracking event fields

Support

For questions regarding Data Flow configuration, please contact the Lobster Support team.

• Website: www.lobster-world.com

• Email: support@lobster.cloud