Lobster CEP Connector

Version 1.0.0

The CEP Connector provides one unified integration for CEP services (Courier, Express, and Parcel providers). It supports shipment booking, label retrieval, and shipment tracking across multiple carriers. Integrate the unified data format LobsterCEP to get started. The CEP Connector handles the rest.

Key use cases

• Book shipments and generate shipping labels in PDF, ZPL, or PNG formats.

• Retrieve tracking updates throughout the delivery process near real time.

• Obtain Proof of Delivery (POD) documents once shipments are delivered.

Requirements

Before you begin, ensure the following requirements are met:

• Lobster Data Platform, version 25.1 or later.

• CEP API access and credentials for each carrier (username/password or client ID/client secret).

How it works

The CEP Connector includes a package of profiles of two different types:

• CEP_CONNECTOR profiles are fixed implementations containing all the functionality of the CEP Connector. They contain mappings between the API formats of the CEP providers and the unified format LobsterCEP.

• TEMPLATE_LobsterCEP profiles are predefined templates. They simplify integration through the LobsterCEP format.

IMPORTANT

Do not modify CEP_CONNECTOR profiles. Modifications break the functionality and maintainability of the CEP Connector.

Shipment booking

The shipment booking process books carrier shipments via their public APIs. The CEP Connector calls a mapper profile. It converts the LobsterCEP transport order into the carrier's required format. The connector then sends the data to the CEP system. The CEP system returns a synchronous response. The connector converts this response back to the LobsterCEP structure using a specific CEP mapper profile. Finally, it forwards the processed data to your customer profile as a transport order acknowledgement. The response is either a booking confirmation including the shipment label, or an error response with carrier feedback.

Tracking

The tracking process retrieves shipment status updates from the CEP providers via their public APIs. It reads Lobster Integration configuration files under ./conf/Connectors/CEP/Tracking. These files contain tracking details in CSV format. The polling logic fetches only new tracking events. This prevents redundant data retrieval. When you use the shipment booking process, the CEP Connector creates and updates the tracking configuration files automatically.

You can also fill these files manually for shipments booked elsewhere. The tracking files use the following format:

carrierTrackingNumber;customerShipmentId;latestEventTimestamp;shipmentCreationTimestamp

The latestEventTimestamp field is filled automatically by the connector. Leave it blank during initial setup.

Example: to automatically track shipment test123 with DHL tracking number 1091847043 (created 2025-09-08 at 10:00:00), add the following line to ./conf/Connectors/CEP/Tracking/List_ShipmentTrackingNumbers_DHL.csv:

1091847043;test123;;1757318400000

For FedEx, UPS, or PostNL, use FEDEX.csv, UPS.csv, or PostNL.csv instead of DHL.csv.

Installation

Preconfigured dataflows are available for Lobster Data Platform customers. They simplify integration with your CEP providers. The dataflows include:

• CEP Connector profiles for integrating various CEP APIs.

• Template profiles for your mappings to or from the LobsterCEP shipment structure.

• Partner channels for your CEP providers.

• System constants to customize the CEP Connector.

Step 1: Import package

1. Select Home > Start from template.

2. Select Install on the CEP Connector template tile.

   You are automatically redirected to Integration > DataFlow > Designer.

   The Import package dialog opens. It is organized into four tabs:

   • Contents: Lists all included profiles and DataFlows with type, name, status, and version.

   • Constants: Displays constants included in the package.

   • Partners/Channels: Displays partners and partner channels included in the package with import options.

   • Information: Contains supplementary information about the package.

Contents tab

The Contents tab displays all components included in the package:

The package contains the following components:

NOTE

All profiles in this package have the status Active and are ready for use immediately after import.

Group selection

Select the appropriate local groups for each profile, or create new ones. Click the folder icon in the Group (local) column.

Constants tab

The Constants tab displays the predefined constants for customizing the CEP Connector.

Existing constants are not overwritten by default, regardless of whether their values differ.

Partners/Channels tab

The Partners/Channels tab displays the partners and partner channels included in the package. This package contains one partner with three partner channels. Select whether to import or overwrite the preconfigured partner channels. If you do not enable overwrite, updates leave existing channel configurations unchanged.

The following options are available:

• Import partners: Imports the CEP Connector partner along with the associated HTTPS channel. This option is enabled by default.

• Overwrite partners/channels: Overwrites existing partners and channels with the same name. Enable this option only if you intentionally want to replace an existing configuration.

IMPORTANT

If the CEP Connector partner already exists in your environment and you have made custom modifications, leave Overwrite partners/channels disabled to preserve your configuration.

Complete the import

Review the displayed components across all tabs. Select Import and confirm to start the import.

Step 2: Configure credentials

Required credentials per carrier

Partner channel configuration

Navigate to Administration > Partner > Partners/Channels > CEP Connector.

DHL_Express_HTTPS

• Enter the DHL Express username in the Own ID field.

• Enter the DHL Express password in the Own Password field.

IMPORTANT

The partner address in the package points to the test environment by default. Before deploying to production, update this URL to the production endpoint.

FedEx_HTTPS

1. Select Configure OAuth 2.0….

2. Enter your client ID and client secret.

3. Select Client Credentials as the grant type.

4. Enter the Endpoint URL:

   • Test: https://apis-sandbox.fedex.com/oauth/token

   • Prod: https://apis.fedex.com/oauth/token

5. Select Fetch Access Token.

6. Enter the OAuth2 Refresh URL:

   • Test: https://apis-sandbox.fedex.com/oauth/token?grant_type=client_credentials&client_id=<clientId>&client_secret=<clientSecret>

   • Prod: https://apis.fedex.com/oauth/token?grant_type=client_credentials&client_id=<clientId>&client_secret=<clientSecret>

UPS_HTTPS

1. Select Configure OAuth 2.0….

2. Enter your client ID and client secret.

3. Select Send 'client credentials' in header.

4. Select Authorization Code as the grant type.

5. Enter the Endpoint URL:

   • Test: https://wwwcie.ups.com/security/v1/oauth/token

   • Prod: https://onlinetools.ups.com/security/v1/oauth/token

6. Enter the Redirect URL. Use the internal address of your Lobster Data Platform:

   https://<your-server>:<your-port>/dw/oauth2/<your-Application-ID>

   The Application ID starts with _data. You can find it in your OAuth 2.0 settings. Register this URL in the UPS Developer Portal application settings as well.

7. Enter the OAuth URL:

   • Test: https://wwwcie.ups.com/security/v1/oauth/authorize

   • Prod: https://onlinetools.ups.com/security/v1/oauth/authorize

8. Select Fetch Access Token. You are redirected to UPS to sign in with your developer account credentials.

9. After a successful redirect, a blank page with ok is displayed. Reopen the channel. The tokens SYS_HTTP_OAUTH2 and SYS_HTTP_OAUTH2_REFRESH are now visible under Additional IDs.

10. Enter the OAuth2 Refresh URL:

    • Test: https://wwwcie.ups.com/security/v1/oauth/refresh?grant_type=refresh_token&refresh_token=<refreshToken>&client_id=<clientId>&client_secret=<clientSecret>&useCredentialsInHeader

    • Prod: https://onlinetools.ups.com/security/v1/oauth/refresh?grant_type=refresh_token&refresh_token=<refreshToken>&client_id=<clientId>&client_secret=<clientSecret>&useCredentialsInHeader

PostNL_HTTPS

• Add your API key as SYS_HTTP_apikey in the Additional IDs tab.

Step 3: Use profile templates for custom mappings

IMPORTANT

Do not modify CEP_CONNECTOR profiles. Modifications break the functionality and maintainability of the CEP Connector.

Configuration constants

The following system constants configure the CEP Connector. (M) = required, (D) = dependent, (O) = optional.

Input data structure: Shipment booking

Pass the following LobsterCEP data structure to the profile CEP_CONNECTOR_ShipmentTransportorder. The connector MAPS it to the carrier format, calls the carrier API, and forwards the result to your acknowledgement profile.

Output data structure: Shipment booking

Example: Confirmed booking

{
  "LobsterCEP": {
    "header": {
      "carrierId": "DHL_EXPRESS",
      "processingStatus": {
        "code": "CONFIRMED",
        "realizationDateTime": "2025-08-13T09:33:59+02:00"
      }
    },
    "shipment": {
      "shipmentId": "ABC123456789",
      "carrierReference": "987654321",
      "estimatedDeliveryDate": "2025-08-18T23:59:00Z",
      "documents": [
        {
          "type": "LABEL",
          "name": "LABEL_987654321.pdf",
          "format": "PDF",
          "content": "<base64-encoded content>"
        }
      ],
      "trackingUrl": "https://express.api.dhl.com/mydhlapi/test/shipments/987654321/tracking"
    }
  }
}

Example: Error response

{
  "LobsterCEP": {
    "header": {
      "carrierId": "DHL_EXPRESS",
      "processingStatus": {
        "code": "ERROR",
        "reason": "1001: The requested product(s) (P) not available based on your search criteria.",
        "realizationDateTime": "2025-08-13T09:33:59+02:00"
      }
    },
    "shipment": {
      "shipmentId": "ABC123456789"
    }
  }
}

Output data structure: Tracking

The profile CEP_CONNECTOR_ShipmentTrackingTrigger polls the carrier automatically for new events. It delivers the following LobsterCEP structure to your tracking target profile. No call is required from your side. Configure the tracking CSV files as described in the Tracking section.

Example: Tracking

{
  "LobsterCEP": {
    "header": { "carrierId": "DHL_EXPRESS" },
    "shipment": [
      {
        "shipmentId": "ABC123456789",
        "carrierReference": "987654321",
        "estimatedDeliveryDate": "2025-08-18T23:59:00Z",
        "trackingEvents": [
          {
            "owner": "DHL_EXPRESS",
            "eventDateTime": "2025-09-01T01:43:03Z",
            "eventLocation": "CPH (Copenhagen-DK)",
            "statusDetails": [{ "code": "OK", "statusDescription": "Delivered" }]
          }
        ],
        "documents": [
          {
            "type": "POD",
            "name": "POD_987654321.pdf",
            "format": "PDF",
            "content": "<base64-encoded content>"
          }
        ],
        "trackingUrl": "https://express.api.dhl.com/mydhlapi/test/shipments/987654321/tracking"
      }
    ]
  }
}

LobsterCEP data format

The LobsterCEP object is the unified data format for all CEP Connector operations. The following sections provide a complete field reference.

Header

Contains carrier selection, action type, and document requirements.

Shipment

Contains all shipment details.

serviceType values by carrier

Parties and addresses

All parties involved in the shipment. shipper and recipient are required.

Address fields

Payment information

Packages and line items

Package fields

Dimensions

All measurement values have a value (number) and unit field.

Line item fields

Customs information

Available at shipment level, package level, and line item level.

Documents

Documents are returned in the booking acknowledgement and tracking responses.

Tracking response

Tracking event fields

Support

For questions about DataFlow configuration, contact the Lobster Support team.

• Website: www.lobster-world.com

• E-mail: support@lobster.cloud