Forwarding HTTP/FTP/SSH/OFTP inwards

Inwards forwarding for HTTP(S) requests

In order to, for example, send data to a profile via HTTP, a dedicated URL needs to be used. The schematic structure of the URL is:

Now, if a DMZ server is in use, you want requests of this type to go to the DMZ server and be forwarded from there to the internal Integration Server (forwarding).

Note: In general, please consider which paths you want to make accessible via the DMZ server and only set up those that you really need.

Standard context on DMZ server

By default, the DMZ server reacts to HTTP requests whose URL path begins with the URL context "/forward". This default can be overridden in configuration file "./etc/startup_dmz.xml" (on the DMZ server) using the parameters "servletContext" and "servletPath" (see table in section Parameter‌s). In the following, we assume that the default settings are used, so "servletContext=/forward and servletPath=/*".

Forwarding rules on DMZ server

Even if the DMZ server responds to the URL context "/forward" by default, forwarding rules must still be defined there, i.e. how such requests are forwarded, as the DMZ server itself does not respond to the requests. These forwarding rules are defined in the configuration file "./etc/forward.properties" (on the DMZ server).

If the DMZ server has multiple HTTP listeners (port 80, 443, and so on) defined in configuration file "./etc/hub.xml", all requests that arrive via one of the HTTP listeners and correspond to a forwarding rule are routed according to the rule.

• Each line of this file represents an independent rule. The left side (source context), to the left of the equals sign, must be unique

• Changes to the configuration file "./etc/forward.properties" are recognised during runtime and are reevaluated. A restart is not necessary.

• For diagnostic purposes, the HTTP request log and the server log can be evaluated under "./logs". Caution: The times in the request log are in the time zone UTC, the times in the server log are in the system time zone.

Let us now assume the following forwarding rule and that the inner Integration Server has the IP shown.

...
/forward/*=http://192.168.213.12:80
...

In this case, all requests to the DMZ server that begin with the path "/forward" are forwarded to the inner Integration Server. All other paths are not forwarded.

The request "http://<URL/IP and port DMZ server>/forward/dw/Request/<URL suffix of the profile>" would therefore be forwarded to "http://192.168.213.12:80/dw/Request/<URL suffix of the profile>".

If you do not want to forward all requests, you can also make it a little more specific.

...
/forward/dw/Request/*=http://192.168.213.12:80/dw/Request
...

Or even more specifically with the following rule. This could be used to set the URL suffixes of HTTP profiles so that only some are accessible from outside via the DMZ server. An HTTP profile with the URL suffix "outside/example1" would then be accessible via the DMZ server and an HTTP profile with the URL suffix "example2" would only be accessible via the internal Integration Server.

...
/forward/dw/Request/outside/*=http://192.168.213.12:80/dw/Request/outside
...

Defining further contexts on DMZ server

In order for the DMZ server to respond to other contexts to which the inner Integration Server normally responds (and not only to "/forward"), the parameter "addStandardServlets" in configuration file "./etc/startup_dmz.xml" (on the DMZ server) must be set to "true" for the CommunicationForwardManager.

<Call name="addApplication">
    <Arg>
        <New class="com.ebd.hub.datawizard.app.CommunicationForwardManager">
 ... 
            <!-- Add /dw/Trigger, /dw/Request (also in lowercase) and /partner servlets on demand -->
            <Call name="addStandardServlets"><Arg type="boolean">true</Arg></Call>
...
        </New>
    </Arg>
</Call>

The following contexts are activated on the DMZ server with this.

Here too, corresponding forwarding rules must first be defined so that forwarding takes place. Important note: Please note that this is case-sensitive and please define both variants "Request"/"request" and "Trigger"/"trigger".

...
/dw/request/*=http://<URL/IP and port inner Integration Server>/dw/request
/dw/Request/*=http://<URL/IP and port inner Integration Server>/dw/Request
/dw/trigger/*=http://<URL/IP and port inner Integration Server>/dw/trigger
/dw/Trigger/*=http://<URL/IP and port inner Integration Server>/dw/Trigger
/dw/oauth2/*=http://<URL/IP and port inner Integration Server>/dw/oauth2
/dw/cloud/*=http://<URL/IP and port inner Integration Server>/dw/cloud
/dw/register/oauth2/*=http://<URL/IP and port inner Integration Server>/dw/register/oauth2
/partner/AS2Retrieve/*=http://<URL/IP and port inner Integration Server>/partner/AS2Retrieve
...

Note: If an HTTPS listener is active on the inner system, forwarding can also take place via HTTPS. However, as the connection from the DMZ server to the inner Integration Server takes place in a protected network section, a second encryption via HTTPS is normally not necessary there.

AS2 handling

AS2 requests can either be handled on the DMZ server or forwarded to the inner Integration Server. A combination of both variants is also possible.

AS2 on DMZ server

• Parameter "handleAS2" in configuration file "./etc/startup_dmz.xml" (on DMZ server) is set to "true".

• AS2-Service on DMZ server is active.

AS2 requests are forwarded by Message to the inner Integration Server. If the inner Integration Server is (temporarily) unavailable, the messages are buffered (as with FTP, etc.).

AS2 forwarding

• Parameter "handleAS2" in configuration file "./etc/startup_dmz.xml" (on DMZ server) is set to "false".

• Forwarding for context "/partner/AS2Retrieve" is configured (see instructions above).

All AS2 HTTP requests are forwarded to the inner Integration Server and the inner Integration Server sends MDNs and messages.

AS2 on DMZ server and forwarding

Here you set up AS2 on the DMZ server (see instructions above), but also define a forwarding via the default context "/forward" (see instructions above).

AS2 requests via the path "/partner/AS2Retrieve/" are then processed on the DMZ server and AS2 requests via the path "/forward/AS2Retrieve/" are forwarded to the inner Integration Server.

AS2 request timeout

When an AS2 message is received by the DMZ server, it forwards the request to the Lobster Data Platform asynchronously and holds the HTTP connection open until the platform responds. Previously, if the platform took too long, the DMZ server would silently hold the connection until the platform’s own read timeout expired, producing a hard TCP/Socket timeout with no meaningful response. This prevented a clean AS2 retry logic, since retries typically require a proper HTTP error response rather than a connection drop.

The DMZ server now supports a configurable timeout via the system property hub.proxy.requestTimeout

If the property is set and the platform has not responded within the configured window, the DMZ server actively cancels the pending request and returns the HTTP response status code 503 (Service Unavailable) to the AS2 sender. This gives the sender a well-formed HTTP response it can act on, specifically, triggering its AS2 retry logic. Without the property, the behavior is unchanged, meaning the DMZ server waits indefinitely (hub.proxy.requestTimeout= -1).

 AS2 sender → DMZ server → Lobster Data Platform                                                                                
                  ↑                                                                                                  
       hub.proxy.requestTimeout expires                                                                           
                  ↓                                                                                                  
       DMZ server sends HTTP response status code 503                                                                                     
                  ↓                                                                                                  
       AS2 sender receives 503 → triggers retry

Recommended values:

The DMZ server request timeout must be shorter than the Lobster Data Platform’s read timeout, otherwise the platform drops the connection before the DMZ server can send status code 503:

hub.proxy.requestTimeout  <  hub.as2.readTimeout

e.g. 300000 ms (5 min)   <  360000 ms (6 min)      

Setting up HTTPS (DMZ server)

You can use HTTPS for the communication between the outside client and the DMZ server.

See section Adding an HTTPS listener.

Tunneling HTTP(S) (from DMZ server to inner system)

It is possible to tunnel HTTP(S) requests via the MessageService. To do so, insert the following option into configuration file ./etc/startup_dmz.xml (on DMZ server). The forwarding rules must also be set in this case (see above).

<Set name="tunnelHttp">true</Set>

Inwards forwarding for FTP, SSH, OFTP‌

The FTP server can be configured to exclude users or single directories of a user from forwarding to the inner Integration Server. In that case, the files will remain in the FTP directory. The administrator is then responsible for copying or deleting them.

File ./etc/startup_dmz.xml contains the path to the configuration file that defines for which FTP/SSH/OFTP users or from what directories files are not forwarded to the inner Integration Server.

<Set name="noForwardRulesForFtp">./etc/admin/datawizard/ftp_user_rules.properties</Set>    

Following an example for file ftp_user_rules.properties:

# No forwarding for user dummy1 at all
dummy1=
# No forwarding for user dummy2 when upload is done in folder tmp and output
# (relative subdirs under home dir of user)
dummy2=tmp;output

If only the name of an FTP/SSH/OFTP user is set, all files this user places on the DMZ server are left in the directories. With the additional specification of one or more subdirectory names, the files in the specified subdirectories are not forwarded to the inner Integration Server. The subdirectory names are separated by semicolons.

OFTP on the DMZ server

It is possible to run the OFTP service on the DMZ server and address this service from the inner Integration Server. In order to do this, the inner Integration Server server needs to establish the connection to the DMZ server. The following settings need to be applied.

1. When OFTP channels are created, they must be made known to the DMZ server. In addition, the associated OFTP home directories must be created (for incoming and outgoing data) on the DMZ server. The following listing shows the entry in the configuration file ./etc/startup.xml of the inner Integration Server in section DataWizard.

   ...
   <!-- Create OFTP, SSH & FTP user homes in DMZ as well & enable OFTP via DMZ, etc -->
   ...
   <Set name="useOFtpUserOfDmz">true</Set>
   ...

   Note: The same is true for FTP (useFtpUserOfDmz) and SSH (useSshUserOfDmz).

2. ISDN: The DMZ server may need to know the ISDN number the OFTP service should use. The following listing shows the entry in configuration file ./etc/startup_dmz.xml of the DMZ server in section CommunicationForwardManager.

   ...
   <Set name="isdnNumber">your_ISDN_number</Set>
   ...

DMZ hybrid mode

It is possible to put the DMZ server into a special mode in which no requests to the DMZ server are forwarded to the inner system, i.e. FTP uploads, for example, remain in the directory of the FTP user, HTTP requests are responded to with status 404, etc.

However, all other functions (e.g. status queries for the dashboard, jobs from inner Integration Server to the DMZ server (sending/fetching data), etc.) are executed as usual.

Thus, for FTP, for example, the DMZ server acts like any other FTP server, since there is no event-based handling of files.

To do this, add the following section to the configuration file ./etc/startup_dmz.xml (inside the CommunicationForwardManager).

<Set name="noForwardEvents">true</Set>

Important note: If the AS2 service is installed and started on the DMZ server with option "handleAS2" in configuration file ./etc/startup_dmz.xml, i.e. not with HTTP forwarding, but with direct handling, AS2 events are still passed inwards.