Certificates

See also: Adding an HTTPS listener, HTTPS client

In Lobster Integration, you can use certificates for encrypted communication in various places. Here you can manage all certificates centrally.

Terms and know-how

The terms in this section refer to a so-called public-key infrastructure, especially to the X.509 standard of the International Telecommunications Union (ITU-T), as well as related standards. We assume that you are familiar with secure data transmission terms like asymmetric encryption, public key, private key, certificate, certification authority (CA), fingerprint, signature, and related terms. The purpose of this documentation is not to explain these terms and their relationships in detail. There are many freely accessible documents on this subject.

A digital certificate always refers to an asymmetric key pair with a private and a public key. It contains structured data that establish a link between the technical key and the identity of the legal owner (person, organization, company, or IT system) of that key. Certificates always contain a validity period and an address of the owner. An important part of this address is the common name (CN).

In the following, we call the combination of a digital certificate and one or both keys a certificate object or simply a certificate.

Partner certificates

We call a certificate object that contains the digital certificate and only the public key, but not the private key, a partner certificate. The two terms partner certificate and public key are often used synonymously.

Local certificates

We call a certificate object that contains the digital certificate and both keys (public and private) a local certificate.

Certificate serial number

For each local certificate (private and public key), a partner certificate (with public key only) must usually be installed on the partner system with which you want to communicate. To be able to unambiguously assign both to each other, a certificate serial number exists. In Lobster Integration, the serial number is displayed as a decimal integer.

If the certificate has been signed by an official certification authority, the validity of the certificate can be checked via the internet. There is also a private and a public key in these cases. The partner certificate (not your local certificate with your private key!) can be exchanged over the net because it is protected against manipulation by the signing. If you e-mail the partner certificate to the partner, they can use the fingerprint or the checksum to ensure that it has not been modified during the transport.

Objectives of the X.509 technology

The X.509 technology allows the encryption of data, as well as signing with a digital signature. The generation of the key pair and assignment of descriptive certificate information is a closed, one-time process. Subsequent modification of the keys or the descriptive information is impossible. Data encrypted with one key can only be recovered (decrypted) with the other key.

One key (private) is known only to the holder of the certificate. The other key (public) is intended for forwarding to communication partners. As long as the holder of the certificate ensures that his private key never gets into other hands, it is ensured that the encrypted data cannot be manipulated. Manipulated data can no longer be decrypted. An interception of the data, decrypting, manipulating, and re-encrypting fails because no one but the certificate holder has both keys. This is a crucial condition.

Therefore, digital certificates protect the confidentiality, authenticity, and integrity of the data, especially during transport over the network.

Encryption

To transfer data securely to a partner, this data is encrypted using the partner's public key (that is, the partner certificate).

Only the partner can decrypt this data with the private key, which only they possess. If the partner wants to send us encrypted data, they have to encrypt it with our public key so that only we can read it with our private key (our local certificate).

The partner must be in possession of the public key of our local certificate. In turn, we must have their public key. They may have received our public key as an exported certificate from us, e.g., by e-mail, or they may have received it online from us during the connection setup. So we need to import the partner certificate we receive from the partner, and the partners must install our exported local certificate on their side as a partner certificate.

In the case of a secure web server, the clients (which the server does not know beforehand) receive the certificate and the public key during the connection setup during the SSL handshake.

Signature

Signing the data is basically an additional encryption. When you send the data to a partner and encrypt it, the data is first encrypted with the partner's public key (encryption). Then a hash (a kind of checksum) is formed over the data, and this hash is then encrypted again (signing) with the private key of the local certificate.

First, the partner decrypts the signature with the public key of your certificate, checks the checksum, and, if it is correct, the data is decrypted with the private key of their local certificate.

If we only send the data signed, it is sent in plaintext, because the signing is only an encryption of the hash with the local (private) key, but no encryption of the data. The signing only proves that we have sent the data, because only we as owners of the private key can encrypt the checksum correctly.

Exchanging a partner certificate

If certificates are to be used, please exchange only the public part with the respective partner. Please never hand out the private part, and keep in mind that certificates are usually valid for a limited period of time. After an expired validity period, the certificate becomes invalid, and thus no communication is possible.

A local certificate that has been exported from Lobster Integration for one of your partners is a zipped .CER file and contains only the public key.

If a partner sends you a certificate (only with his public key), Lobster Integration supports the following X.509 certificate formats.

• .CER - CER-encoded certificate or certificate sequence.

• .CRT - DER-encoded or Base64-encoded certificate.

• .DER - DER-encoded certificate.

• .PEM - Base64-encoded certificate.

• The formats .P12 and .PFX (PKCS#12) can be imported as a local certificate if they are error-free and contain the private key.  CAUTION  If the password is not correct, the (misleading) error message will show that no certificate is included.

• The format .P7B (PKCS#7) can be imported as a partner certificate if it is error-free.

   CAUTION  If the password is not correct, the (misleading) error message will show that no certificate is included. Only exception: If you create a CSR (Certificate Signing Request) from your certificate and have it signed by the Certification Authority (CA), you may get the certificate response back as .P7B or .P7C, which cannot be added as a certificate response to the current certificate. But you can open the .P7B file on Windows with the crypto shell extension and save all components of the certificate chain (typically 3 - own, intermediate, and CA root) as Base64-encoded .PEM or .CER files, and then copy all three file contents in reverse order (CA root last) into a .CER file. This file can then be, as usual, added as a certificate response to the existing certificate from which the CSR was created.

Notes on PEM format

The PEM format is a Base64 string between -----BEGIN CERTIFICATE----- and -----END CERTIFICATE-----.

NOTE If there is a section beginning with -----BEGIN RSA PRIVATE KEY----- and ending with -----END RSA PRIVATE KEY----- in the PEM format, this would be the private key, which must never be passed on.

Self-signed or certified by a Certificate Authority?

To be able to use a certificate for an SSL-based browser application, for example, where many anonymous clients are communicating with a server, the certificate should be authenticated by an official Certificate Authority (CA). This authentication is a confirmation from an independent, trustworthy instance that the server uses this certificate legitimately.

In business-to-business (B2B) communication, self-signed certificates can be used because a third instance is not necessary between two business partners to certify the authenticity.

Each certificate generated here in Local certificates is a self-signed certificate that is not certified by a certification authority. However, it can in principle be authenticated. To do this, you have to create a Certificate Signing Request (CSR) from your local certificate and send this CSR file to a certification authority. To do this, open the certificate with a double click (1).

Then click button Create CSR (2). In a further dialog, you can then copy the CSR (Certificate Signing Request Response) request and send it to the certification authority.

The Certificate Response of the Certificate Authority must then be imported into the self-signed certificate with the button Import CSR (3). Such a certification by a public certification authority must usually be paid for and is in most cases not required when using Lobster Integration.

The order of the individual components of the CSR is as follows:

• CSR Response

• Issuer

• Root

If you only have those components at hand separately, please merge them into one file in an editor in the above mentioned order.

Java Cryptography Extension

Lobster Integration requires encryption and signing cryptography of unlimited strength (i.e. keys of unlimited length). Because of US export restrictions, extensions must be installed for your Java version to work with unlimited key lengths.

For Java JDK 1.6 this is the Java(TM) Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files 6. If the JCE is missing, the start of the Integration Server aborts with the following error message:

'The Java Virtual Machine can't handle strong cryptography'.

 CAUTION  Please download the file for the Java version only from Oracle.

NOTE After upgrading the Java version, the appropriate extension must be downloaded and installed again.

Local certificates

Here you can create, import, export, view, and revoke your local certificates. A Certificate Signing Request (CSR) can also be generated. In addition, a CSV report can be created via the context menu, listing the channels in which a certificate is used. The ID of the certificate and the Common Name (CN) can be found via (1). The ID should not be confused with the serial number.

(1) and (2): Certificates are automatically invalidated after the expiration of their validity period. If there is reason to believe that this certificate is no longer safe because it has been compromised, it can be revoked through (1) and the Revoke certificate (2) button, (if you have admin rights). It will not be deleted, because that would mean the data encrypted or signed with this certificate would no longer be readable.

IMPORTANT Revocation in Lobster Integration is merely a local deactivation of the certificate within the platform. No official entry is made in a Certificate Revocation List (CRL). This means:

• A locally deactivated certificate can be reactivated at any time via (1). Locally deactivated certificates are hidden by default in the overview. To make them visible, the filter must first be adjusted accordingly so that revoked certificates are also displayed.

• If the certificate has also been listed as revoked on an official CRL from a certification authority, local reactivation is technically possible; however, the certificate will continue to be rejected by communication partners who check the CRL.

(3) Usage (Signing, Encryption, TLS Client, TLS Server): Sets the usage flags, i.e. what the certificate can be used for.

(4) Usage (Signing/encryption must be supported, TLS usage must be supported): With these checkboxes, the so-called 'critical flags' can be set for the settings in (3). They can be used to request that your communication partner, when importing the certificate, checks whether they support the settings made in (3). If they do not, they should discard the certificate. We recommend using the default settings shown here in (4) to avoid the certificate being rejected. These settings are correct for almost all cases. Only change them if you really know what you are doing.

(5) Import certificate/SSH key: See the 'Import local certificate' and 'Import SSH key pair as a local certificate' sections below.

Let's Encrypt

See the 'Let's Encrypt/ACME/Certbot' section  below (receiving and renewing certificates from the Certificate Authority free of charge and automatically).

Creating local certificates

The following screenshot shows the dialog for creating self-signed certificates in the X.509 standard that appears after pressing the plus (+) symbol button on the bottom right (1) of the screen. A description of the fields can be found in the X.509 standard.

When creating a local certificate for OFTP2, an ODETTE ID can also be assigned as a property.

NOTE Use the value 'Internal' for the 'Key material' option. This means it is a platform certificate and not from a vault.

Exporting a local certificate

A local certificate can be exported via the context menu. Here you can download either only the certificate, only the private key, or the certificate and the private key in several formats.

IMPORTANT

• If you want to send the certificate to a partner in an e-mail, for example, make sure that you never export and send the private key! If you do, your certificate will be compromised! Where possible, use the automatic Certificate exchange.

• It is recommended that you create a secure copy of your local certificates and keep them safe to be able to restore them after a system failure. In principle, it is not possible to create the matching private key for an existing public key after the private key has been lost!

Importing a local certificate

A local certificate can only be imported if it has previously been fully exported, including the private key. Possible situations are the transfer of the certificate from the test system to the productive or standby system, the recovery of the local certificate after a system failure, or the import of a key from OpenSSH.

IMPORTANT You will never receive a local certificate from a communication partner. If so, it would be compromised, and it must not be used. The following screenshot shows the dialog for importing a local certificate. The certificate is either copied into the text field in the PEM format or dragged directly into the dialog as a CER file. If necessary, a password can also be entered during the import.

Importing SSH key pair as a local certificate

An OpenSSH key is not a complete certificate. The assignment of an address to the key is missing. Since the partner administration cannot deal with individual key pairs but only with certificates, which also contain a name or an address, a name must be assigned when importing an SSH key. This name must be entered in the SSH Common Name (1) input field (when importing as a local certificate). A certificate is then generated for the SSH key, and the SSH Common Name is entered as a common name (CN=...) in this certificate.

Let's Encrypt/ACME/Certbot

To be able to provide access to Lobster Integration via HTTPS, you need a certificate. See also: Adding an HTTPS listener.

Let’s Encrypt (https://letsencrypt.org/docs/) is a free certificate authority (CA) through which Lobster Integration can automatically obtain such certificates. These are then also automatically renewed when they expire.

Lobster Integration uses the ACME protocol and the Certbot client internally for this purpose. The corresponding components are included by default and only need to be configured as described below.

Requirements

• The CertificateExchangeService must be active (./etc/factory.xml, see there).

   

• Configuration file ./etc/cex.xml must be adjusted (see below).

• Lobster Integration must be accessible from the outside (port 80 and 443 must be open).

• forceSSL must not be active, otherwise no certificate can be generated! Two files must be adjusted for this purpose.

  • ./etc/webdefault.xml - Surround this part in comments as shown.

     

  • ./etc/startup.xml - Set the parameter to 'false'.

     

Adjusting configuration file ./etc/cex.xml

With an installation from Lobster Integration 4.5 onward, there is already a prepared block for adding Certbot handlers in the configuration file. Change the tags NoCall to Call for 'activation' (do not forget the identical change for the closing tag, otherwise the system start will end with an error).

IMPORTANT You can copy the template below, but please ensure that the lines mentioned below have been adjusted. Otherwise there could be authentication problems!

The following parameters must be adjusted:

Optional:

<?xml version="1.0"  encoding="ISO-8859-1"?>
<!DOCTYPE Configure PUBLIC
 "-//Lobster//DTD Configure 1.0//EN"
 "http://www.lobster.de/dtd/configure_1_3.dtd">
<Configure class="com.ebd.hub.services.certexchange.CertificateExchangeService">
 
    <!-- Name of the DB alias to be used for internal data -->
    <Set name="dBAlias">hub</Set>
       <Set name="mailSenderAddress">mail@example.com</Set> 
    <!--
        Certbot Handler configuration for the management of certificates
        that should be signed by an ACME supporting CA, e.g. Let's Encrypt
    -->
    <!-- Change NoCall to Call (don't forget the </NoCall> to enable this -->
    <Call name="addCertbotHandler">
        <Arg><New class="com.ebd.hub.services.certexchange.CertbotHandler">
            <!--
                Name of the handler. Certificates handled by this handler
                will have the name in the keystore notes and can be used
                as search term in TLS-configurations
            -->
            <Set name="handlerName">example.com</Set>
            <!--
                Used as contact for ACME-server account registrations
            -->
            <!--
                The URL of the ACME endpoint. The URL in this sample
                is the staging server, the URL of the production
                server is acme://letsencrypt.org
            -->
            <Set name="certbotURL">acme://letsencrypt.org/</Set>
            <Set name="accountContact">mailto:mail@example.com</Set>
            <!--
                Adds a certificate algorithm to be handled. This will
                lead to the creation of a keypair and finally a
                CA-signed certificate.
            -->
            <Call name="addCertificateAlgorithm">
                <!-- the non-EC algorithm -->
                <Arg>RSA</Arg>
                <!-- key size in bits -->
                <Arg type="int">2048</Arg>
            </Call>
            <Call name="addCertificateAlgorithm">
                <!-- the EC algorithm -->
                <Arg>ECDSA</Arg>
                <!-- the curve name -->
                <Arg type="String">secp256r1</Arg>
            </Call>
            <!--
                Adds a domain the certificate should contain as common name
                or alternate subject name.
            -->
            <Call name="addDomain">
                <Arg>example.com</Arg>
            </Call>           
            <!--
                Number of days a certificate should be valid. This is part
                of the request to the CA and its support is CA-dependent. It
                might be ignored or denied, leading to an error. In that case
                setting it to 0 will omit adding it to the request
            -->
            <Set name="requestedCertificateValidityDays">0</Set>
            <!--
                (change NoSet to Set to activate this setting)
                Number of days before expiry a renewal request should be
                sent to the CA
            -->
            <Set name="daysBeforeExpireRefresh">10</Set>
        </New></Arg>
    </Call>
 
</Configure>

Enabling HTTPS listener/Specifying a certificate

If the Integration Server has been restarted, the newly generated certificates are displayed in the certificate overview (local certificates) after a few minutes. These are marked with a note, the handler name.

To finally use the new Let's Encrypt certificate, you must activate the HTTPS listener and adjust the configuration file ./etc/hub.xml. See the following example:

IMPORTANT

Both the CN (Common Name) and the ksnote prefix can be used. The latter acts as a kind of alias and uses the certificate note as a unique reference.

Usage on a DMZ server

Since a DMZ server is in most cases the interface to the outside or from the outside to the inside, Certbot must be installed there. In addition to the standard changes to the ./etc/factory.xml, ./etc/cex.xml, and ./etc/hub.xml configuration files (as described above), the following must be adjusted.

• ./etc/cex.xml: The parameter dBAlias must have the value: dmzcommauth.

• ./etc/auth_dmz.xml: The parameter <Set name="localDbFile">dmz/hsql/cacheHsql</Set> must not be active and must be commented out.

Error handling

If errors occur, they can be found in the following logs.

• ./logs/console.txt

  If Lobster Integration no longer starts, this is usually due to faulty configuration files. Please check the configuration files ./etc/cex.xml, ./etc/hub.xml and ./etc/factory.xml.

• ./logs/services/message.log

  To narrow down the results, you can search for CERTBOTHANDLER. This will give you a rough overview of whether a CSR (Certificate Signing Request) was successful.

• ./logs/services/error.log

  If the authentication fails, this can be seen here.

Partner certificates

The ID of the certificate and the Common Name (CN) can be found by double clicking on the certificate. The ID should not be confused with the serial number.

A partner certificate contains only the public key of your communication partner and the certificate that assigns the information on the validity period and the owner to the key. Usually, you receive a partner certificate from your partner, e.g. by e-mail either in .pem format or as a file.

Partner certificates can be managed (imported, viewed and revoked) here. NOTE Partner certificates cannot be created. Your partner has to export their certificate together with their public key and make it available to you. Each partner certificate is assigned to a specific partner and can only be used for its channels. Since partner certificates contain only the public key, but no private key, they can be passed on without hesitation. However, it is not possible to import a partner certificate as a local certificate.

Importing partner certificates

First, click the Import certificate/SSH key button (1). Then you can either copy the contents of a certificate file in .pem format directly into the window (as shown in the screenshot) or, alternatively, drag and drop a file in .cer format into the window.

Authentication by client certificate

As a precondition, an HTTPS listener must be defined in the configuration file ./etc/hub.xml.

The protocols HTTPS, AS2 over HTTPS, and OFTP2.0 over TLS use the SSL or TLS protocol on the transport layer to establish a secure channel through encryption. Usernames and passwords sent within this channel are much better protected against an interception than they would be if sent via a 'clear text' channel (e.g. HTTP). However, this security may not be sufficient for a communication partner and they might, therefore, either in addition to or instead of the password, require the client to be identified with a certificate. This certificate is called a client certificate because it proves the identity of the client, not the server, or the service. It is a local certificate on the client side. This means that only the client can access the private key.

Lobster Integration supports client certificates for HTTPS, AS2, and OFTP if the communication partner requests an authentication with a client certificate. You can also request that the partner needs to log on with its client certificate. See the 'Local Certificates' section.

A special case is OFTP via TLS. The specification demands a mutual identification of the partners via a certificate. Therefore, this certificate is both a server and client certificate simultaneously, depending on the direction of the connection setup. For outbound connections, it is the client certificate.

For all outgoing connections of the protocols AS2 via HTTPS, OFTP via TLS and HTTPS, the client certificate used is the local certificate assigned to the partner channel. So it is the same certificate that is used to sign messages/files. With AS2, separate certificates can be used for encryption and signing.

An authentication by client certificate is only possible through a partner channel. A response without a partner channel cannot be used for this.

See also: HTTPS client.

Vault certificates

You can use a vault provider to store your certificates (local certificates and partner certificates) and use them in your Lobster Data Platform.

Azure

(1) Select value 'External'. NOTE This means it is a vault certificate and not a platform certificate.

(2) Choose your vault provider alias.

(3) Choose the certificate alias.

(4) Those fields are read only and are set automatically.

HashiCorp

HashiCorp works the same way, but you have to set two additional fields. Mount (1) is the path where a secrets engine is enabled. Think of it as the 'root namespace' or 'drive letter' for a category of secrets. Name (2) (sometimes called the path within a mount) is the specific identifier for an individual secret within the mount. See details in your HashiCorp documentation.