_data release 4.6.9

IMPORTANT

• new internal update service to provide _data prereleases and updates for _pro as well (see DATA-38750). Please make sure your firewall does not prevent access to: https://api.lobster-world.com

• all TestCases must be recorded again (by Bulk operation in GUI) due to a bugfix in the internal structure (see DATA-40900).

• CloudStorage: with this _data version CloudStorage "Legacy Connections" were moved to the new implementation. In this context, "Box" will no longer be supported. Please also note that CloudStorage channels of type DropBox, which are still running with the old implementation, must be re-registered via OAuth2 after this update.

• failover-logging was redesigned. There is no longer a failover.log. It is split into message.log and error.log.

Web app "api" (Swagger display) renamed to "openapi" due to "One Lobster Platform" and preventing url conflict with Lobster_pro.

DATA-40852

Use longer client secret for OAuth2 server

DATA-40725

Bumped version of logback to 1.2.13 to fix CVE-2023-6378 CVE-2023-6481 "just to make sure" - _data's standard configuration doesn't use the vulnerable functionality

DATA-40800

DataWizardHttpAgent: Copy Authorization Header in Global Variable only if System-Property is defined

DATA-40970

CloudStorage: Google Cloud Storage upload is not working correctly

DATA-40989

Uncaught NullPointerException while trying to add SSO

DATA-41028

CertificateExchangeService: Prevent start error if no SmtpService is configured

DATA-40949

Text editor: possibility to format file contents again after changing encoding

DATA-40862

Users: newly created admin user should not be forced to change password

DATA-40930

Prevent loss of profiles in special cases while performing a suspend-operation

DATA-40900

Test Case fails when path of a node is a variable with multiple iterations

DATA-40888

LicenseCheck: fixed bug leading to a license error if format restrictions are in place and XML is licensed

DATA-40745

System Monitor: Fixed considering ssh connections tests as failed all the time

DATA-40827

Fixed failing SCP-transfers if a particular way of naming the uploaded file is used by the client

DATA-40322

AwsS3Storage: fixed removing of temp files

DATA-40494

Kafka: Add missing translations in Phase 1

DATA-39788

MQTT: fixed setting debug flag

DATA-41162

ETL: If destination Template is selected not using template engine (text), the footer is not placed correctly

DATA-41157

ETL: Namespaces are not resolved for Preview if no XPath Variable extraction was done before (init bug)

DATA-41154

ETL: VTD preview / test XPath generates different result for extracting XPath ETL variables than expected when executing XPath on document

DATA-41059

ETL: Fix auto creation of XPath and JSONPath to guaranteed valid expressions

DATA-40737

ETL: Pipeline import disrupts exported format of prefix, body and suffix for Template as Destination data

DATA-40198

ETL: Export Pipeline with Log level "Error & info & debug" returns Error

DATA-38045

ETL: XML whose elements are in one line are not resolved correctly

DATA-40104

ETL: VelocityConnection support variables in Footer

DATA-41020

ETL: Emergency Halt doesn't pause ETL cron jobs

DATA-40759

Cloud Storage: Azure File Storage: missing slash separator between path and file name

DATA-40770

Profile: creating backup when profile is deleted

DATA-40802

Phase 1 (Cron:HTTP): missing elements in soap envelope header request after import

DATA-41086

Phase 1 (Cron:MAIL): save column visibility user preference for all columns in rule grid

DATA-40944

Phase 1 (CrossSQLCron): selection of columns for selects in configuration wizard is not displayed

DATA-40943

Phase 1 (CrossSQLCron): missing fixed assignment between select and alias from Business Connector leads to incorrect behaviour in the configuration

DATA-39881

Phase 4: skip committing inactive SQL-nodes

DATA-40957

Phase 4: set internal skipping value back to null

DATA-40787

Phase 6: URI setting in HTTP response should not be mandatory

DATA-40781

Phase 6 (HTTP): functionality broken for dynamic multipart entries

DATA-40929

Phase 6 (FTP): avoid NullPointerException if no server address is given in used channel

DATA-40986

Phase 6 (FTP): Fixed wrong creation of a control filename for ftp-responses if prefix is selected as type and the destination name contained a path

DATA-40978

Phase 6 (FTP): Fixed wrong creation of a temporary filename for ftp-responses, "losing" a character, if prefix is selected as type and the destination name contained a path

DATA-40343

TransportManager: replacing a profile with Cron:DB fails when only selecting "without response units"

DATA-41070

TransportManager: ASM name missing in DIFF of source profile

DATA-40600

Date functions handling working days used the wrong days as weekend for some countries

DATA-40667

File encoding of profile is changed during structure import for destination tree

DATA-40677

HTTP Oauth2ServerServlet: deadlock/race condition when accessing token endpoint by many clients at the same time

DATA-40692

Function 'create json from node(a,b,c,d,e,f,g,h,i,j)': Parameter E 'Escape < and >' does not work if type safe option is set to false

DATA-40713

Running jobs: remove DMZ option in load balance combo box

DATA-40712

Plugins: SQL Console is using wrong method for calling CData Procedures

DATA-40748

DatabaseService: Consistent behavior over all databases when handling values of timezone containing columns

DATA-40826

JSON IU defect when using cut-off params and created data > 1024 chars

DATA-40941

Datatype of variable changed from blob to string on profile import

DATA-41161

OpenApi: missing reference occurs when openapi.war file is deployed on a different HTTP server than the request-servlet

DATA-40999

SSHService: Workaround to be able to react on file transfers that are finished after the client (falsely) logged off

DATA-40867

Offer tile display in each module overview

DATA-40865

Possibility to upload an image for module entries

DATA-38750

New update service which also includes _pro

DATA-41034

Home: new panel "Quota" for used/available module entries

DATA-41047

Dashboard: show amount of started jobs for each _data system since last start

DATA-40822

ETL: use Text driver instead of Velocity if the template does not contain velocity specific functions / macros

DATA-33224

ETL: support compressed files (zip, rar etc.) as input source

DATA-40648

ETL: new input type "Loop"

DATA-37652

Kafka: support Authentication using SASL OAuth with Keycloak

DATA-41135

HTTP input parameters: support regex for allowed values

DATA-41055

Phase 1 (Kafka): offer DateTimePicker for reset timestamp field

DATA-40698

Phase 1 (HTTP): mapping variables should be available in HTTP response message

DATA-41104

Phase 2 (X12 Parsing): Added support to interpret the repetition character 'U' as "unused" (as it's sometimes used in a non-standard-way)

DATA-40640

Phase 3: make all columns in mapping trees resizable

DATA-40659

Phase 3: csv export for field overview table

DATA-40977

Phase 3: more space for node/field description

DATA-40657

Phase 5: group constants like in phase 3

DATA-40676

Phase 6 (HTTP): adjust wrong content of constant(s) in host field

DATA-41149

Phase 6 (HTTP): add badge with entry count of adjusted HTTP headers

DATA-39894

Phase 6 (HTTP): add default response option for responses with empty body

DATA-40856

Functions: offer test functionality with optional result data of each function

DATA-40734

New function to change additional values of partner channels

DATA-40792

Workflow: option to render text or localized text as HTML

DATA-40869

Workflow: show triggered jobs of an active workflow in jobs overview

DATA-40282

Workflow: use of variables and constants in e-mail settings

DATA-39595

Failover: include date and version header in logs

DATA-40308

Failover: log exceptions in FOExternalUrlPing

DATA-40610

Failover: FOExternalURLCheck skip certificate validation

DATA-40637

Add autoserialize maps/lists to Kafka header - on demand

DATA-40639

Reorg: arbitrary time adjustment

DATA-40963

CommLog: offer SFIDs (Originator/Destination) in details

DATA-40736

Offer export/import of additional partner keys

DATA-40655

Add variable/placeholder VAR_SYS_MESSAGE_ID for HTTP agent

DATA-40651

Avoid file scanning if just one particular file is specified

DATA-40633

DatabaseService: keep milliseconds of time-containing values on PostgreSQL DBs

DATA-40697

DatabaseService: handle unconstrained decimals and numeric columns on PostgreSQL as real and not as long

DATA-40799

MessageService: allow the configuration to omit log entries of repeatedly failed messages consumers

DATA-40709

Generic Bearer Token: token available in a variable for response unit

DATA-40825

Generic Bearer Token: save (defined/given) HTTP-Header

DATA-41108

Generic Bearer Token: support credentials not only for requesting a new token

DATA-40815

Partner channel: opt. HTTP header name for access token of OAuth2

DATA-40931

ASM: handle duplicate OFTP filename in ENV File of simultaneous jobs in LB environment

DATA-40948

Plugins Upload: offer refresh button for directory

DATA-40836

Forms: show multi line stacktraces in error notification

DATA-40953

DataCockpit: Forms: Option to set the column width for fields that are shown in a list

DATA-41141

Option to set the 'valid until' date for trace log settings to infinite

DATA-41116

SwapManager: use tmp.IO as default root folder if not defined

DATA-41104

Enable X12-Parser to accept "U" for "unused" as repeating character

DATA-39493

CloudStorage: add scopes to channel settings; open links in new tab

DATA-40835

Optimization to Generic Token Handling in communication channels

DATA-40818

PostExecutor ChangeVariable

DATA-40768

Phase 1 (Cron:SQL): different error behavior of 'Delete data' and 'Delete data using where-clause'

DATA-40699

Dashboard: hide request count when working node is selected

DATA-40754

New DB preset for Snowflake

• All _data and DataCockpit cookies are now set to "httpOnly"

• Custom user styling in embedded view like function help view

• Profile selection window: implement wildcard search

• Phase 3: functions are now searched in grid via name and also via description

• Phase 3: add index of field in section in context menu header

• Phase 6 (File): make file path field wider

• ETL: add fields to template via double click

• Added Error handling for queries in Scriptella library

• JSONSplitter throws exception if created part is invalid due to wrong settings

• Fixed wrong URL check in de.lobster.tunnel.TunnelDriver.connect()

• Translation fixes

• New Vaadin Version 23.3.33