Limitation: Additional Connection Parameters getting removed when connecting to Oracle Database via JDBC on Spark shared clusters

Affected Version

6.1.1

Description

When configuring a JDBC connection to an Oracle database in a Databricks Spark shared cluster, any additional connection parameters (such as timezoneAsRegion or any other custom flags) that are passed separately in the configuration might get removed during runtime. This behavior results in these parameters not being applied, potentially causing failures or incomplete configurations during the connection.

Root Cause

The Spark shared cluster's underlying mechanism for handling JDBC connection configurations does not consistently preserve or apply additional parameters specified outside the primary connection URL. This limitation is specific to shared Spark cluster environments and affects how the Oracle JDBC driver processes the provided configuration.

To Resolve

To overcome this limitation, include the additional connection parameters as part of the JDBC connection URL instead of specifying them separately as configuration parameters.

How to Implement the Workaround

Instead of passing additional parameters through the configuration (e.g., as a dictionary or key-value pair in the Spark session), append them directly to the JDBC connection URL. Follow the steps below:

1. Identify the Additional Parameters: Determine the additional connection parameters you need to include, such as
2. oracle.jdbc.timezoneAsRegion,oracle.jdbc.ReadTimeout etc, or other custom options.

Format the JDBC URL with Parameters:Combine these parameters with the main connection URL using the following format:

Structure of JDBC URL:

xxxxxxxxxx

jdbc:oracle:thin:@<host>:<port>/<service>?<parameter1>=<value1>&<parameter2>=<value2>

Example URL with params:

xxxxxxxxxx

jdbc:oracle:thin:@db-host:1521/xe?oracle.jdbc.timezoneAsRegion=false&oracle.jdbc.ReadTimeout=15000