JDBC Import

Import Overview

When importing, the following five steps will need to be completed:

• Setup
• Source
• Destination
• Configure
• Summary

At any step, the following actions are available at the bottom of the screen:

• Next - proceed to the next step
• Back - return to the previous step
• Cancel - exit the import process
• Import - once enough information has been specified to begin importing data, the Import button will become active

Setup

In this step, the data source used to connect to the remote database is selected.

• Create New Data Source - click to create a new data source that connects to the remote database, then enter its configuration and click Create to create the data source and proceed to the Source selection page:
  • Name - enter a unique name for the data source
  • URL - enter the JDBC connection string to connect to the remote database
  • Username - username for logging into the remote database
  • Password - password for logging into the remote database
  • Driver Class - name of the JDBC driver class to use in the specified JDBC driver JAR
  • JAR Path Type - select a JAR path specification type to use when specifying the Driver JAR Path
  • Driver JAR Path - specify the file path to the JDBC driver JAR file, based on the selected JAR Path Type:
    • Manual - specify a directory path manually
    • KiFS - click the control to open the KiFS file selector, which shows a list of the KiFS directories, and as each is clicked, the files contained within that directory; click the JAR containing the JDBC driver to use for the connection
• Select Existing Data Source - click to select an existing data source that connects to the remote database and then click Next to proceed to the Source selection page:
  • Data Source - click to open a drop-down of available data sources that connect to the remote database and select one

Source

In this step, the source table is selected or the source query entered.

• Source - pre-selected as the name of the data source chosen in the previous step
• Remote Table - enter the name of the table located within the remote database; click Browse Tables to pop up the remote table browsing interface and make a selection from the available tables, and click Select next to the table to import
• Remote Query - enter a query to run on the remote database; click Verify Remote Query to ensure the query is able to be executed on the remote database
• Subscription Options - check the Enable box to activate a subscription to the remote data
  • Poll Interval - the time (in seconds) between successive requests to the remote cluster for more data
  • Increasing Column - remote table/query column to use to determine whether a remote record is new and should be imported; only records with a higher value than the highest value from the last poll will be imported

Destination

In this step, the target table to import into is selected.

• Schema - name of the schema containing the target table; if blank, the user's default schema will be used
• Table - name of the target table, which must meet table naming criteria; Workbench will suggest a table name here, if possible
• Batch Size - number of records from the source to insert into the database at a time; this can be used to tune the ingest speed for source data with many records or wide columns
• Abort on Error - check, to have the import stop at the first record import failure; any records imported by this point will remain in the target table
• Bad Records Table - when Abort on Error is unchecked, the errant records will be written to the specified table
  • Schema - schema in which the bad records table should reside
  • Table - name for the bad records table

Once the destination has been specified, click Next to proceed to the Configure page.

Configure

In this step, the target table's structure can be specified, if the table does not exist. Not specifying any structure will cause the import process to infer the table's structure from the source data.

To specify a table structure, click + Add Column once for each field in the source data, then enter the specification for each column, including:

• Name - name of the column, which must meet the standard naming criteria
• Type - type of the column, and sub-type, if applicable
• Nullable - check the box if the column should allow null values
• Properties - check any properties that should apply to this column:
  • Primary Key - make this column the primary key or part of a composite primary key
  • Shard Key - make this column the shard key or part of a composite shard key
  • Dict. Encoded - apply dictionary encoding to the column's values, reducing storage used for columns with more often repeated data
  • Init. with Now - replace empty or invalid values inserted into this column with the current date/time
  • Init. with UUID - replace empty values inserted into this column with a universally unique identifier (UUID)
  • Text Search - make this column full-text searchable, using FILTER_BY_STRING in search mode

To remove a column from the proposed target table, click the trash can icon at the far right of the column's definition.

Once the table configuration has been established, click Next to proceed to the Summary page.

Summary

In this step, the import configuration will be displayed.

All Source, Destination, & Error Handling configuration will be displayed in their respective sections.

The Generated SQL section will contain the SQL LOAD INTO command corresponding to the import operation that will take place. The copy-to-clipboard icon can be used to copy the SQL statement for subsequent use, to re-import data from the same file into the same table.

Once the import configuration has been confirmed, click Import to import the table's data.