CREATE CATALOG - Kinetica Docs

Creates a new data lake catalog, which holds the location & connection information for an Apache Iceberg or Delta Lake catalog that is external to the database.

CREATE CATALOG Syntax

CREATE [OR REPLACE] [EXTERNAL] CATALOG <catalog name>
[LOCATION = '<catalog uri>']
TABLE FORMAT = '<table format>'
TYPE = '<catalog type>'
[CREDENTIAL = [<schema name>.]<credential name>]
[DATASOURCE = [<schema name>.]<data source name>]
[WITH OPTIONS (<option name> = '<option value>'[,...])]

The following TABLE FORMAT & TYPE combinations are supported:

Table Format	Catalog Type	Description
`iceberg`	`rest`	Apache Iceberg REST catalog
`iceberg`	`glue`	AWS Glue Data Catalog
`delta`	`unity`	Delta Lake Unity Catalog

A catalog must be given either a DATASOURCE or an access_delegation of vended_credentials.

Parameters

OR REPLACE

Any existing catalog with the same name will be dropped before creating this one. As there is no ALTER CATALOG statement, this is how an existing catalog is modified.

EXTERNAL

Optional keyword; has no effect on the catalog created.

Name of the catalog to create; must adhere to the supported naming criteria. Cannot be schema-qualified.

LOCATION

URI of the external catalog. Optional; its meaning depends on the catalog type—see Format-Specific Syntax.

TABLE FORMAT

Required. Table format the catalog serves.

TYPE

Required. Catalog implementation. Must pair with TABLE FORMAT as listed above.

CREDENTIAL

Name of the credential used to authenticate to the external catalog.

DATASOURCE

Name of the data source supplying the object store connection for the data files.

Catalog Options

The following options may be given in the WITH OPTIONS clause:

access_delegation

How Kinetica authenticates to the object store holding the data files.

datasource_credentials (default) - use the credentials on the DATASOURCE
vended_credentials - use short-lived credentials issued by the external catalog

recreate

Whether to replace an existing catalog of the same name.

skip_validation

Whether to bypass validation of the connection to the external catalog. The default value is false.

Connection properties specific to a catalog type—such as region & warehouse for AWS Glue—are supplied this way; see Format-Specific Syntax.

Format-Specific Syntax

Apache Iceberg (REST)

The LOCATION is the REST catalog URI, and the DATASOURCE supplies the object store connection for the data files.

CREATE CATALOG (Iceberg REST) Example

CREATE CATALOG iceberg_rest_cat
LOCATION = 'https://iceberg.example.com/catalog'
TABLE FORMAT = 'iceberg'
TYPE = 'rest'
DATASOURCE = datalake_ds

Apache Iceberg (AWS Glue)

For AWS Glue, the CREDENTIAL carries the Glue API keys and the DATASOURCE carries the object store keys. The region & warehouse options are required; LOCATION is an optional Glue endpoint override.

CREATE CATALOG (Iceberg Glue) Example

CREATE CATALOG iceberg_glue_cat
TABLE FORMAT = 'iceberg'
TYPE = 'glue'
CREDENTIAL = glue_cred
DATASOURCE = datalake_ds
WITH OPTIONS
(
    region = 'us-east-1',
    warehouse = 's3://example-warehouse/'
)

Setting access_delegation to vended_credentials has Glue issue short-lived credentials for reading the data files, and requires no DATASOURCE:

CREATE CATALOG (Iceberg Glue, Vended Credentials) Example

CREATE CATALOG iceberg_vended_cat
LOCATION = 'https://glue.us-east-1.amazonaws.com'
TABLE FORMAT = 'iceberg'
TYPE = 'glue'
CREDENTIAL = glue_cred
WITH OPTIONS
(
    region = 'us-east-1',
    warehouse = 's3://example-warehouse/',
    access_delegation = 'vended_credentials'
)

Delta Lake (Unity Catalog)

Unity Catalog authenticates with a bearer token. There is no token credential type, so the token is given in the WITH OPTIONS clause of a rest credential with an empty SECRET:

CREATE CREDENTIAL (Unity Catalog) Example

CREATE CREDENTIAL unity_cred
TYPE = 'rest',
SECRET = ''
WITH OPTIONS
(
    'token' = '<unity catalog bearer token>'
)

The LOCATION is the Unity Catalog workspace URL; a trailing /api/2.1/unity-catalog is accepted and ignored.

CREATE CATALOG (Delta Lake Unity) Example

CREATE CATALOG delta_unity_cat
LOCATION = 'https://unity.example.com'
TABLE FORMAT = 'delta'
TYPE = 'unity'
CREDENTIAL = unity_cred
DATASOURCE = datalake_ds

Examples

To replace an existing catalog, which is how a catalog is modified in the absence of an ALTER CATALOG statement:

CREATE OR REPLACE CATALOG Example

CREATE OR REPLACE CATALOG iceberg_rest_cat
LOCATION = 'https://iceberg.example.com/catalog/v2'
TABLE FORMAT = 'iceberg'
TYPE = 'rest'
DATASOURCE = datalake_ds

To create a logical external table over a table in a catalog:

CREATE EXTERNAL TABLE over a Catalog Example

CREATE LOGICAL EXTERNAL TABLE example.ext_iceberg_flights
CATALOG PATH 'sales.flights'
WITH OPTIONS
(
    DATALAKE_CATALOG = 'iceberg_rest_cat'
)

​Parameters

​Catalog Options

​Format-Specific Syntax

​Apache Iceberg (REST)

​Apache Iceberg (AWS Glue)

​Delta Lake (Unity Catalog)

​Examples

Parameters

Catalog Options

Format-Specific Syntax

Apache Iceberg (REST)

Apache Iceberg (AWS Glue)

Delta Lake (Unity Catalog)

Examples