> ## Documentation Index
> Fetch the complete documentation index at: https://docs.kinetica.com/llms.txt
> Use this file to discover all available pages before exploring further.

# CREATE DATA SINK

<a id="sql-create-data-sink" />

Creates a new [data sink](/content/concepts/data_sinks), which contains the
location and connection information for a data consumer that is *generally*
external to *Kinetica*.  A *data sink* serves as an authentication & access
mechanism to the remote resource.

```sql title="CREATE DATA SINK Syntax" theme={null}
CREATE [OR REPLACE] [EXTERNAL] DATA SINK [<schema name>.]<data sink name>
LOCATION = '<consumer>://<target>[:<port>]'
[WITH OPTIONS (<option name> = '<option value>'[,...])]
```

A *data sink* can be referenced in an [EXPORT ... INTO](/content/sql/export#sql-export) call (for
exporting local table data to a remote system) or a
[CREATE STREAM](/content/sql/ddl/create-stream#sql-create-stream) call (for streaming local table data to a
remote system).

<Info>
  The *data sink* will be validated upon creation, by default, and
  will fail to be created if an authorized connection cannot be established.
</Info>

The following *data sink* consumers are supported:

* Azure *(Microsoft blob storage)*
* GCS *(Google Cloud Storage)*
* HDFS *(Apache Hadoop Distributed File System)*
* JDBC *(Java Database Connectivity, using a user-supplied driver or one of the*
  *drivers on the [supported list](/content/concepts/jdbc_drivers))*
* Kafka *(Apache Kafka streaming feed)*
* Local *(Table within the same Kinetica instance)*
* S3 *(Amazon S3 Bucket)*
* Webhook *(HTTP/HTTPS)*

<Info>
  Kafka *data sinks* will be validated upon creation, by default, and will
  fail to be created if an authorized connection cannot be established.
</Info>

For consumer-specific syntax, see [Consumer-Specific Syntax](/content/sql/ddl/create-data-sink#sql-create-data-sink-syntax).
For consumer-specific examples, see [Creating Data Sinks](/content/snippets/create-data-sinks).

## Parameters

<AccordionGroup>
  <Accordion title="OR REPLACE" id="or-replace" defaultOpen>
    Any existing *data sink* with the same name will be dropped before creating this one
  </Accordion>

  <Accordion title="EXTERNAL" id="external" defaultOpen>
    Optional keyword for clarity
  </Accordion>

  <Accordion title="<schema name>" id="<schema-name>" defaultOpen>
    Name of the *schema* that will contain the created *data sink*; if no *schema* is specified,
    the *data sink* will be created in the user's [default schema](/content/concepts/schemas#schema-default)
  </Accordion>

  <Accordion title="<data sink name>" id="<data-sink-name>" defaultOpen>
    Name of the *data sink* to create; must adhere to the supported
    [naming criteria](/content/sql/naming#sql-naming-criteria)
  </Accordion>

  <Accordion title="<consumer>" id="<consumer>" defaultOpen>
    Consumer of the *data sink*

    Supported consumers include:

    <div>
      <table class="table w-full [&_td]:min-w-[150px] [&_th]:text-left [&_td[data-numeric]]:tabular-nums">
        <thead>
          <tr>
            <th>Consumer</th>
            <th>Description</th>
          </tr>
        </thead>

        <tbody>
          <tr>
            <td><code>AZURE</code></td>
            <td>Microsoft Azure blob storage</td>
          </tr>

          <tr>
            <td><code>GCS</code></td>
            <td>Google Cloud Storage</td>
          </tr>

          <tr>
            <td><code>HDFS</code></td>
            <td>Apache Hadoop Distributed File System</td>
          </tr>

          <tr>
            <td><code>HTTP</code></td>
            <td>Unsecured webhook</td>
          </tr>

          <tr>
            <td><code>HTTPS</code></td>
            <td>Secured webhook</td>
          </tr>

          <tr>
            <td><code>JDBC</code></td>
            <td>JDBC connection, where <code>LOCATION</code> is the JDBC URL. <pre><code class="language-sql">LOCATION = 'jdbc:postgresql://example.com:5432/mydb'</code></pre> See the [supported list](/content/concepts/jdbc_drivers) for the full list of supported drivers, or specify one with <code>JDBC\_DRIVER\_JAR\_PATH</code> and <code>JDBC\_DRIVER\_CLASS\_NAME</code>.</td>
          </tr>

          <tr>
            <td><code>KAFKA</code></td>
            <td>Apache Kafka broker</td>
          </tr>

          <tr>
            <td><code>TABLE</code></td>
            <td>Local Kinetica table residing in the same database as the *data sink*: <pre><code class="language-sql">LOCATION = 'TABLE://example.employee\_backup'</code></pre></td>
          </tr>

          <tr>
            <td><code>S3</code></td>
            <td>Amazon S3 bucket</td>
          </tr>
        </tbody>
      </table>
    </div>
  </Accordion>

  <Accordion title="<target>" id="<target>" defaultOpen>
    Target to connect to via *data sink*; the following consumers have a default *host target*, if
    `target` is left blank:

    <div>
      <table class="table w-full [&_td]:min-w-[150px] [&_th]:text-left [&_td[data-numeric]]:tabular-nums">
        <thead>
          <tr>
            <th>Consumer</th>
            <th>Default Host</th>
          </tr>
        </thead>

        <tbody>
          <tr>
            <td>*Azure*</td>
            <td><code>\<storage\_account\_name>.blob.core.windows.net</code></td>
          </tr>

          <tr>
            <td>*GCS*</td>
            <td><code>storage.googleapis.com</code></td>
          </tr>

          <tr>
            <td>*S3*</td>
            <td><code>\<region>.amazonaws.com</code></td>
          </tr>
        </tbody>
      </table>
    </div>

    <Info>
      For local Kinetica tables, `target` is the name of the name of the target table:

      ```
      [<target table schema name>].<target table name>
      ```
    </Info>
  </Accordion>

  <Accordion title="<port>" id="<port>" defaultOpen>
    Port to connect to via the *data sink*, if applicable
  </Accordion>

  <Accordion title="WITH OPTIONS" id="with-options" defaultOpen>
    Optional indicator that a comma-delimited list of connection option/value assignments will
    follow.  See [Options](/content/sql/ddl/create-data-sink#sql-dsink-opts) for the full list of options.
  </Accordion>
</AccordionGroup>

## Examples

To create a *data sink*, `kin_dsink`, that connects to an Apache Kafka broker:

```sql CREATE DATA SINK Example theme={null}
CREATE DATA SINK kin_dsink
LOCATION = 'kafka://kafka.abc.com:9092'
WITH OPTIONS
(
    CREDENTIAL = 'kafka_credential',
    KAFKA_TOPIC_NAME = 'kafka_topic'
)
```

<a id="sql-dsink-opts" />

## Options

| Option                             | Consumer                 | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| ---------------------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `CREDENTIAL`                       | *Any*                    | [Credential](/content/sql/ddl/create-credential#sql-create-credential) object to use to authenticate to the remote consumer                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| `VALIDATE`                         | *Any*                    | Whether to test the connection to the *data sink* upon creation; if `TRUE` *(default)*, the creation of a *data sink* that cannot be connected to will fail; if `FALSE`, the *data sink* will be created regardless of connectivity                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| `WAIT TIMEOUT`                     | *Any* except `JDBC`      | Timeout in seconds for reading from the consumer                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| `CONNECTION TIMEOUT`               | *Any* except `JDBC`      | Timeout in seconds for connecting to a consumer                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `USE_HTTPS`                        | `AZURE`, `GCS`, `S3`     | Whether to connect to the storage consumer over HTTPS or not <br /> <br /> `true`: Use HTTPS *(default)* <br /> <br /> `false`: Use HTTP                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `USE_MANAGED_CREDENTIALS`          | `AZURE`, `GCS`, `S3`     | Whether to connect to the storage provider with consumer-managed credentials <br /> <br /> `AZURE`: Use the *Azure Instance Metadata Service (IMDS)* endpoint local to the Kinetica cluster to acquire credentials *(only for on-prem clusters deployed within Azure)* <br /> <br /> `GCS`: Satisfy the [Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials) requirements for credentials, making sure any supporting server-side configuration (files, environment variables) is done on every node in the cluster *(only for on-prem clusters deployed within GCS)* <br /> <br /> `S3`: Use the [AWS Default Credential Provider Chain](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html) to acquire credentials, making sure any supporting server-side configuration (files, environment variables) is done on every node in the cluster *(only for on-prem clusters deployed within S3)* |
| `KAFKA_TOPIC_NAME`                 | `KAFKA`                  | Kafka topic to write to                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| `JSON_FORMAT`                      | `HTTP`, `HTTPS`, `KAFKA` | The desired format of JSON encoded notifications message. <br /> <br /> `flat`: A single record is returned per message <br /> <br /> `nested`: Records are returned in an array per message                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| `MAX_BATCH_SIZE`                   | `HTTP`, `HTTPS`, `KAFKA` | Maximum number of records per notification message                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| `MAX_MESSAGE_SIZE`                 | `HTTP`, `HTTPS`, `KAFKA` | Maximum size in bytes of each notification message                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| `CONTAINER NAME`                   | `AZURE`                  | Azure storage container name                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| `SAS TOKEN`                        | `AZURE`                  | Azure storage account shared access signature token; this should be an account-level access token, not a container-level one                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| `STORAGE ACCOUNT NAME`             | `AZURE`                  | Azure storage account name *(only used if* `TENANT ID` *is specified)*                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| `TENANT ID`                        | `AZURE`                  | Azure Active Directory tenant identifier                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `GCS_BUCKET_NAME`                  | `GCS`                    | Name of the GCS bucket to use as the data sink                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| `GCS_PROJECT_ID`                   | `GCS`                    | Name of the Google Cloud project to use for request billing                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| `GCS_SERVICE_ACCOUNT_KEYS`         | `GCS`                    | Text of the JSON key file containing the GCS private key                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `DELEGATION TOKEN`                 | `HDFS`                   | Optional Kerberos delegation token for *worker nodes*; if not specified, the token will be acquired from HDFS                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| `KERBEROS KEYTAB`                  | `HDFS`                   | Location of the Kerberos keytab file in [KiFS](/content/tools/kifs)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| `USE KERBEROS`                     | `HDFS`                   | Whether to attempt Kerberos authentication to HDFS                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| `JDBC_DRIVER_CLASS_NAME`           | `JDBC`                   | JDBC driver class name *(optional, if the name is available in the* *JAR file's manifest)*                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| `JDBC_DRIVER_JAR_PATH`             | `JDBC`                   | [KiFS](/content/tools/kifs) path of the JDBC driver JAR file to use                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| `BUCKET NAME`                      | `S3`                     | Amazon S3 bucket name                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `REGION`                           | `S3`                     | Amazon S3 region identifier                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| `S3_AWS_ROLE_ARN`                  | `S3`                     | Amazon Resource Name (ARN) specifying the role                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| `S3_ENCRYPTION_CUSTOMER_ALGORITHM` | `S3`                     | Algorithm used to encrypt/decrypt data                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| `S3_ENCRYPTION_CUSTOMER_KEY`       | `S3`                     | Key used to encrypt/decrypt data                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| `S3_ENCRYPTION_TYPE`               | `S3`                     | Server side encryption type                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| `S3_KMS_KEY_ID`                    | `S3`                     | KMS key                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| `S3_USE_VIRTUAL_ADDRESSING`        | `S3`                     | Whether to use virtual addressing when referencing the S3 sink <br /> <br /> `true`: The requests URI should be specified in virtual-hosted-style format where the bucket name is part of the domain name in the URL <br /> <br /> `false`: Use path-style URI for requests                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| `S3_VERIFY_SSL`                    | `S3`                     | Whether to verify SSL connections <br /> <br /> `true`: Verify SSL connections *(default)* <br /> <br /> `false`: Don't verify SSL connections; for testing purposes, bypassing TLS errors, self-signed certificates, etc.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |

<a id="sql-create-data-sink-syntax" />

## Consumer-Specific Syntax

Several authentication schemes across multiple consumers are supported.

* [Azure](/content/sql/ddl/create-data-sink#sql-create-data-sink-azure)
* [GCS](/content/sql/ddl/create-data-sink#sql-create-data-sink-gcs)
* [HDFS](/content/sql/ddl/create-data-sink#sql-create-data-sink-hdfs)
* [JDBC](/content/sql/ddl/create-data-sink#sql-create-data-sink-jdbc)
* [Kafka](/content/sql/ddl/create-data-sink#sql-create-data-sink-kafka)
* [Local (Kinetica)](/content/sql/ddl/create-data-sink#sql-create-data-sink-local)
* [S3](/content/sql/ddl/create-data-sink#sql-create-data-sink-s3)
* [Webhook](/content/sql/ddl/create-data-sink#sql-create-data-sink-webhook)

<a id="sql-create-data-sink-azure" />

### Azure

Syntax below, examples [here](/content/snippets/create-data-sinks#snippet-sql-create-data-sink-azure).

<CodeGroup>
  ```sql Credential theme={null}
  CREATE DATA SINK [<data sink schema name>.]<data sink name>
  LOCATION = 'AZURE[://<host>]'
  WITH OPTIONS
  (
  	CREDENTIAL = '[<credential schema name>.]<credential name>',
  	CONTAINER NAME = '<azure container name>'
  )
  ```

  ```sql Managed Credentials theme={null}
  CREATE DATA SINK [<data sink schema name>.]<data sink name>
  LOCATION = 'AZURE[://<host>]'
  WITH OPTIONS
  (
      USE_MANAGED_CREDENTIALS = true,
      STORAGE ACCOUNT NAME = '<azure storage account name>',
      CONTAINER NAME = '<azure container name>',
      TENANT ID = '<ad tenant id>'
  )
  ```
</CodeGroup>

<a id="sql-create-data-sink-gcs" />

### GCS

Syntax below, examples [here](/content/snippets/create-data-sinks#snippet-sql-create-data-sink-gcs).

<CodeGroup>
  ```sql Credential theme={null}
  CREATE DATA SINK [<data sink schema name>.]<data sink name>
  LOCATION = 'GCS[://<host>]'
  WITH OPTIONS
  (
  	CREDENTIAL = '[<credential schema name>.]<credential name>',
  	[GCS_PROJECT_ID = '<gcs project id>',]
  	GCS_BUCKET_NAME = '<gcs bucket name>'
  )
  ```

  ```sql Managed Credentials theme={null}
  CREATE DATA SINK [<data sink schema name>.]<data sink name>
  LOCATION = 'GCS[://<host>]'
  WITH OPTIONS
  (
  	USE_MANAGED_CREDENTIALS = true,
  	[GCS_PROJECT_ID = '<gcs project id>',]
  	GCS_BUCKET_NAME = '<gcs bucket name>'
  )
  ```

  ```sql Public (No Auth) theme={null}
  CREATE DATA SINK [<data sink schema name>.]<data sink name>
  LOCATION = 'GCS[://<host>]'
  WITH OPTIONS
  (
  	[GCS_PROJECT_ID = '<gcs project id>',]
  	GCS_BUCKET_NAME = '<gcs bucket name>'
  )
  ```

  ```sql JSON Key theme={null}
  CREATE DATA SINK [<data sink schema name>.]<data sink name>
  LOCATION = 'GCS[://<host>]'
  WITH OPTIONS
  (
  	GCS_SERVICE_ACCOUNT_KEYS = '<gcs account json key text>',
  	[GCS_PROJECT_ID = '<gcs project id>',]
  	GCS_BUCKET_NAME = '<gcs bucket name>'
  )
  ```
</CodeGroup>

<a id="sql-create-data-sink-hdfs" />

### HDFS

Syntax below, examples [here](/content/snippets/create-data-sinks#snippet-sql-create-data-sink-hdfs).

```sql Credential theme={null}
CREATE DATA SINK [<data sink schema name>.]<data sink name>
LOCATION = 'HDFS://<host>:<port>'
WITH OPTIONS (CREDENTIAL = '[<credential schema name>.]<credential name>')
```

<a id="sql-create-data-sink-jdbc" />

### JDBC

Syntax below, examples [here](/content/snippets/create-data-sinks#snippet-sql-create-data-sink-jdbc).

<CodeGroup>
  ```sql Credential theme={null}
  CREATE DATA SINK [<data sink schema name>.]<data sink name>
  LOCATION = '<jdbc url>'
  WITH OPTIONS
  (
  	[JDBC_DRIVER_CLASS_NAME = '<jdbc driver class full path>',]
  	[JDBC_DRIVER_JAR_PATH = 'kifs://<jdbc driver jar path>',]
  	CREDENTIAL = '[<credential schema name>.]<credential name>'
  )
  ```

  ```sql Password in URL theme={null}
  CREATE DATA SINK [<data sink schema name>.]<data sink name>
  LOCATION = '<jdbc url with username/password>'
  ```
</CodeGroup>

<a id="sql-create-data-sink-kafka" />

### Kafka

Syntax below, examples [here](/content/snippets/create-data-sinks#snippet-sql-create-data-sink-kafka).

<CodeGroup>
  ```sql Credential theme={null}
  CREATE DATA SINK [<data sink schema name>.]<data sink name>
  LOCATION = 'kafka://<host>[:<port>]'
  WITH OPTIONS
  (
  	CREDENTIAL = '[<credential schema name>.]<credential name>',
  	KAFKA_TOPIC_NAME = '<kafka topic name>'
  )
  ```

  ```sql Public (No Auth) theme={null}
  CREATE DATA SINK [<data sink schema name>.]<data sink name>
  LOCATION = 'kafka://<host>[:<port>]'
  WITH OPTIONS (KAFKA_TOPIC_NAME = '<kafka topic name>')
  ```
</CodeGroup>

<a id="sql-create-data-sink-local" />

### Local (Kinetica)

Syntax below, examples [here](/content/snippets/create-data-sinks#snippet-sql-create-data-sink-local).

```sql User Auth theme={null}
CREATE DATA SINK [<data sink schema name>.]<data sink name>
LOCATION = 'TABLE://[<table schema name>.]<table name>'
```

<a id="sql-create-data-sink-s3" />

### S3

Syntax below, examples [here](/content/snippets/create-data-sinks#snippet-sql-create-data-sink-s3).

<CodeGroup>
  ```sql Credential theme={null}
  CREATE DATA SINK [<data sink schema name>.]<data sink name>
  LOCATION = 'S3[://<host>]'
  WITH OPTIONS
  (
      CREDENTIAL = '[<credential schema name>.]<credential name>',
      BUCKET NAME = '<aws bucket name>',
      REGION = '<aws region>'
  )
  ```

  ```sql Managed Credentials theme={null}
  CREATE DATA SINK [<data sink schema name>.]<data sink name>
  LOCATION = 'S3[://<host>]'
  WITH OPTIONS
  (
      USE_MANAGED_CREDENTIALS = true,
      BUCKET NAME = '<aws bucket name>',
      REGION = '<aws region>'
  )
  ```

  ```sql Public (No Auth) theme={null}
  CREATE DATA SINK [<data sink schema name>.]<data sink name>
  LOCATION = 'S3[://<host>]'
  WITH OPTIONS
  (
      BUCKET NAME = '<aws bucket name>',
      REGION = '<aws region>'
  )
  ```
</CodeGroup>

<a id="sql-create-data-sink-webhook" />

### Webhook

Syntax below, examples [here](/content/snippets/create-data-sinks#snippet-sql-create-data-sink-webhook).

<CodeGroup>
  ```sql Credential w/ HTTPS theme={null}
  CREATE DATA SINK [<data sink schema name>.]<data sink name>
  LOCATION = 'https://<host>[:<port>]'
  WITH OPTIONS (CREDENTIAL = '[<credential schema name>.]<credential name>')
  ```

  ```sql HTTP theme={null}
  CREATE DATA SINK [<data sink schema name>.]<data sink name>
  LOCATION = 'http://<host>[:<port>]'
  ```
</CodeGroup>