> ## Documentation Index
> Fetch the complete documentation index at: https://docs.kinetica.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Sinks

A *data sink* is reference object for a data target that is *generally* external
to the database.  It consists of the location & connection information to that
target.  A *data sink* can make use of a
[credential](/content/concepts/credentials) object for storing remote authentication
information.

A *data sink* name must adhere to the standard
[naming criteria](/content/concepts/tables#table-naming-criteria).  Each *data sink*
exists within a [schema](/content/concepts/schemas) and follows the standard
[name resolution rules](/content/concepts/tables#table-name-resolution) for *tables*.

The following *data sink* types are supported:

* Azure *(Microsoft blob storage)*
* GCS *(Google Cloud Storage)*
* HDFS *(Apache Hadoop Distributed File System)*
* JDBC *(Java Database Connectivity, using a user-supplied driver or one of the*
  *drivers on the [supported list](/content/concepts/jdbc_drivers))*
* Kafka *(Apache Kafka streaming feed)*
* Local *(Table within the same Kinetica instance)*
* S3 *(Amazon S3 Bucket)*
* Webhook *(HTTP/HTTPS)*

<Info>
  The following default hosts are used for Azure, GCS, & S3, but can be
  overridden in the `destination` parameter:

  * Azure:  `<service_account_name>.blob.core.windows.net`
  * GCS:    `storage.googleapis.com`
  * S3:     `<region>.amazonaws.com`
</Info>

*Data sinks* perform no function by themselves, but act as proxies for
transmitting data when referenced as a destination in the creation of a
[table monitor](/content/concepts/table_monitors) (see also the
[CREATE STREAM](/content/sql/ddl#sql-create-stream) command in SQL), or when referenced as the
target of an [/export/records/totable](/content/api/rest/export_records_totable_rest) call (see also the
[Exporting Data](/content/sql/export) command in SQL).

<Info>
  Kafka *data sinks* will be validated upon creation, by default, and will
  fail to be created if an authorized connection cannot be established.
</Info>

## Managing Data Sinks

A *data sink* can be managed using the following API endpoint calls.  For
managing *data sinks* in SQL, see [CREATE DATA SINK](/content/sql/ddl#sql-create-data-sink).

| API Call                                                       | Description                                                                                                                                       |
| -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| [/create/datasink](/content/api/rest/create_datasink_rest)     | Creates a *data sink*, given a location and connection information                                                                                |
| [/alter/datasink](/content/api/rest/alter_datasink_rest)       | Modifies the properties of a *data sink*, validating the new connection                                                                           |
| [/drop/datasink](/content/api/rest/drop_datasink_rest)         | Removes the *data sink* reference from the database; optionally removing all dependent [table monitors](/content/concepts/table_monitors) as well |
| [/show/datasink](/content/api/rest/show_datasink_rest)         | Outputs the *data sink* properties                                                                                                                |
| [/grant/permission](/content/api/rest/grant_permission_rest)   | Grants the [permission](/content/security/sec_concepts#security-concepts-permissions-datasink) for a user to connect to a *data sink*             |
| [/revoke/permission](/content/api/rest/revoke_permission_rest) | Revokes the [permission](/content/security/sec_concepts#security-concepts-permissions-datasink) for a user to connect to a *data sink*            |

## Creating a Data Sink

To create a *data sink*, `kin_dsink`, that targets *Apache Kafka*:

<CodeGroup>
  ```sql SQL theme={null}
  CREATE DATA SINK kin_dsink
  LOCATION = 'kafka://kafka.abc.com:9092'
  WITH OPTIONS
  (
      CREDENTIAL = 'kafka_credential',
      KAFKA_TOPIC_NAME = 'kafka_topic'
  )
  ```

  ```python Python theme={null}
  kinetica.create_datasink(
      name = 'kin_dsink',
      destination = 'kafka://kafka.abc.com:9092',
      options = {
          'credential': 'kafka_credential',
          'kafka_topic_name': 'kafka_topic'
      }
  )
  ```
</CodeGroup>

To create a *data sink* that targets a local database table:

<CodeGroup>
  ```sql SQL theme={null}
  CREATE DATA SINK kin_dsink_local
  LOCATION = 'table://example.ds_employee_backup'
  ```

  ```python Python theme={null}
  kinetica.create_datasink(
      name = 'kin_dsink',
      destination = 'table://example.ds_employee_backup'
  )
  ```
</CodeGroup>

<a id="create-data-sink-consumers" />

### Consumer-Specific Syntax

Several authentication schemes across multiple providers are supported.

* [Azure](/content/concepts/data_sinks#create-data-sink-azure)
* [GCS](/content/concepts/data_sinks#create-data-sink-gcs)
* [HDFS](/content/concepts/data_sinks#create-data-sink-hdfs)
* [JDBC](/content/concepts/data_sinks#create-data-sink-jdbc)
* [Kafka](/content/concepts/data_sinks#create-data-sink-kafka)
* [Local (Kinetica)](/content/concepts/data_sinks#create-data-sink-local)
* [S3](/content/concepts/data_sinks#create-data-sink-s3)
* [Webhook](/content/concepts/data_sinks#create-data-sink-webhook)

<a id="create-data-sink-azure" />

#### Azure

<CodeGroup>
  ```python Credential theme={null}
  kinetica.create_datasink(
      name = '[<data sink schema name>.]<data sink name>',
      destination = 'azure[://<host>]',
      options = {
          'credential': '[<credential schema name>.]<credential name>',
          'azure_container_name': '<azure container name>'
      }
  )
  ```

  ```python Managed Credentials theme={null}
  kinetica.create_datasink(
      name = '[<data sink schema name>.]<data sink name>',
      destination = 'azure[://<host>]',
      options = {
      	'use_managed_credentials': 'true',
          'azure_storage_account_name': '<azure storage account name>',
          'azure_container_name': '<azure container name>',
          'azure_tenant_id': '<azure tenant id>'
      }
  )
  ```
</CodeGroup>

<a id="create-data-sink-gcs" />

#### GCS

<CodeGroup>
  ```python Credential theme={null}
  kinetica.create_datasink(
      name = '[<data sink schema name>.]<data sink name>',
      destination = 'gcs[://<host>]',
      options = {
      	'credential': '[<credential schema name>.]<credential name>',
      	['gcs_project_id': '<gcs project id>',]
      	'gcs_bucket_name': '<gcs bucket name>'
      }
  )
  ```

  ```python Managed Credentials theme={null}
  kinetica.create_datasink(
      name = '[<data sink schema name>.]<data sink name>',
      destination = 'gcs[://<host>]',
      options = {
      	'use_managed_credentials': 'true',
      	['gcs_project_id': '<gcs project id>',]
      	'gcs_bucket_name': '<gcs bucket name>'
      }
  )
  ```

  ```python Public (No Auth) theme={null}
  kinetica.create_datasink(
      name = '[<data sink schema name>.]<data sink name>',
      destination = 'gcs[://<host>]',
      options = {
      	['gcs_project_id': '<gcs project id>',]
      	'gcs_bucket_name': '<gcs bucket name>'
      }
  )
  ```

  ```python JSON Key theme={null}
  kinetica.create_datasink(
      name = '[<data sink schema name>.]<data sink name>',
      destination = 'gcs[://<host>]',
      options = {
      	'gcs_service_account_keys': '<gcs account json key text>',
      	['gcs_project_id': '<gcs project id>',]
      	'gcs_bucket_name': '<gcs bucket name>'
      }
  )
  ```
</CodeGroup>

<a id="create-data-sink-hdfs" />

#### HDFS

```python Credential theme={null}
kinetica.create_datasink(
    name = '[<data sink schema name>.]<data sink name>',
    destination = 'hdfs://<host>:<port>',
    options = {
        'credential': '[<credential schema name>.]<credential name>'
    }
)
```

<a id="create-data-sink-jdbc" />

#### JDBC

<CodeGroup>
  ```python Credential theme={null}
  kinetica.create_datasink(
      name = '[<data sink schema name>.]<data sink name>',
      destination = '<jdbc url>',
      options = {
          ['jdbc_driver_class_name': '<jdbc driver class full path>',]
          ['jdbc_driver_jar_path': 'kifs://<jdbc driver jar path>',]
          'credential': '[<credential schema name>.]<credential name>'
      }
  )
  ```

  ```python Password in URL theme={null}
  kinetica.create_datasink(
      name = '[<data sink schema name>.]<data sink name>',
      destination = '<jdbc url with username/password>'
  )
  ```
</CodeGroup>

<a id="create-data-sink-kafka" />

#### Kafka

<CodeGroup>
  ```python Credential theme={null}
  kinetica.create_datasink(
      name = '[<data sink schema name>.]<data sink name>',
      destination = 'kafka://<kafka.host>:<kafka.port>',
      options = {
          'credential': '[<credential schema name>.]<credential name>',
          'kafka_topic_name': '<kafka topic name>'
      }
  )
  ```

  ```python Public (No Auth) theme={null}
  kinetica.create_datasink(
      name = '[<schema name>.]<data sink name>',
      destination = 'kafka://<kafka.host>:<kafka.port>',
      options = {
          'kafka_topic_name': '<kafka topic name>'
      }
  )
  ```
</CodeGroup>

<a id="create-data-sink-local" />

#### Local (Kinetica)

```python User Auth theme={null}
kinetica.create_datasink(
    name = '[<data sink schema name>.]<data sink name>',
    destination = 'table://[<table schema name>.]<table name>'
)
```

<a id="create-data-sink-s3" />

#### S3

<CodeGroup>
  ```python Credential theme={null}
  kinetica.create_datasink(
      name = '[<data sink schema name>.]<data sink name>',
      destination = 's3[://<host>]',
      options = {
      	'credential': '[<credential schema name>.]<credential name>',
          's3_bucket_name': '<aws s3 bucket name>',
          's3_region': '<aws s3 region>'
      }
  )
  ```

  ```python Managed Credentials theme={null}
  kinetica.create_datasink(
      name = '[<data sink schema name>.]<data sink name>',
      destination = 's3[://<host>]',
      options = {
      	'use_managed_credentials': 'true',
          's3_bucket_name': '<aws s3 bucket name>',
          's3_region': '<aws s3 region>'
      }
  )
  ```

  ```python Public (No Auth) theme={null}
  kinetica.create_datasink(
      name = '[<data sink schema name>.]<data sink name>',
      destination = 's3[://<host>]',
      options = {
          's3_bucket_name': '<aws s3 bucket name>',
          's3_region': '<aws s3 region>'
      }
  )
  ```
</CodeGroup>

<a id="create-data-sink-webhook" />

#### Webhook

<CodeGroup>
  ```python Credential (with HTTPS) theme={null}
  kinetica.create_datasink(
      name = '[<data sink schema name>.]<data sink name>',
      destination = 'https://<webhook.host>:<webhook.port>',
      options = {
          'credential': '[<credential schema name>.]<credential name>'
      }
  )
  ```

  ```python HTTP theme={null}
  kinetica.create_datasink(
      name = '[<schema name>.]<data sink name>',
      destination = 'http://<webhook.host>:<webhook.port>'
  )
  ```
</CodeGroup>