Manual Kinetica Installation

Default Ports

The default ports used for communication with KAgent, Kinetica (and between servers, if operating in a cluster), and various important services follow. The Nodes column will list either Head--that the corresponding port only needs to be opened on the head node, or All--that the corresponding port needs to be opened on the head node & worker nodes.

Port Usage Scenarios

Kinetica highly encourages that proper firewalls be maintained and used to protect the database and the network at large. A full tutorial on how to properly set up a firewall is beyond the scope of this document, but the following are some best practices and starting points for more research.

All machines connected to the Internet at large should be protected from intrusion. As shown in the list above, there are no ports which are necessarily required to be accessible from outside of a trusted network, so we recommend only opening ports to the Internet and/or untrusted network(s) which are truly needed based on requirements.

There are some common scenarios which can act as guidelines on which ports should be available.

Firewall Settings

firewall-cmd --zone=public --add-port=8082/tcp --permanent
firewall-cmd --reload

firewall-cmd --zone=public --add-port=8082/tcp --permanent
firewall-cmd --reload

sudo ufw allow 8181

Transparent Huge Pages

Transparent Huge Pages are the kernel’s attempt to reduce the overhead of Translation Lookaside Buffer (TLB) lookups by increasing the size of memory pages. This setting is enabled by default, but can lead to sparse memory usage and decreased performance.

sudo sh -c 'echo "never" > /sys/kernel/mm/transparent_hugepage/enabled'

Active Directory

If your environment uses Microsoft Active Directory for authentication and there are security processes running on servers that check for and automatically remove accounts that are not registered in Active Directory, the gpudb user must be added to Active Directory as a Linux-type account prior to installing Kinetica.

Nvidia Drivers

If Nvidia GPUs are present in the target servers, but the drivers have not been installed yet, they should be installed now. See either Install Nvidia Drivers on RHEL or Install Nvidia Drivers on Debian/Ubuntu for details.

Visual Initialization

The Visual Installer is run through the Kinetica Administration Application (GAdmin) and simplifies the installation of Kinetica.

Browse to the head node, using IP or host name:

http://localhost:8080/

Once you've arrived at the login page, you'll need to change your password and initialize the system using the following steps:

1. Log into the admin application

   1. Enter Username: admin
   2. Enter Password: admin
   3. Click Login

2. If a license key has not already been configured, a Product Activation page will be displayed, where the license key is to be entered:

   
   1. Enter the license key under Enter License Key
   2. When complete, click Activate, then confirm the activation

3. At the Setup Wizard page, configure the system basics:

   1. Enter the IP Address and number of GPUs (if any) for each server in the cluster
   2. Optionally, select the Public Head IP Address checkbox and update the address as necessary
   3. The license key under Configure License Key should already be populated
   4. When complete, click Save

   Important

   For additional configuration options, see the Configuration Reference.

4. Start the system. This will start all Kinetica processes on the head node, and if in a clustered environment, the corresponding processes on the worker nodes.

   1. Click Admin on the left menu
   2. Click Start.

5. See Changing the Administrator Password for instructions on updating the administration account's password.

Skip ahead to Passwordless SSH.

Console Initialization

System configuration is done primarily through the configuration file /opt/gpudb/core/etc/gpudb.conf, and while all nodes in a cluster have this file, only the copy on the head node needs to be modified.

For details on the parameters used in this section, see Configuration Reference.

1. Log in to the head node and open /opt/gpudb/core/etc/gpudb.conf in an editor.

2. Specify the configuration for each host in the cluster. In this example, there are two servers with three ranks on the first and two ranks on the second:

   host0.address = 172.123.45.67
   host0.public_address = 172.123.45.67
   host0.host_manager_public_url = http://172.123.45.67:9300
   host0.ram_limit = 6769400000
   host0.gpus =
   host0.accepts_failover = true
   
   host1.address = 172.123.45.68
   host1.public_address = 172.123.45.68
   host1.host_manager_public_url = http://172.123.45.68:9300
   host1.ram_limit = 6769400000
   host1.gpus =
   host1.accepts_failover = true
   
   rank0.host = host0
   rank1.host = host0
   rank2.host = host0
   rank3.host = host1
   rank4.host = host1
   

3. For CUDA builds, the GPUs need to be assigned to ranks. To display the installed GPUs and their status run:

   nvidia-smi
   

   If the program is not installed or doesn't run, see Nvidia Drivers.

   Once the number of GPUs on each server has been established, enter them into the configuration file by associated rank. In this example, there are two servers with a GPU assigned to each of two ranks per host (none for rank0):

   rank0.gpu = 0 # This GPU can be shared with a worker rank, typically rank 1.
   
   rank1.taskcalc_gpu = 0
   rank2.taskcalc_gpu = 1
   rank3.taskcalc_gpu = 0 # On new host, restart at 0
   rank4.taskcalc_gpu = 1
   

4. For non-CUDA builds, the Numa CPUs need to be assigned to ranks. To display the Numa nodes, run:

   numactl -H
   

   Once the number of Numa nodes on each server has been established, enter them into the configuration file by associated rank. In this example, there are two servers with a Numa node assigned to each of two ranks per host (none for rank0):

   rank0.numa_node =        # Preferring a node for the head node HTTP server is often not necessary.
   
   rank1.base_numa_node = 0
   rank2.base_numa_node = 1
   rank3.base_numa_node = 0 # On new host, restart at 0
   rank4.base_numa_node = 1
   
   rank1.data_numa_node = 0
   rank2.data_numa_node = 1
   rank3.data_numa_node = 0 # On new host, restart at 0
   rank4.data_numa_node = 1
   

5. Set the license key:

   license_key = ...
   

6. Optionally, enable the text search capability:

   enable_text_search = true
   

   Caution!

   Text search is required if KML usage is desired.

7. Determine the directory in which database files will be stored. It should meet the following criteria:

   • Available disk space that is 4x memory
   • Writable by the gpudb user
   • Consist of raided SSDs
   • Not be part of a network share or NFS mount

8. Enter the database file directory path into the configuration:

   persist_directory = /opt/gpudb/persist
   

   Important

   For additional configuration options, see the Configuration Reference.

9. Save the file.

10. Start the gpudb service. This will start all Kinetica processes on the head node, and if in a clustered environment, processes on the worker nodes:

    service gpudb start
    

11. Log into the admin application and change the administration account's default password.