Skip to main content

Class GPUdbFileHandler

java.lang.Object
com.gpudb.filesystem.GPUdbFileHandler

public class GPUdbFileHandler extends Object
This is the main class which exposes the API to be used by the end users. This class exposes methods to upload e.g., (upload(String, String, UploadOptions, FileUploadListener), upload(List, String, UploadOptions, FileUploadListener)) and similarly for download.

The entire filesystem API is exposed through this class and an instance of this class is enough to consume all functionalities provided by the KiFS. *

Supported file path patterns include standard Glob syntax (e.g. *.csv, data/**, file_?.txt).

Apart from this class the other classes which could be used by the users of this API are: UploadOptions DownloadOptions FileUploadListener FileDownloadListener

  • Field Details

    • KIFS_PATH_SEPARATOR

      public static final String KIFS_PATH_SEPARATOR
      Separator character between a KiFS directory and KiFS file name. This can also be used within a file name to create a virtual directory within a path.
    • KIFS_PATH_PREFIX

      public static final String KIFS_PATH_PREFIX
      Prefix to use when referencing KiFS files; e.g., for purposes for file ingest.
    • REMOTE_USER_HOME_DIR_PREFIX

      public static final String REMOTE_USER_HOME_DIR_PREFIX
      Alias for a user’s home directory.
  • Constructor Details

  • Method Details

    • getDefaultThreadPoolTerminationTimeout

      public static int getDefaultThreadPoolTerminationTimeout()
    • getDefaultFullFileDispatcherThreadpoolSize

      public static int getDefaultFullFileDispatcherThreadpoolSize()
    • getOptions

      public GPUdbFileHandler.Options getOptions()
      Returns the GPUdbFileHandler.Options instance
    • upload

      public void upload(String fileName, String remoteDirName) throws GPUdbException
      Uploads a single file from a given local path to a given KiFS directory. The filename passed in is preserved between the client and the server. This method uses the default upload options and does not use a callback.
      Parameters:
      fileName - Name and path of the file (or Glob pattern) to upload; e.g., “/home/user1/dir1/dir2/a.txt” or “/data/*.csv”.
      remoteDirName - Name of the KiFS directory to upload to.
      Throws:
      GPUdbException - If an error occurs uploading the file(s) to the server.
      See Also:
    • upload

      public void upload(String fileName, String remoteDirName, UploadOptions uploadOptions, FileUploadListener callback) throws GPUdbException
      Uploads a single file from a given local path to a given KiFS directory using the given upload options and callback. The filename passed in is preserved between the client and the server.
      Parameters:
      fileName - Name and path of the file (or Glob pattern) to upload; e.g., “/home/user1/dir1/dir2/a.txt” or “/data/*.csv”.
      remoteDirName - Name of the KiFS directory to upload to.
      uploadOptions - Various options to be used for uploading a file; use null for default options.
      callback - An instance of FileUploadListener, which is used to report the upload progress.
      Throws:
      GPUdbException - If an error occurs uploading the file(s) to the server.
      See Also:
    • upload

      public void upload(List<String> fileNames, String remoteDirName) throws GPUdbException
      Uploads multiple files from the given local paths to a given KiFS directory. The filenames passed in are preserved between the client and the server. This method uses the default upload options and does not use a callback.
      Parameters:
      fileNames - Names and paths of the files (or Glob patterns) to upload; e.g., “/home/user1/dir1/dir2/a.txt” or “/data/*.csv”.
      remoteDirName - Name of the KiFS directory to upload to.
      Throws:
      GPUdbException - If an error occurs uploading the file(s) to the server.
      See Also:
    • upload

      public void upload(List<String> fileNames, String remoteDirName, UploadOptions uploadOptions, FileUploadListener callback) throws GPUdbException
      Uploads multiple files from the given local paths to a given KiFS directory. The filenames passed in are preserved between the client and the server.
      Parameters:
      fileNames - Names and paths of the files (or Glob patterns) to upload; e.g., “/home/user1/dir1/dir2/a.txt” or “/data/*.csv”.
      remoteDirName - Name of the KiFS directory to upload to.
      uploadOptions - Various options to be used for uploading a file; use null for default options.
      callback - An instance of FileUploadListener, which is used to report the upload progress.
      Throws:
      GPUdbException - If an error occurs uploading the file(s) to the server. The exception message contains all collected error messages from failed uploads.
      See Also:
    • download

      public void download(List<String> fileNames, String localDirName) throws GPUdbException
      Downloads KiFS files with the given paths to a given local directory. This method uses the default download options and does not use a callback.
      Parameters:
      fileNames - Paths of the KiFS files to download.
      localDirName - Path of the writable local directory to download to.
      Throws:
      See Also:
    • download

      public void download(List<String> fileNames, String localDirName, DownloadOptions downloadOptions, FileDownloadListener callback) throws GPUdbException
      Downloads KiFS files with the given paths to a given local directory.
      Parameters:
      fileNames - Paths of the KiFS files to download.
      localDirName - Path of the writable local directory to download to.
      downloadOptions - Various options to be used for downloading a file; use null for default options.
      callback - An instance of FileDownloadListener, which is used to report the download progress.
      Throws:
    • download

      public void download(String fileName, String localDirName) throws GPUdbException
      Downloads a KiFS file with the given path to a given local directory. This method uses the default download options and does not use a callback.
      Parameters:
      fileName - Paths of the KiFS file to download.
      localDirName - Path of the writable local directory to download to.
      Throws:
      See Also:
    • download

      public void download(String fileName, String localDirName, DownloadOptions downloadOptions, FileDownloadListener callback) throws GPUdbException
      Downloads a KiFS file with the given path to a given local directory.
      Parameters:
      fileName - Paths of the KiFS file to download.
      localDirName - Path of the writable local directory to download to.
      downloadOptions - Various options to be used for downloading a file; use null for default options.
      callback - An instance of FileDownloadListener, which is used to report the download progress.
      Throws:
    • downloadDir

      public void downloadDir(String remoteDirName, String localDirName, DownloadOptions downloadOptions, FileDownloadListener callback) throws GPUdbException
      Downloads all files in a given KiFS directory to a given local directory.
      Parameters:
      remoteDirName - - Path of the KiFS directory to download.
      localDirName - - Path of the local directory to download to.
      downloadOptions - Various options to be used for downloading a file; use null for default options.
      callback - An instance of FileDownloadListener, which is used to report the download progress.
      Throws:
      GPUdbException - If: 1. the remoteDirName is not existing 2. the localDirName is not existing 3. something goes wrong with the download
      See Also:
    • ingest

      public void ingest(List<String> fileNames, String tableName, IngestOptions ingestOptions, TableCreationOptions createTableOptions) throws GPUdbException
      Uploads and ingests multiple files from the given local paths to a given table.
      Parameters:
      fileNames - Names and paths of the files (or Glob patterns) to ingest; e.g., “/home/user1/dir1/dir2/a.txt” or “/data/*.csv”.
      tableName - Name of the table to ingest into.
      ingestOptions - Various options to be used for ingesting a file; use null for default options.
      createTableOptions - Various options to be used for creating the target table; use null for default options.
      Throws:
      GPUdbException - If the upload or ingest fails.
      See Also:
    • deleteFiles

      public void deleteFiles(List<String> fileNames) throws GPUdbException
      Deletes the given list of fully-qualified KiFS file paths, suppressing the errors if the files are not found in KiFS.
      Parameters:
      fileNames - List of files in KiFS to delete.
      Throws:
      GPUdbException - If an error occurs deleting the files.
    • deleteFiles

      public void deleteFiles(List<String> fileNames, boolean noErrorIfNotExists) throws GPUdbException
      Deletes the given list of fully-qualified KiFS file paths.
      Parameters:
      fileNames - List of files in KiFS to delete.
      noErrorIfNotExists - Whether or not to suppress errors if the given files are not found in KiFS.
      Throws:
      GPUdbException - If an error occurs deleting the files.
    • deleteFilesInDir

      public void deleteFilesInDir(String remoteDirName) throws GPUdbException
      Deletes the files in the given KiFS directory, suppressing the errors if the directory is not found in KiFS.
      Parameters:
      remoteDirName - Name of the KiFS directory containing files to delete.
      Throws:
      GPUdbException - If an error occurs deleting the files.
    • deleteFilesInDir

      public void deleteFilesInDir(String remoteDirName, boolean noErrorIfNotExists) throws GPUdbException
      Deletes the files in the given KiFS directory.
      Parameters:
      remoteDirName - Name of the KiFS directory containing files to delete.
      noErrorIfNotExists - Whether or not to suppress errors if the given directory is not found in KiFS.
      Throws:
      GPUdbException - If an error occurs deleting the files.
    • showFiles

      public List<KifsFileInfo> showFiles(List<String> remotePaths) throws GPUdbException
      Returns statistics about the given KiFS files and/or directories. Wildcards ”?” and ”*” can be used to represent one character and zero or more characters, respectively.
      Parameters:
      remotePaths - List of KiFS files and/or directory names to report statistics on.
      Returns:
      List of KifsFileInfo objects, containing statistics about the given files.
      Throws:
      GPUdbException - If an error occurs looking up the files.
    • showDirectories

      @Deprecated(since=“7.2.3”, forRemoval=true) public List<KifsDirectoryInfo> showDirectories(Set<String> remoteDirNames) throws GPUdbException
      Deprecated, for removal: This API element is subject to removal in a future version.
      Returns statistics about the given KiFS directories.
      Parameters:
      remoteDirNames - Set of KiFS directory names to report statistics on.
      Returns:
      List of KifsDirectoryInfo objects, containing statistics about the given directories.
      Throws:
      GPUdbException - If an error occurs looking up the directories.
      See Also:
    • showDirectories

      @Deprecated(since=“7.2.3”, forRemoval=true) public List<KifsDirectoryInfo> showDirectories(Set<String> remoteDirNames, Map<String,String> showDirectoryOptions) throws GPUdbException
      Deprecated, for removal: This API element is subject to removal in a future version.
      Returns statistics about the given KiFS directories.
      Parameters:
      remoteDirNames - Set of KiFS directory names to report statistics on.
      showDirectoryOptions - Unused.
      Returns:
      List of KifsDirectoryInfo objects, containing statistics about the given directories.
      Throws:
      GPUdbException - If an error occurs looking up the directories.
    • showDirectories

      public List<KifsDirectoryInfo> showDirectories(List<String> remoteDirNames) throws GPUdbException
      Returns statistics about the given KiFS directories.
      Parameters:
      remoteDirNames - List of KiFS directory names.
      Returns:
      List of KifsDirectoryInfo objects, containing statistics about the given directories.
      Throws:
      GPUdbException - If an error occurs looking up the directories.
    • showDirectories

      public List<KifsDirectoryInfo> showDirectories(List<String> remoteDirNames, Map<String,String> showDirectoryOptions) throws GPUdbException
      Returns statistics about the given KiFS directories.
      Parameters:
      remoteDirNames - List of KiFS directory names.
      showDirectoryOptions - Reserved for future use.
      Returns:
      List of KifsDirectoryInfo objects, containing statistics about the given directories.
      Throws:
      GPUdbException - If an error occurs looking up the directories.
    • showDirectory

      public List<KifsDirectoryInfo> showDirectory(String remoteDirName) throws GPUdbException
      Returns statistics about the given KiFS directory.
      Parameters:
      remoteDirName - Name of the KiFS directory.
      Returns:
      List of KifsDirectoryInfo objects, containing statistics about the given directory.
      Throws:
      GPUdbException - If an error occurs looking up the directory.
    • showDirectory

      public List<KifsDirectoryInfo> showDirectory(String remoteDirName, Map<String,String> showDirectoryOptions) throws GPUdbException
      Returns statistics about the given KiFS directory.
      Parameters:
      remoteDirName - Name of the KiFS directory.
      showDirectoryOptions - Reserved for future use.
      Returns:
      List of KifsDirectoryInfo objects, containing statistics about the given directory.
      Throws:
      GPUdbException - If an error occurs looking up the directory.
    • showAllDirectories

      public List<KifsDirectoryInfo> showAllDirectories() throws GPUdbException
      Returns statistics about all KiFS directories.
      Returns:
      List of KifsDirectoryInfo objects, containing statistics about all directories.
      Throws:
      GPUdbException - If an error occurs looking up the directories.
    • showAllDirectories

      public List<KifsDirectoryInfo> showAllDirectories(Map<String,String> showDirectoryOptions) throws GPUdbException
      Returns statistics about all KiFS directories.
      Parameters:
      showDirectoryOptions - Reserved for future use.
      Returns:
      List of KifsDirectoryInfo objects, containing statistics about all directories.
      Throws:
      GPUdbException - If an error occurs looking up the directories.
    • createDirectory

      public void createDirectory(String remoteDirName) throws GPUdbException
      Creates a KiFS directory with the given name, suppressing the error if the directory already exists in KiFS.
      Parameters:
      remoteDirName - Name of the KiFS directory to create.
      Throws:
      GPUdbException - If the directory creation fails.
    • createDirectory

      public void createDirectory(String remoteDirName, boolean noErrorIfExists) throws GPUdbException
      Creates a KiFS directory with the given name.
      Parameters:
      remoteDirName - Name of the KiFS directory to create.
      noErrorIfExists - Whether or not to suppress an error if the given directory already exists in KiFS.
      Throws:
      GPUdbException - If the directory creation fails.
    • deleteDirectory

      public void deleteDirectory(String remoteDirName) throws GPUdbException
      Deletes the given KiFS directory and all files under it, suppressing the error if the directory is not found in KiFS.
      Parameters:
      remoteDirName - Name of the KiFS directory to delete.
      Throws:
      GPUdbException - If the directory deletion fails.
      See Also:
    • deleteDirectory

      public void deleteDirectory(String remoteDirName, boolean recursive, boolean noErrorIfNotExists) throws GPUdbException
      Deletes the given KiFS directory.
      Parameters:
      remoteDirName - Name of the KiFS directory to delete.
      recursive - Whether or not to delete all files and virtual subdirectories under remoteDirName as well.
      noErrorIfNotExists - Whether or not to suppress an error if the given directory does not exist in KiFS.
      Throws:
      GPUdbException - If the directory deletion fails.
    • kifsDirectoriesExist

      public boolean kifsDirectoriesExist(Set<String> dirNames)
      Checks whether the given KiFS directories exist or not.
      Parameters:
      dirNames - The set of full KiFS directory paths to check for.
      Returns:
      Whether or not all of the given directories exist.
    • kifsDirectoryExists

      public boolean kifsDirectoryExists(String dirName)
      Checks whether the given KiFS directory exists or not.

      Directory name may contain separators for nested directory checks.

      Parameters:
      dirName - The full KiFS directory path to check for.
      Returns:
      Whether or not the given directory exists.
    • kifsFileExists

      public boolean kifsFileExists(String fileName)
      Checks whether the given KiFS file exists or not.
      Parameters:
      fileName - The full KiFS file path to check for.
      Returns:
      Whether or not the given file exists.