Package com.gpudb.filesystem

Class GPUdbFileHandler

    • Field Detail

      • KIFS_PATH_SEPARATOR

        public static final String KIFS_PATH_SEPARATOR
        Separator character between a KiFS directory and KiFS file name. This can also be used within a file name to create a virtual directory within a path.
        See Also:
        Constant Field Values

      • KIFS_PATH_PREFIX

        public static final String KIFS_PATH_PREFIX
        Prefix to use when referencing KiFS files; e.g., for purposes for file ingest.
        See Also:
        Constant Field Values

      • REMOTE_USER_HOME_DIR_PREFIX

        public static final String REMOTE_USER_HOME_DIR_PREFIX
        Alias for a user's home directory.
        See Also:
        Constant Field Values

    • Constructor Detail

      • GPUdbFileHandler

        public GPUdbFileHandler​(GPUdb db,
                        GPUdbFileHandler.Options options)
        Constructs a GPUdbFileHandler object that allows the user to upload and download files.
        Parameters:
        db - The GPUdb instance used to access KiFS.
        options - Options for setting up the files for transfer.

    • Method Detail

      • getDefaultThreadPoolTerminationTimeout

        public static int getDefaultThreadPoolTerminationTimeout()

      • getDefaultFullFileDispatcherThreadpoolSize

        public static int getDefaultFullFileDispatcherThreadpoolSize()

      • upload

        public void upload​(String fileName,
                   String remoteDirName,
                   UploadOptions uploadOptions,
                   FileUploadListener callback)
            throws GPUdbException
        Uploads a single file from a given local path to a given KiFS directory using the given upload options and callback. The filename passed in is preserved between the client and the server.
        Parameters:
        fileName - Name and path of the file (or Glob pattern) to upload; e.g., "/home/user1/dir1/dir2/a.txt" or "/data/*.csv".
        remoteDirName - Name of the KiFS directory to upload to.
        uploadOptions - Various options to be used for uploading a file; use null for default options.
        callback - An instance of FileUploadListener, which is used to report the upload progress.
        Throws:
        GPUdbException - If an error occurs uploading the file(s) to the server.
        See Also:
        FileUploadListener, UploadOptions

      • upload

        public void upload​(List<String> fileNames,
                   String remoteDirName,
                   UploadOptions uploadOptions,
                   FileUploadListener callback)
            throws GPUdbException
        Uploads multiple files from the given local paths to a given KiFS directory. The filenames passed in are preserved between the client and the server.
        Parameters:
        fileNames - Names and paths of the files (or Glob patterns) to upload; e.g., "/home/user1/dir1/dir2/a.txt" or "/data/*.csv".
        remoteDirName - Name of the KiFS directory to upload to.
        uploadOptions - Various options to be used for uploading a file; use null for default options.
        callback - An instance of FileUploadListener, which is used to report the upload progress.
        Throws:
        GPUdbException - If an error occurs uploading the file(s) to the server.
        See Also:
        FileUploadListener, UploadOptions

      • download

        public void download​(List<String> fileNames,
                     String localDirName,
                     DownloadOptions downloadOptions,
                     FileDownloadListener callback)
              throws GPUdbException
        Downloads KiFS files with the given paths to a given local directory.
        Parameters:
        fileNames - Paths of the KiFS files to download.
        localDirName - Path of the writable local directory to download to.
        downloadOptions - Various options to be used for downloading a file; use null for default options.
        callback - An instance of FileDownloadListener, which is used to report the download progress.
        Throws:
        GPUdbException

      • download

        public void download​(String fileName,
                     String localDirName,
                     DownloadOptions downloadOptions,
                     FileDownloadListener callback)
              throws GPUdbException
        Downloads a KiFS file with the given path to a given local directory.
        Parameters:
        fileName - Paths of the KiFS file to download.
        localDirName - Path of the writable local directory to download to.
        downloadOptions - Various options to be used for downloading a file; use null for default options.
        callback - An instance of FileDownloadListener, which is used to report the download progress.
        Throws:
        GPUdbException

      • downloadDir

        public void downloadDir​(String remoteDirName,
                        String localDirName,
                        DownloadOptions downloadOptions,
                        FileDownloadListener callback)
                 throws GPUdbException
        Downloads all files in a given KiFS directory to a given local directory.
        Parameters:
        remoteDirName - - Path of the KiFS directory to download.
        localDirName - - Path of the local directory to download to.
        downloadOptions - Various options to be used for downloading a file; use null for default options.
        callback - An instance of FileDownloadListener, which is used to report the download progress.
        Throws:
        GPUdbException - If: 1. the remoteDirName is not existing 2. the localDirName is not existing 3. something goes wrong with the download
        See Also:
        DownloadOptions

      • ingest

        public void ingest​(List<String> fileNames,
                   String tableName,
                   IngestOptions ingestOptions,
                   TableCreationOptions createTableOptions)
            throws GPUdbException
        Uploads and ingests multiple files from the given local paths to a given table.
        Parameters:
        fileNames - Names and paths of the files (or Glob patterns) to ingest; e.g., "/home/user1/dir1/dir2/a.txt" or "/data/*.csv".
        tableName - Name of the table to ingest into.
        ingestOptions - Various options to be used for ingesting a file; use null for default options.
        createTableOptions - Various options to be used for creating the target table; use null for default options.
        Throws:
        GPUdbException - If the upload or ingest fails.
        See Also:
        IngestOptions, TableCreationOptions

      • deleteFiles

        public void deleteFiles​(List<String> fileNames)
                 throws GPUdbException
        Deletes the given list of fully-qualified KiFS file paths, suppressing the errors if the files are not found in KiFS.
        Parameters:
        fileNames - List of files in KiFS to delete.
        Throws:
        GPUdbException - If an error occurs deleting the files.

      • deleteFiles

        public void deleteFiles​(List<String> fileNames,
                        boolean noErrorIfNotExists)
                 throws GPUdbException
        Deletes the given list of fully-qualified KiFS file paths.
        Parameters:
        fileNames - List of files in KiFS to delete.
        noErrorIfNotExists - Whether or not to suppress errors if the given files are not found in KiFS.
        Throws:
        GPUdbException - If an error occurs deleting the files.

      • deleteFilesInDir

        public void deleteFilesInDir​(String remoteDirName)
                      throws GPUdbException
        Deletes the files in the given KiFS directory, suppressing the errors if the directory is not found in KiFS.
        Parameters:
        remoteDirName - Name of the KiFS directory containing files to delete.
        Throws:
        GPUdbException - If an error occurs deleting the files.

      • deleteFilesInDir

        public void deleteFilesInDir​(String remoteDirName,
                             boolean noErrorIfNotExists)
                      throws GPUdbException
        Deletes the files in the given KiFS directory.
        Parameters:
        remoteDirName - Name of the KiFS directory containing files to delete.
        noErrorIfNotExists - Whether or not to suppress errors if the given directory is not found in KiFS.
        Throws:
        GPUdbException - If an error occurs deleting the files.

      • showFiles

        public List<KifsFileInfo> showFiles​(List<String> remotePaths)
                             throws GPUdbException
        Returns statistics about the given KiFS files and/or directories. Wildcards "?" and "*" can be used to represent one character and zero or more characters, respectively.
        Parameters:
        remotePaths - List of KiFS files and/or directory names to report statistics on.
        Returns:
        List of KifsFileInfo objects, containing statistics about the given files.
        Throws:
        GPUdbException - If an error occurs looking up the files.

      • showDirectories

        @Deprecated(since="7.2.3",
            forRemoval=true)
public List<KifsDirectoryInfo> showDirectories​(Set<String> remoteDirNames,
                                               Map<String,​String> showDirectoryOptions)
                                        throws GPUdbException
        Deprecated, for removal: This API element is subject to removal in a future version.
        Returns statistics about the given KiFS directories.
        Parameters:
        remoteDirNames - Set of KiFS directory names to report statistics on.
        showDirectoryOptions - Unused.
        Returns:
        List of KifsDirectoryInfo objects, containing statistics about the given directories.
        Throws:
        GPUdbException - If an error occurs looking up the directories.

      • showDirectories

        public List<KifsDirectoryInfo> showDirectories​(List<String> remoteDirNames)
                                        throws GPUdbException
        Returns statistics about the given KiFS directories.
        Parameters:
        remoteDirNames - List of KiFS directory names.
        Returns:
        List of KifsDirectoryInfo objects, containing statistics about the given directories.
        Throws:
        GPUdbException - If an error occurs looking up the directories.

      • showDirectories

        public List<KifsDirectoryInfo> showDirectories​(List<String> remoteDirNames,
                                               Map<String,​String> showDirectoryOptions)
                                        throws GPUdbException
        Returns statistics about the given KiFS directories.
        Parameters:
        remoteDirNames - List of KiFS directory names.
        showDirectoryOptions - Reserved for future use.
        Returns:
        List of KifsDirectoryInfo objects, containing statistics about the given directories.
        Throws:
        GPUdbException - If an error occurs looking up the directories.

      • showDirectory

        public List<KifsDirectoryInfo> showDirectory​(String remoteDirName)
                                      throws GPUdbException
        Returns statistics about the given KiFS directory.
        Parameters:
        remoteDirName - Name of the KiFS directory.
        Returns:
        List of KifsDirectoryInfo objects, containing statistics about the given directory.
        Throws:
        GPUdbException - If an error occurs looking up the directory.

      • showDirectory

        public List<KifsDirectoryInfo> showDirectory​(String remoteDirName,
                                             Map<String,​String> showDirectoryOptions)
                                      throws GPUdbException
        Returns statistics about the given KiFS directory.
        Parameters:
        remoteDirName - Name of the KiFS directory.
        showDirectoryOptions - Reserved for future use.
        Returns:
        List of KifsDirectoryInfo objects, containing statistics about the given directory.
        Throws:
        GPUdbException - If an error occurs looking up the directory.

      • showAllDirectories

        public List<KifsDirectoryInfo> showAllDirectories​(Map<String,​String> showDirectoryOptions)
                                           throws GPUdbException
        Returns statistics about all KiFS directories.
        Parameters:
        showDirectoryOptions - Reserved for future use.
        Returns:
        List of KifsDirectoryInfo objects, containing statistics about all directories.
        Throws:
        GPUdbException - If an error occurs looking up the directories.

      • createDirectory

        public void createDirectory​(String remoteDirName)
                     throws GPUdbException
        Creates a KiFS directory with the given name, suppressing the error if the directory already exists in KiFS.
        Parameters:
        remoteDirName - Name of the KiFS directory to create.
        Throws:
        GPUdbException - If the directory creation fails.

      • createDirectory

        public void createDirectory​(String remoteDirName,
                            boolean noErrorIfExists)
                     throws GPUdbException
        Creates a KiFS directory with the given name.
        Parameters:
        remoteDirName - Name of the KiFS directory to create.
        noErrorIfExists - Whether or not to suppress an error if the given directory already exists in KiFS.
        Throws:
        GPUdbException - If the directory creation fails.

      • deleteDirectory

        public void deleteDirectory​(String remoteDirName)
                     throws GPUdbException
        Deletes the given KiFS directory and all files under it, suppressing the error if the directory is not found in KiFS.
        Parameters:
        remoteDirName - Name of the KiFS directory to delete.
        Throws:
        GPUdbException - If the directory deletion fails.
        See Also:
        deleteDirectory(String, boolean, boolean)

      • deleteDirectory

        public void deleteDirectory​(String remoteDirName,
                            boolean recursive,
                            boolean noErrorIfNotExists)
                     throws GPUdbException
        Deletes the given KiFS directory.
        Parameters:
        remoteDirName - Name of the KiFS directory to delete.
        recursive - Whether or not to delete all files and virtual subdirectories under remoteDirName as well.
        noErrorIfNotExists - Whether or not to suppress an error if the given directory does not exist in KiFS.
        Throws:
        GPUdbException - If the directory deletion fails.

      • kifsDirectoriesExist

        public boolean kifsDirectoriesExist​(Set<String> dirNames)
        Checks whether the given KiFS directories exist or not.
        Parameters:
        dirNames - The set of full KiFS directory paths to check for.
        Returns:
        Whether or not all of the given directories exist.

      • kifsDirectoryExists

        public boolean kifsDirectoryExists​(String dirName)
        Checks whether the given KiFS directory exists or not.

        Directory name may contain separators for nested directory checks.

        Parameters:
        dirName - The full KiFS directory path to check for.
        Returns:
        Whether or not the given directory exists.

      • kifsFileExists

        public boolean kifsFileExists​(String fileName)
        Checks whether the given KiFS file exists or not.
        Parameters:
        fileName - The full KiFS file path to check for.
        Returns:
        Whether or not the given file exists.