com.gpudb.filesystem

Class GPUdbFileHandler

java.lang.Object

com.gpudb.filesystem.GPUdbFileHandler

public class GPUdbFileHandler
extends Object

This is the main class which exposes the API to be used by the end users. This class exposes methods to upload e.g., (upload(String, String, UploadOptions, FileUploadListener), upload(List, String, UploadOptions, FileUploadListener)) and similarly for download. The entire filesystem API is exposed through this class and an instance of this class is enough to consume all functionalities provided by the KIFS. Apart from this class the other classes which could be used by the users of this API are: UploadOptions DownloadOptions FileUploadListener FileDownloadListener

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	GPUdbFileHandler.Options This class models the options available for modifying some behaviours of the GPUdbFileHandler class The two options available right now are: 1. threadPoolTerminationTimeout - in seconds 2. fullFileDispatcherThreadpoolSize - an integer

Constructor Summary

Constructors

Constructor and Description
GPUdbFileHandler(GPUdb db) Constructor using default GPUdbFileHandler.Options
GPUdbFileHandler(GPUdb db, GPUdbFileHandler.Options options) Constructs a GPUdbFileHandler object that allows the user to upload and download files.

Method Summary

Modifier and Type	Method and Description
void	createDirectory(String remoteDirName) This method will create a KIFS directory with default options
void	createDirectory(String remoteDirName, boolean noErrorIfExists) This method will create a KIFS directory with options as explained below.
void	deleteDirectory(String remoteDirName) This method deletes a KIFS directory.
void	deleteDirectory(String remoteDirName, boolean recursive, boolean noErrorIfNotExists) This method deletes a KIFS directory.
void	deleteFiles(List<String> fileNames) This method takes a list of fully qualified KIFS file paths and deletes the files in one go.
void	deleteFiles(List<String> fileNames, boolean noErrorIfNotExists) This method takes a list of fully qualified KIFS file paths and deletes the files in one go.
void	deleteFilesInDir(String remoteDirName) This method deletes files in the KIFS directory whose name is passed in using remoteDirName.
void	deleteFilesInDir(String remoteDirName, boolean noErrorIfNotExists) This method deletes files in the KIFS directory whose name is passes in using remoteDirName.
void	download(List<String> fileNames, String localDirName) This method downloads files from a KIFS directory to a local directory.
void	download(List<String> fileNames, String localDirName, DownloadOptions downloadOptions, FileDownloadListener callback) This method downloads files from a KIFS directory to a local directory.
void	download(String fileName, String localDirName) This method downloads a single file from a KIFS directory to a local directory.
void	download(String fileName, String localDirName, DownloadOptions downloadOptions, FileDownloadListener callback) This method downloads a single file from a KIFS directory to a local directory.
void	downloadDir(String remoteDirName, String localDirName, DownloadOptions downloadOptions, FileDownloadListener callback) This method will download all files in a KIFS directory to a directory on the local filesystem.
static int	getDefaultFullFileDispatcherThreadpoolSize()
static int	getDefaultThreadPoolTerminationTimeout()
GPUdbFileHandler.Options	getOptions() Returns the GPUdbFileHandler.Options instance
void	ingest(List<String> fileNames, String tableName, IngestOptions ingestOptions, TableCreationOptions createTableOptions) This method will ingest several files in one go.
boolean	kifsDirectoriesExist(Set<String> dirNames) This method checks whether KIFS paths exist or not.
boolean	kifsDirectoryExists(String dirName) This method checks whether a KIFS path exists or not.
boolean	kifsFileExists(String fileName) This method checks whether a KIFS file exists or not.
List<KifsDirectoryInfo>	showAllDirectories() This method retrieves the directory information all KIFS directories and returns the information as a list of KifsDirectoryInfo objects.
List<KifsDirectoryInfo>	showAllDirectories(Map<String,String> options) This method retrieves the directory information all KIFS directories and returns the information as a list of KifsDirectoryInfo objects.
List<KifsDirectoryInfo>	showDirectories(Set<String> remoteDirNames) This method retrieves the directory information for a given list of KIFS directories and returns the information as a list of KifsDirectoryInfo objects.
List<KifsDirectoryInfo>	showDirectories(Set<String> remoteDirNames, Map<String,String> options) This method retrieves the directory information for a given list of KIFS directories and returns the information as a list of KifsDirectoryInfo objects.
List<KifsDirectoryInfo>	showDirectory(String remoteDirName) This method retrieves the directory information for a given KIFS directory and returns the information as a list of KifsDirectoryInfo objects.
List<KifsDirectoryInfo>	showDirectory(String remoteDirName, Map<String,String> options) This method retrieves the directory information for a given KIFS directory and returns the information as a list of KifsDirectoryInfo objects.
List<KifsFileInfo>	showFiles(List<String> remoteDirNames) This method shows the files
void	upload(List<String> fileNames, String remoteDirName) This method facilitates uploading of multiple files from the same local directory to a given remote directory.
void	upload(List<String> fileNames, String remoteDirName, UploadOptions uploadOptions, FileUploadListener callback) This method facilitates uploading of multiple files from the same local directory to a given remote directory.
void	upload(String fileName, String remoteDirName) This method facilitates the upload of a single file from a given local directory to a given remote directory.
void	upload(String fileName, String remoteDirName, UploadOptions options, FileUploadListener callback) This method facilitates the upload of a single file from a given local directory to a given remote directory.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GPUdbFileHandler

public GPUdbFileHandler(GPUdb db,
                        GPUdbFileHandler.Options options)

Constructs a GPUdbFileHandler object that allows the user to upload and download files.

Parameters:

db - - GPUdb instance

options - - An instance of GPUdbFileHandler.Options class

GPUdbFileHandler

public GPUdbFileHandler(GPUdb db)

Constructor using default GPUdbFileHandler.Options

Parameters:

db - - the GPUdb instance

Method Detail

getDefaultThreadPoolTerminationTimeout

public static int getDefaultThreadPoolTerminationTimeout()

getDefaultFullFileDispatcherThreadpoolSize

public static int getDefaultFullFileDispatcherThreadpoolSize()

getOptions

public GPUdbFileHandler.Options getOptions()

Returns the GPUdbFileHandler.Options instance

upload

public void upload(String fileName,
                   String remoteDirName)
            throws GPUdbException

This method facilitates the upload of a single file from a given local directory to a given remote directory. The filename passed in is preserved between the client and the server. This method uses the default Upload options and does not use a callback.

Parameters:

fileName - - Name of the file along with the full path e.g., "/home/user1/dir1/dir2/a.txt"

remoteDirName - - Name of the remote directory to which the file is to be saved.

Throws:

GPUdbException - - from the method upload(List, String, UploadOptions, FileUploadListener)

See Also:

FileUploadListener, UploadOptions, UploadOptions.defaultOptions(), upload(String, String, UploadOptions, FileUploadListener)

upload

public void upload(String fileName,
                   String remoteDirName,
                   UploadOptions options,
                   FileUploadListener callback)
            throws GPUdbException

This method facilitates the upload of a single file from a given local directory to a given remote directory. The filename passed in is preserved between the client and the server.

Parameters:

fileName - - Name of the file along with the full path e.g., "/home/user1/dir1/dir2/a.txt"

remoteDirName - - Name of the remote directory to which the file is to be saved.

options - - UploadOptions - Various options to be used for uploading a file. If the value passed is null then default values would be used. The default values for the options can also be set explicitly using the method UploadOptions.defaultOptions().

callback - - An instance of FileUploadListener class which is used to report status of an ongoing/completed upload to the caller of the method.

Throws:

GPUdbException - - from the method upload(List, String, UploadOptions, FileUploadListener)

See Also:

FileUploadListener, UploadOptions

upload

public void upload(List<String> fileNames,
                   String remoteDirName)
            throws GPUdbException

This method facilitates uploading of multiple files from the same local directory to a given remote directory. The filenames are preserved between the client and the server. This method uses default Upload options and does not use a callback.

Parameters:

fileNames - - List - Names of the files to be uploaded including the full local path.

remoteDirName - - Name of the remote directory to which the file is to be saved. This indicates a KIFS directory

Throws:

GPUdbException - - thrown from upload(List, String, UploadOptions, FileUploadListener)

See Also:

FileUploadListener, UploadOptions, UploadOptions.defaultOptions(), upload(List, String, UploadOptions, FileUploadListener)

upload

public void upload(List<String> fileNames,
                   String remoteDirName,
                   UploadOptions uploadOptions,
                   FileUploadListener callback)
            throws GPUdbException

This method facilitates uploading of multiple files from the same local directory to a given remote directory. The filenames are preserved between the client and the server.

Parameters:

fileNames - - List - Names of the files to be uploaded including the full local path.

remoteDirName - - Name of the remote directory to which the file is to be saved. This indicates a KIFS directory

uploadOptions - - UploadOptions - Various uploadOptions to be used for uploading a file. If the value passed is null then default values would be used. The default values for the uploadOptions can also be set explicitly using the method UploadOptions.defaultOptions().

callback - - An instance of FileUploadListener class which is used to report status of an ongoing/completed upload to the caller of the method.

Throws:

GPUdbException - - Where parameters value checks fail

See Also:

FileUploadListener, UploadOptions

download

public void download(List<String> fileNames,
                     String localDirName)
              throws GPUdbException

This method downloads files from a KIFS directory to a local directory. The local directory must exist or else this method will throw an exception. The 'fileNames' argument contains a list of fully qualified file names on a KIFS directory. This method uses default download options and doesn't use a callback.

Parameters:

fileNames - - A list of fully qualified file names residing on the KIFS directory e.g., 'a/b/c/d.txt' where 'a/b/c' is a full path on the KIFS.

localDirName - - String - This is the name of a local directory where the files will be saved. This directory must exist on the local filesystem.

Throws:

GPUdbException

See Also:

DownloadOptions, DownloadOptions.defaultOptions(), FileDownloadListener

download

public void download(List<String> fileNames,
                     String localDirName,
                     DownloadOptions downloadOptions,
                     FileDownloadListener callback)
              throws GPUdbException

This method downloads files from a KIFS directory to a local directory. The local directory must exist or else this method will throw an exception. The 'fileNames' argument contains a list of fully qualified file names on a KIFS directory.

Parameters:

fileNames - - A list of fully qualified file names residing on the KIFS directory e.g., 'a/b/c/d.txt' where 'a/b/c' is a full path on the KIFS.

localDirName - - String - This is the name of a local directory where the files will be saved. This directory must exist on the local filesystem.

downloadOptions - - DownloadOptions - Options to be used for downloading files from KIFS to the local directory. if the value passed is null then default values would be used. The default values for the options can also be set explicitly using the method DownloadOptions.defaultOptions()

callback - - An instance of FileDownloadListener class which is used to report status of an ongoing/completed download to the caller of the method.

Throws:

GPUdbException

download

public void download(String fileName,
                     String localDirName)
              throws GPUdbException

This method downloads a single file from a KIFS directory to a local directory. The local directory must exist or else this method will throw an exception. The 'fileNames' argument contains a list of fully qualified file names on a KIFS directory.

Parameters:

fileName - - String - A fully qualified file name residing on the KIFS directory e.g., 'a/b/c/d.txt' where 'a/b/c' is a full path on the KIFS.

localDirName - - String - This is the name of a local directory where the files will be saved. This directory must exist on the local filesystem.

Throws:

GPUdbException

download

public void download(String fileName,
                     String localDirName,
                     DownloadOptions downloadOptions,
                     FileDownloadListener callback)
              throws GPUdbException

This method downloads a single file from a KIFS directory to a local directory. The local directory must exist or else this method will throw an exception. The 'fileNames' argument contains a list of fully qualified file names on a KIFS directory.

Parameters:

fileName - - String - A fully qualified file name residing on the KIFS directory e.g., 'a/b/c/d.txt' where 'a/b/c' is a full path on the KIFS.

localDirName - - String - This is the name of a local directory where the files will be saved. This directory must exist on the local filesystem.

downloadOptions - - DownloadOptions - Options to be used for downloading files from KIFS to the local directory. if the value passed is null then default values would be used. The default values for the options can also be set explicitly using the method DownloadOptions.defaultOptions()

Throws:

GPUdbException

downloadDir

public void downloadDir(String remoteDirName,
                        String localDirName,
                        DownloadOptions downloadOptions,
                        FileDownloadListener callback)
                 throws GPUdbException

This method will download all files in a KIFS directory to a directory on the local filesystem.

Parameters:

remoteDirName - - Name of the KIFS directory

localDirName - - Name of the local directory

downloadOptions - - A DownloadOptions object

Throws:

GPUdbException - - throws a GPUdbException if 1. the remoteDirName is not existing 2. the localDirName is not existing 3. something goes wrong with the download

See Also:

DownloadOptions

ingest

public void ingest(List<String> fileNames,
                   String tableName,
                   IngestOptions ingestOptions,
                   TableCreationOptions createTableOptions)
            throws GPUdbException

This method will ingest several files in one go. It will use the method upload(List, String, UploadOptions, FileUploadListener) to upload the files to the directory 'sys_temp' on the KIFS and use the method GPUdb.insertRecordsFromFiles(String, List, Map, Map, Map) to update the table data.

Parameters:

fileNames - - Names of the local files to be ingested

tableName - - Name of the table to ingest into

ingestOptions - - Options for ingestion

createTableOptions - - Options for table creation

Throws:

- - GPUdbException - if the ingestion or the underlying upload fails

GPUdbException

See Also:

IngestOptions, TableCreationOptions

deleteFiles

public void deleteFiles(List<String> fileNames)
                 throws GPUdbException

This method takes a list of fully qualified KIFS file paths and deletes the files in one go.

Parameters:

fileNames - - a list of files on the KIFS

Throws:

GPUdbException

deleteFiles

public void deleteFiles(List<String> fileNames,
                        boolean noErrorIfNotExists)
                 throws GPUdbException

This method takes a list of fully qualified KIFS file paths and deletes the files in one go.

Parameters:

fileNames - - a list of files on the KIFS

noErrorIfNotExists - - true indicates not to throw an error if the files are not found on KIFS and false otherwise

Throws:

GPUdbException

deleteFilesInDir

public void deleteFilesInDir(String remoteDirName)
                      throws GPUdbException

This method deletes files in the KIFS directory whose name is passed in using remoteDirName. This method uses default options

Parameters:

remoteDirName - - Name of the KIFS directory

Throws:

GPUdbException - - from deleteFilesInDir(String, boolean)

deleteFilesInDir

public void deleteFilesInDir(String remoteDirName,
                             boolean noErrorIfNotExists)
                      throws GPUdbException

This method deletes files in the KIFS directory whose name is passes in using remoteDirName.

Parameters:

remoteDirName - - Name of the KIFS directory

noErrorIfNotExists - - Indicates whether to allow deletion of files in non-existent KIFS directory or not.

Throws:

GPUdbException - - from downstream API calls

showFiles

public List<KifsFileInfo> showFiles(List<String> remoteDirNames)
                             throws GPUdbException

This method shows the files

Parameters:

remoteDirNames - - List of KIFS directory names

Returns:

- List of KifsFileInfo objects

Throws:

GPUdbException

showDirectories

public List<KifsDirectoryInfo> showDirectories(Set<String> remoteDirNames)
                                        throws GPUdbException

This method retrieves the directory information for a given list of KIFS directories and returns the information as a list of KifsDirectoryInfo objects.

Parameters:

remoteDirNames - - Set of KIFS directory names

Returns:

- list of KifsDirectoryInfo objects

Throws:

GPUdbException - - from showDirectories(Set, Map)

See Also:

showDirectories(Set, Map)

showDirectories

public List<KifsDirectoryInfo> showDirectories(Set<String> remoteDirNames,
                                               Map<String,String> options)
                                        throws GPUdbException

This method retrieves the directory information for a given list of KIFS directories and returns the information as a list of KifsDirectoryInfo objects.

Parameters:

remoteDirNames - - Set of names of the KIFS directories

options - - Map of String to String; the endpoint expects an empty map.

Returns:

- List of KifsDirectoryInfo objects.

Throws:

GPUdbException

showDirectory

public List<KifsDirectoryInfo> showDirectory(String remoteDirName)
                                      throws GPUdbException

This method retrieves the directory information for a given KIFS directory and returns the information as a list of KifsDirectoryInfo objects. This method uses default options.

Parameters:

remoteDirName - - Name of the KIFS directory, cannot be null or empty

Returns:

- List of KifsDirectoryInfo objects.

Throws:

GPUdbException

See Also:

showDirectory(String, Map)

showDirectory

public List<KifsDirectoryInfo> showDirectory(String remoteDirName,
                                             Map<String,String> options)
                                      throws GPUdbException

This method retrieves the directory information for a given KIFS directory and returns the information as a list of KifsDirectoryInfo objects.

Parameters:

remoteDirName - - Name of the KIFS directory, cannot be null or empty

options - - Map of String to String; the endpoint expects an empty map.

Returns:

- List of KifsDirectoryInfo objects.

Throws:

GPUdbException

showAllDirectories

public List<KifsDirectoryInfo> showAllDirectories()
                                           throws GPUdbException

This method retrieves the directory information all KIFS directories and returns the information as a list of KifsDirectoryInfo objects.

Returns:

- a list of KifsDirectoryInfo objects

Throws:

GPUdbException - - from showAllDirectories(Map)

See Also:

showAllDirectories(Map)

showAllDirectories

public List<KifsDirectoryInfo> showAllDirectories(Map<String,String> options)
                                           throws GPUdbException

This method retrieves the directory information all KIFS directories and returns the information as a list of KifsDirectoryInfo objects.

Parameters:

options - - Map of String to String; the endpoint expects an empty map.

Returns:

- List of KifsDirectoryInfo objects.

Throws:

GPUdbException - - from downstream API call

createDirectory

public void createDirectory(String remoteDirName)
                     throws GPUdbException

This method will create a KIFS directory with default options

Parameters:

remoteDirName - - String - Name of the KIFS directory to create

Throws:

GPUdbException - - from createDirectory(String, boolean)

createDirectory

public void createDirectory(String remoteDirName,
                            boolean noErrorIfExists)
                     throws GPUdbException

This method will create a KIFS directory with options as explained below.

Parameters:

remoteDirName - - String - Name of the KIFS directory to create

noErrorIfExists - - boolean - true means if the directory exists there will be no error thrown and false means an existing directory name will throw an error

Throws:

GPUdbException

deleteDirectory

public void deleteDirectory(String remoteDirName)
                     throws GPUdbException

This method deletes a KIFS directory. This method uses default options

Parameters:

remoteDirName - - Name of the KIFS directory

Throws:

GPUdbException - - from deleteDirectory(String, boolean, boolean)

See Also:

deleteDirectory(String, boolean, boolean)

deleteDirectory

public void deleteDirectory(String remoteDirName,
                            boolean recursive,
                            boolean noErrorIfNotExists)
                     throws GPUdbException

This method deletes a KIFS directory. It will do a recursive delete if such an option is given.

Parameters:

remoteDirName - - Name of the KIFS directory

recursive - - Indicates deletion of all sub-directories

noErrorIfNotExists - - If set to true will ignore if the directory does not exist

Throws:

GPUdbException

kifsDirectoriesExist

public boolean kifsDirectoriesExist(Set<String> dirNames)

This method checks whether KIFS paths exist or not.

Parameters:

dirNames - - the set of full KIFS paths

Returns:

- true if all the directories exist and false if one or more of them does not exist

kifsDirectoryExists

public boolean kifsDirectoryExists(String dirName)

This method checks whether a KIFS path exists or not.

Parameters:

dirName - - the full KIFS paths

Returns:

- true if the directory exists and false otherwise Also returns false if the GPUdb instance is null.

kifsFileExists

public boolean kifsFileExists(String fileName)

This method checks whether a KIFS file exists or not.

Parameters:

fileName - - the full KIFS path to a file

Returns:

- true if the file exists and false otherwise