Storage Module

This guide will show you how to import, configure and use Tru AI Storage Module, to read and write file to Tru AI Storage.

Configuration

To start using the Storage module you need first to import the package

  from gjirafatech.truai.storage import Storage

After importing the package you have to initialize an object of the Storage class by using an ApiKey.

To create an ApiKey, go to https://aihub.gjirafa.tech/settings, and create an ApiKey with at least Editor permission.

Then copy the key and use it as in the example below:

  api_key = "00000000-0000-0000-0000-000000000000"
storage = Storage(api_key)

List objects

list_objects(prefix="", recursive=False, last_key=None)

Lists all object metadata from Storage.

  page1 = storage.list_objects(prefix="directory/data")
page2 = storage.list_objects(prefix="directory/data", last_key=page1[-1])

Parameters

Param	Type	Description	Required
prefix	str	Object key starts with prefix	False
recursive	bool	List recursively through directory structure	False
last_key	str	List objects after this key name	False

Get object

get_object(key, encoding="utf-8")

Gets an object from Storage and returns it as a stream.

  stream = storage.get_object(key="directory/file.ext")

Parameters

Param	Type	Description	Required
key	str	Object name in the storage	True
encoding	str	Encoding type	False

Get directory

get_directory(prefix, directory)

Downloads all objects from Storage and saves to the specified directory.

  storage.get_directory(prefix="directory/", directory="/home/jovyan/data/")

Parameters

Param	Type	Description	Required
prefix	str	Object key starts with prefix	True
directory	str	Local directory	True

Put object

put_object(key, content, content_type="application/octet-stream")

Uploads the `content` object to Storage under the name `key`

  f = open("directory/file.txt", "r")
storage.put_object(key="directory/file.txt", content=f, content_type="text/plain")

Parameters

Param	Type	Description	Required
key	str	Object name in the storage	True
content	object	An object having callable read() returning bytes object.	True
content_type	str	Content type of the object	False

Put directory

put_directory(prefix, directory)

Uploads all files from specified directory to the Storage

  storage.put_directory(directory="/home/jovyan/data/", prefix="directory/")

Parameters

Param	Type	Description	Required
directory	str	Local directory	True
prefix	str	Object key starts with prefix	True

Move objects

move_objects(keys, prefixes)

Moves keys/prefixes to the new keys/prefixes

  #move keys
storage.move_objects(
    keys={
        "directory/file.ext": "directory/data.ext",
        "directory/sample.csv": "raw-data/sample.csv",
    }
)

#move by prefix
storage.move_objects(
    prefixes={
        "directory1/": "directory2/",
        "raw-data/": "data/",
    }
)

Parameters

Param	Type	Description	Required
keys	dict	Dictionary with keys as object key names, and values as new object key names	False
prefixes	dict	Dictionary with keys as prefix, and values as new prefix	False

Note: At least one of keys or prefixes should be passed

Delete objects

delete_objects(keys, prefixes)

Deletes keys/prefixes from Storage

  # delete keys
storage.delete_objects(keys=["directory/file.ext", "directory/sample.csv"])

# delete by prefix
storage.delete_objects(prefixes=["directory/", "raw-data/"])

Parameters

Param	Type	Description	Required
keys	list	List of keys to delete	False
prefixes	list	List of prefixes to delete	False

Note: At least one of keys or prefixes should be passed