Storage Base API¶

class libcloud.storage.base.StorageDriver(key, secret=None, secure=True, host=None, port=None, api_version=None, region=None, **kwargs)[source]¶

A base StorageDriver to derive from.

Parameters

key (str) – API key or username to be used (required)
secret (str) – Secret password to be used (required)
secure (bool) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.
host (str) – Override hostname used for connections.
port (int) – Override port used for connections.
api_version (str) – Optional API version. Only used by drivers which support multiple API versions.
region (str) – Optional driver region. Only used by drivers which support multiple regions.

Return type

None

connectionCls¶: alias of libcloud.common.base.ConnectionUserAndKey

create_container(container_name)[source]¶

Create a new container.

Parameters: container_name (str) – Container name.
Returns: Container instance on success.
Return type: libcloud.storage.base.Container

delete_container(container)[source]¶

Delete a container.

Parameters: container (libcloud.storage.base.Container) – Container instance
Returns: True on success, False otherwise.
Return type: bool

delete_object(obj)[source]¶

Delete an object.

Parameters: obj (libcloud.storage.base.Object) – Object instance.
Returns: bool True on success.
Return type: bool

download_object(obj, destination_path, overwrite_existing=False, delete_on_failure=True)[source]¶

Download an object to the specified destination path.

Parameters

obj (libcloud.storage.base.Object) – Object instance.
destination_path (str) – Full path to a file or a directory where the incoming file will be saved.
overwrite_existing (bool) – True to overwrite an existing file, defaults to False.
delete_on_failure (bool) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).

Returns

True if an object has been successfully downloaded, False otherwise.

Return type

bool

download_object_as_stream(obj, chunk_size=None)[source]¶

Return a iterator which yields object data.

Parameters

obj (libcloud.storage.base.Object) – Object instance
chunk_size (int) – Optional chunk size (in bytes).

Return type

iterator of bytes

download_object_range(obj, destination_path, start_bytes, end_bytes=None, overwrite_existing=False, delete_on_failure=True)[source]¶

Download part of an object.

Parameters

obj (libcloud.storage.base.Object) – Object instance.
destination_path (str) – Full path to a file or a directory where the incoming file will be saved.
start_bytes (int) – Start byte offset (inclusive) for the range download. Offset is 0 index based so the first byte in file file is “0”.
end_bytes (int) – End byte offset (non-inclusive) for the range download. If not provided, it will default to the end of the file.
overwrite_existing (bool) – True to overwrite an existing file, defaults to False.
delete_on_failure (bool) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).

Returns

True if an object has been successfully downloaded, False otherwise.

Return type

bool

download_object_range_as_stream(obj, start_bytes, end_bytes=None, chunk_size=None)[source]¶

Return a iterator which yields range / part of the object data.

Parameters

obj (libcloud.storage.base.Object) – Object instance
start_bytes (int) – Start byte offset (inclusive) for the range download. Offset is 0 index based so the first byte in file file is “0”.
end_bytes (int) – End byte offset (non-inclusive) for the range download. If not provided, it will default to the end of the file.
chunk_size (int) – Optional chunk size (in bytes).

Return type

iterator of bytes

enable_container_cdn(container)[source]¶

Enable container CDN.

Parameters: container (libcloud.storage.base.Container) – Container instance
Return type: bool

enable_object_cdn(obj)[source]¶

Enable object CDN.

Parameters: obj (libcloud.storage.base.Object) – Object instance
Return type: bool

get_container(container_name)[source]¶

Return a container instance.

Parameters: container_name (str) – Container name.
Returns: Container instance.
Return type: libcloud.storage.base.Container

get_container_cdn_url(container)[source]¶

Return a container CDN URL.

Parameters: container (libcloud.storage.base.Container) – Container instance
Returns: A CDN URL for this container.
Return type: str

get_object(container_name, object_name)[source]¶

Return an object instance.

Parameters

container_name (str) – Container name.
object_name (str) – Object name.

Returns

Object instance.

Return type

libcloud.storage.base.Object

get_object_cdn_url(obj)[source]¶

Return an object CDN URL.

Parameters: obj (libcloud.storage.base.Object) – Object instance
Returns: A CDN URL for this object.
Return type: str

iterate_container_objects(container, prefix=None, ex_prefix=None)[source]¶

Return a iterator of objects for the given container.

Parameters

container (libcloud.storage.base.Container) – Container instance
prefix (str) – Filter objects starting with a prefix.
ex_prefix (str) – (Deprecated.) Filter objects starting with a prefix.

Returns

A iterator of Object instances.

Return type

iterator of libcloud.storage.base.Object

iterate_containers()[source]¶

Return a iterator of containers for the given account

Returns: A iterator of Container instances.
Return type: iterator of libcloud.storage.base.Container

list_container_objects(container, prefix=None, ex_prefix=None)[source]¶

Return a list of objects for the given container.

Parameters

container (libcloud.storage.base.Container) – Container instance.
prefix (str) – Filter objects starting with a prefix.
ex_prefix (str) – (Deprecated.) Filter objects starting with a prefix.

Returns

A list of Object instances.

Return type

list of libcloud.storage.base.Object

list_containers()[source]¶

Return a list of containers.

Returns: A list of Container instances.
Return type: list of Container

upload_object(file_path, container, object_name, extra=None, verify_hash=True, headers=None)[source]¶

Upload an object currently located on a disk.

Parameters

file_path (str) – Path to the object on disk.
container (libcloud.storage.base.Container) – Destination container.
object_name (str) – Object name.
verify_hash (bool) – Verify hash
extra (dict) – Extra attributes (driver specific). (optional)
headers (dict) – (optional) Additional request headers, such as CORS headers. For example: headers = {‘Access-Control-Allow-Origin’: ‘http://mozilla.com’}

Return type

libcloud.storage.base.Object

upload_object_via_stream(iterator, container, object_name, extra=None, headers=None)[source]¶

Upload an object using an iterator.

If a provider supports it, chunked transfer encoding is used and you don’t need to know in advance the amount of data to be uploaded.

Otherwise if a provider doesn’t support it, iterator will be exhausted so a total size for data to be uploaded can be determined.

Note: Exhausting the iterator means that the whole data must be buffered in memory which might result in memory exhausting when uploading a very large object.

If a file is located on a disk you are advised to use upload_object function which uses fs.stat function to determine the file size and it doesn’t need to buffer whole object in the memory.

Parameters

iterator (object) – An object which implements the iterator interface.
container (libcloud.storage.base.Container) – Destination container.
object_name (str) – Object name.
extra (dict) – (optional) Extra attributes (driver specific). Note: This dictionary must contain a ‘content_type’ key which represents a content type of the stored object.
headers (dict) – (optional) Additional request headers, such as CORS headers. For example: headers = {‘Access-Control-Allow-Origin’: ‘http://mozilla.com’}

Return type

libcloud.storage.base.Object

class libcloud.storage.base.Container(name, extra, driver)[source]¶

Represents a container (bucket) which can hold multiple objects.

Parameters

name (str) – Container name (must be unique).
extra (dict) – Extra attributes.
driver (libcloud.storage.base.StorageDriver) – StorageDriver instance.

class libcloud.storage.base.Object(name, size, hash, extra, meta_data, container, driver)[source]¶

Represents an object (BLOB).

Parameters

name (str) – Object name (must be unique per container).
size (int) – Object size in bytes.
hash (str) – Object hash.
container (libcloud.storage.base.Container) – Object container.
extra (dict) – Extra attributes.
meta_data (dict) – Optional object meta data.
driver (libcloud.storage.base.StorageDriver) – StorageDriver instance.

Error Classes¶

class libcloud.storage.types.ContainerError(value, driver, container_name)[source]¶

class libcloud.storage.types.ContainerAlreadyExistsError(value, driver, container_name)[source]¶

class libcloud.storage.types.ContainerDoesNotExistError(value, driver, container_name)[source]¶

class libcloud.storage.types.ContainerIsNotEmptyError(value, driver, container_name)[source]¶

class libcloud.storage.types.ObjectDoesNotExistError(value, driver, object_name)[source]¶

class libcloud.storage.types.ObjectHashMismatchError(value, driver, object_name)[source]¶

class libcloud.storage.types.InvalidContainerNameError(value, driver, container_name)[source]¶