sparseml.keras.datasets package


sparseml.keras.datasets.dataset module

General dataset implementations for Keras

class sparseml.keras.datasets.dataset.Dataset[source]

Bases: object

Generic dataset implementation for Keras. Expected to work with the APIs

build(batch_size: int, repeat_count: Optional[int] = None, shuffle_buffer_size: Optional[int] = None, prefetch_buffer_size: Optional[int] = None, num_parallel_calls: Optional[int] = None)[source]

Create the dataset in the current graph using APIs :param batch_size: the batch size to create the dataset for :param repeat_count: the number of times to repeat the dataset,

if unset or None, will repeat indefinitely

  • shuffle_buffer_size – None if not shuffling, otherwise the size of the buffer to use for shuffling data

  • prefetch_buffer_size – None if not prefetching, otherwise the size of the buffer to use for buffering

  • num_parallel_calls – the number of parallel calls to run the processor function with


a instance

abstract creator()[source]

Implemented by sub classes to create a dataset for the given impl.


a created dataset

abstract processor(*args, **kwargs)[source]

Implemented by sub classes to parallelize and map processing functions for loading the data of the dataset into memory. :param args: generic inputs for processing :param kwargs: generic inputs for processing :return: the processed tensors

sparseml.keras.datasets.helpers module

General utilities for dataset implementations for Keras

sparseml.keras.datasets.helpers.random_scaling_crop(scale_range: Tuple[int, int] = (0.8, 1.0), ratio_range: Tuple[int, int] = (0.75, 1.3333333333333333))[source]

Random crop implementation which also randomly scales the crop taken as well as the aspect ratio of the crop. :param scale_range: the (min, max) of the crop scales to take from the orig image :param ratio_range: the (min, max) of the aspect ratios to take from the orig image :return: the callable function for random scaling crop op,

takes in the image and outputs randomly cropped image

sparseml.keras.datasets.registry module

Code related to the Keras dataset registry for easily creating datasets.

class sparseml.keras.datasets.registry.DatasetRegistry[source]

Bases: object

Registry class for creating datasets

static attributes(key: str)Dict[str, Any][source]

key – the dataset key (name) to create


the specified attributes for the dataset

static create(key: str, *args, **kwargs)[source]

Create a new dataset for the given key :param key: the dataset key (name) to create :return: the instantiated model

static register(key: Union[str, List[str]], attributes: Dict[str, Any])[source]

Register a dataset with the registry. Should be used as a decorator :param key: the model key (name) to create :param attributes: the specified attributes for the dataset :return: the decorator

Module contents

Code for creating and loading datasets in Keras