sparseml.keras.datasets package

Subpackages

• sparseml.keras.datasets.classification package
  • Submodules
  • sparseml.keras.datasets.classification.imagefolder module
  • sparseml.keras.datasets.classification.imagenet module
  • sparseml.keras.datasets.classification.imagenette module
  • Module contents

Submodules

sparseml.keras.datasets.dataset module

General dataset implementations for Keras

class sparseml.keras.datasets.dataset.Dataset

Bases: object

Generic dataset implementation for Keras. Expected to work with the tensorflow.data APIs

build(batch_size: int, repeat_count: Optional[int] = None, shuffle_buffer_size: Optional[int] = None, prefetch_buffer_size: Optional[int] = None, num_parallel_calls: Optional[int] = None) → tensorflow.python.data.ops.dataset_ops.DatasetV2

Create the dataset in the current graph using tensorflow.data APIs :param batch_size: the batch size to create the dataset for :param repeat_count: the number of times to repeat the dataset,

if unset or None, will repeat indefinitely

Parameters

• shuffle_buffer_size – None if not shuffling, otherwise the size of the buffer to use for shuffling data

• prefetch_buffer_size – None if not prefetching, otherwise the size of the buffer to use for buffering

• num_parallel_calls – the number of parallel calls to run the processor function with

Returns

a tensorflow.data.Dataset instance

abstract creator() → tensorflow.python.data.ops.dataset_ops.DatasetV2

Implemented by sub classes to create a tensorflow.data dataset for the given impl.

Returns

a created tensorflow.data dataset

abstract processor(*args, **kwargs)

Implemented by sub classes to parallelize and map processing functions for loading the data of the dataset into memory. :param args: generic inputs for processing :param kwargs: generic inputs for processing :return: the processed tensors

sparseml.keras.datasets.helpers module

General utilities for dataset implementations for Keras

sparseml.keras.datasets.helpers.random_scaling_crop(scale_range: Tuple[int, int] = (0.8, 1.0), ratio_range: Tuple[int, int] = (0.75, 1.3333333333333333))

Random crop implementation which also randomly scales the crop taken as well as the aspect ratio of the crop. :param scale_range: the (min, max) of the crop scales to take from the orig image :param ratio_range: the (min, max) of the aspect ratios to take from the orig image :return: the callable function for random scaling crop op,

takes in the image and outputs randomly cropped image

sparseml.keras.datasets.registry module

Code related to the Keras dataset registry for easily creating datasets.

class sparseml.keras.datasets.registry.DatasetRegistry

Bases: object

Registry class for creating datasets

static attributes(key: str) → Dict[str, Any]

Parameters

key – the dataset key (name) to create

Returns

the specified attributes for the dataset

static create(key: str, *args, **kwargs)

Create a new dataset for the given key :param key: the dataset key (name) to create :return: the instantiated model

static register(key: Union[str, List[str]], attributes: Dict[str, Any])

Register a dataset with the registry. Should be used as a decorator :param key: the model key (name) to create :param attributes: the specified attributes for the dataset :return: the decorator

Module contents

Code for creating and loading datasets in Keras