sparseml.tensorflow_v1.datasets.classification package

Submodules

sparseml.tensorflow_v1.datasets.classification.cifar module

class sparseml.tensorflow_v1.datasets.classification.cifar.Cifar100DataSet(root: str, train: bool = True, image_size: int = 32, pre_resize_transforms: Optional[sparseml.tensorflow_v1.datasets.classification.imagefolder.SplitsTransforms] = SplitsTransforms(train=(<function preprocess_for_train>, ), val=(<function preprocess_for_eval>, )), post_resize_transforms: Optional[sparseml.tensorflow_v1.datasets.classification.imagefolder.SplitsTransforms] = SplitsTransforms(train=None, val=None), download: bool = True)[source]

Bases: sparseml.tensorflow_v1.datasets.classification.cifar.CifarDataSet

name_scope() → str[source]

Returns: the name scope the dataset should be built under in the graph

class sparseml.tensorflow_v1.datasets.classification.cifar.Cifar10DataSet(root: str, train: bool = True, image_size: int = 32, pre_resize_transforms: Optional[sparseml.tensorflow_v1.datasets.classification.imagefolder.SplitsTransforms] = SplitsTransforms(train=(<function preprocess_for_train>, ), val=(<function preprocess_for_eval>, )), post_resize_transforms: Optional[sparseml.tensorflow_v1.datasets.classification.imagefolder.SplitsTransforms] = SplitsTransforms(train=None, val=None), download: bool = True)[source]

Bases: sparseml.tensorflow_v1.datasets.classification.cifar.CifarDataSet

name_scope() → str[source]

Returns: the name scope the dataset should be built under in the graph

sparseml.tensorflow_v1.datasets.classification.imagefolder module

ImageFolder dataset implementations for the image classification field in computer vision.

class sparseml.tensorflow_v1.datasets.classification.imagefolder.ImageFolderDataset(root: str, train: bool, image_size: int = 224, pre_resize_transforms: Optional[sparseml.tensorflow_v1.datasets.classification.imagefolder.SplitsTransforms] = SplitsTransforms(train=(<function random_scaling_crop.<locals>.rand_crop>, <function random_flip_left_right>, <function random_flip_up_down>), val=None), post_resize_transforms: Optional[sparseml.tensorflow_v1.datasets.classification.imagefolder.SplitsTransforms] = SplitsTransforms(train=(<function imagenet_normalizer>, ), val=(<function center_square_crop.<locals>.cent_crop>, <function imagenet_normalizer>)))[source]

Bases: sparseml.tensorflow_v1.datasets.dataset.Dataset

Implementation for loading an image folder structure into a dataset.

Image folders should be of the form:

root/class_x/xxx.ext
root/class_x/xxy.ext
root/class_x/xxz.ext

root/class_y/123.ext
root/class_y/nsdf3.ext
root/class_y/asd932_.ext

Parameters

root – the root location for the dataset’s images to load
train – True to load the training dataset from the root, False for validation
image_size – the size of the image to reshape to
pre_resize_transforms – transforms to be applied before resizing the image
post_resize_transforms – transforms to be applied after resizing the image

creator()[source]

Returns: a created dataset that gives the file_path and label for each image under self.root

format_iterator_batch(iter_batch: Tuple[tensorflow.python.framework.ops.Tensor, …]) → Tuple[Dict[str, tensorflow.python.framework.ops.Tensor], Dict[str, tensorflow.python.framework.ops.Tensor]][source]

Parameters: iter_batch – the batch ref returned from the iterator
Returns: a tuple of image and label tensors

property image_size

the size of the images to resize to

Type: return

name_scope() → str[source]

Returns: the name scope the dataset should be built under in the graph

property num_classes

the number of classes found for the dataset

Type: return

property num_images

the number of images found for the dataset

Type: return

property post_resize_transforms

transforms to be applied after resizing the image

Type: return

property pre_resize_transforms

transforms to be applied before resizing the image

Type: return

processor(file_path: tensorflow.python.framework.ops.Tensor, label: tensorflow.python.framework.ops.Tensor)[source]

Parameters

file_path – the path to the file to load an image from
label – the label for the given image

Returns

a tuple containing the processed image and label

property root

the root location for the dataset’s images to load

Type: return

property train

True to load the training dataset from the root, False for validation

Type: return

class sparseml.tensorflow_v1.datasets.classification.imagefolder.SplitsTransforms(train, val)

Bases: tuple

property train: Alias for field number 0

property val: Alias for field number 1

sparseml.tensorflow_v1.datasets.classification.imagefolder.imagenet_normalizer(img)[source]

Normalize an image using mean and std of the imagenet dataset

Parameters: img – The input image to normalize
Returns: The normalized image

sparseml.tensorflow_v1.datasets.classification.imagenet module

Imagenet dataset implementations for the image classification field in computer vision. More info for the dataset can be found here.

class sparseml.tensorflow_v1.datasets.classification.imagenet.ImageNetDataset(root: str = '~/.cache/nm_datasets/imagenet', train: bool = True, image_size: int = 224)[source]

Bases: sparseml.tensorflow_v1.datasets.classification.imagefolder.ImageFolderDataset

ImageNet dataset implementation

Parameters

root – the root location for the dataset’s images to load
train – True to load the training dataset from the root, False for validation
image_size – the size of the image to reshape to

name_scope() → str[source]

Returns: the name scope the dataset should be built under in the graph

sparseml.tensorflow_v1.datasets.classification.imagenette module

Imagenette and Imagewoof dataset implementations for the image classification field in computer vision. More info for the dataset can be found here.

class sparseml.tensorflow_v1.datasets.classification.imagenette.ImagenetteDataset(root: str = '~/.cache/nm_datasets/imagenette', train: bool = True, dataset_size: sparseml.utils.datasets.imagenette.ImagenetteSize = <ImagenetteSize.s160: 's160'>, image_size: Optional[int] = None, download: bool = True)[source]

Bases: sparseml.tensorflow_v1.datasets.classification.imagefolder.ImageFolderDataset, sparseml.utils.datasets.imagenette.ImagenetteDownloader

Wrapper for the imagenette (10 class) dataset that fastai created. Handles downloading and applying standard transforms.

Parameters

root – The root folder to find the dataset at, if not found will download here if download=True
train – True if this is for the training distribution, False for the validation
dataset_size – The size of the dataset to use and download: See ImagenetteSize for options
image_size – The image size to output from the dataset
download – True to download the dataset, False otherwise

name_scope() → str[source]

Returns: the name scope the dataset should be built under in the graph

class sparseml.tensorflow_v1.datasets.classification.imagenette.ImagenetteSize(value)[source]

Bases: enum.Enum

Dataset size for Imagenette / Imagewoof. full does not resize the original dataset at all. s320 resizes the images to 320px. s160 resizes the images to 160px.

full = 'full'

s160 = 's160'

s320 = 's320'

class sparseml.tensorflow_v1.datasets.classification.imagenette.ImagewoofDataset(root: str = '~/.cache/nm_datasets/imagewoof', train: bool = True, dataset_size: sparseml.utils.datasets.imagenette.ImagenetteSize = <ImagenetteSize.s160: 's160'>, image_size: Optional[int] = None, download: bool = True)[source]

Bases: sparseml.tensorflow_v1.datasets.classification.imagefolder.ImageFolderDataset, sparseml.utils.datasets.imagenette.ImagewoofDownloader

Wrapper for the imagewoof (10 class) dataset that fastai created. Handles downloading and applying standard transforms. More info for the dataset can be found here <https://github.com/fastai/imagenette>

Parameters

root – The root folder to find the dataset at, if not found will download here if download=True
train – True if this is for the training distribution, False for the validation
rand_trans – True to apply RandomCrop and RandomHorizontalFlip to the data, False otherwise
dataset_size – The size of the dataset to use and download: See :py:func ~ImagewoofSize for options
image_size – The image size to output from the dataset
download – True to download the dataset, False otherwise

name_scope() → str[source]

Returns: the name scope the dataset should be built under in the graph

Module contents

Datasets related to image classification field in computer vision