sparseml.tensorflow_v1.datasets.classification package

Submodules

sparseml.tensorflow_v1.datasets.classification.cifar module

class sparseml.tensorflow_v1.datasets.classification.cifar.Cifar100DataSet(root: str, train: bool = True, image_size: int = 32, pre_resize_transforms: Optional[sparseml.tensorflow_v1.datasets.classification.imagefolder.SplitsTransforms] = SplitsTransforms(train=(<function preprocess_for_train>, ), val=(<function preprocess_for_eval>, )), post_resize_transforms: Optional[sparseml.tensorflow_v1.datasets.classification.imagefolder.SplitsTransforms] = SplitsTransforms(train=None, val=None), download: bool = True)

Bases: sparseml.tensorflow_v1.datasets.classification.cifar.CifarDataSet

name_scope() → str

Returns

the name scope the dataset should be built under in the graph

class sparseml.tensorflow_v1.datasets.classification.cifar.Cifar10DataSet(root: str, train: bool = True, image_size: int = 32, pre_resize_transforms: Optional[sparseml.tensorflow_v1.datasets.classification.imagefolder.SplitsTransforms] = SplitsTransforms(train=(<function preprocess_for_train>, ), val=(<function preprocess_for_eval>, )), post_resize_transforms: Optional[sparseml.tensorflow_v1.datasets.classification.imagefolder.SplitsTransforms] = SplitsTransforms(train=None, val=None), download: bool = True)

Bases: sparseml.tensorflow_v1.datasets.classification.cifar.CifarDataSet

name_scope() → str

Returns

the name scope the dataset should be built under in the graph

sparseml.tensorflow_v1.datasets.classification.imagefolder module

ImageFolder dataset implementations for the image classification field in computer vision.

class sparseml.tensorflow_v1.datasets.classification.imagefolder.ImageFolderDataset(root: str, train: bool, image_size: int = 224, pre_resize_transforms: Optional[sparseml.tensorflow_v1.datasets.classification.imagefolder.SplitsTransforms] = SplitsTransforms(train=(<function random_scaling_crop.<locals>.rand_crop>, <function random_flip_left_right>, <function random_flip_up_down>), val=None), post_resize_transforms: Optional[sparseml.tensorflow_v1.datasets.classification.imagefolder.SplitsTransforms] = SplitsTransforms(train=(<function imagenet_normalizer>, ), val=(<function center_square_crop.<locals>.cent_crop>, <function imagenet_normalizer>)))

Bases: sparseml.tensorflow_v1.datasets.dataset.Dataset

Implementation for loading an image folder structure into a dataset.

Image folders should be of the form:

root/class_x/xxx.ext

root/class_x/xxy.ext

root/class_x/xxz.ext

root/class_y/123.ext

root/class_y/nsdf3.ext

root/class_y/asd932_.ext

Parameters

• root – the root location for the dataset’s images to load

• train – True to load the training dataset from the root, False for validation

• image_size – the size of the image to reshape to

• pre_resize_transforms – transforms to be applied before resizing the image

• post_resize_transforms – transforms to be applied after resizing the image

creator()

Returns

a created dataset that gives the file_path and label for each image under self.root

format_iterator_batch(iter_batch: Tuple[tensorflow.python.framework.ops.Tensor, …]) → Tuple[Dict[str, tensorflow.python.framework.ops.Tensor], Dict[str, tensorflow.python.framework.ops.Tensor]]

Parameters

iter_batch – the batch ref returned from the iterator

Returns

a tuple of image and label tensors

property image_size

the size of the images to resize to

Type

return

name_scope() → str

Returns

the name scope the dataset should be built under in the graph

property num_classes

the number of classes found for the dataset

Type

return

property num_images

the number of images found for the dataset

Type

return

property post_resize_transforms

transforms to be applied after resizing the image

Type

return

property pre_resize_transforms

transforms to be applied before resizing the image

Type

return

processor(file_path: tensorflow.python.framework.ops.Tensor, label: tensorflow.python.framework.ops.Tensor)

Parameters

• file_path – the path to the file to load an image from

• label – the label for the given image

Returns

a tuple containing the processed image and label

property root

the root location for the dataset’s images to load

Type

return

property train

True to load the training dataset from the root, False for validation

Type

return

class sparseml.tensorflow_v1.datasets.classification.imagefolder.SplitsTransforms(train, val)

Bases: tuple

property train

Alias for field number 0

property val

Alias for field number 1

sparseml.tensorflow_v1.datasets.classification.imagefolder.imagenet_normalizer(img)

Normalize an image using mean and std of the imagenet dataset

Parameters

img – The input image to normalize

Returns

The normalized image

sparseml.tensorflow_v1.datasets.classification.imagenet module

Imagenet dataset implementations for the image classification field in computer vision. More info for the dataset can be found here.

class sparseml.tensorflow_v1.datasets.classification.imagenet.ImageNetDataset(root: str = '~/.cache/nm_datasets/imagenet', train: bool = True, image_size: int = 224)

Bases: sparseml.tensorflow_v1.datasets.classification.imagefolder.ImageFolderDataset

ImageNet dataset implementation

Parameters

• root – the root location for the dataset’s images to load

• train – True to load the training dataset from the root, False for validation

• image_size – the size of the image to reshape to

name_scope() → str

Returns

the name scope the dataset should be built under in the graph

sparseml.tensorflow_v1.datasets.classification.imagenette module

Imagenette and Imagewoof dataset implementations for the image classification field in computer vision. More info for the dataset can be found here.

class sparseml.tensorflow_v1.datasets.classification.imagenette.ImagenetteDataset(root: str = '~/.cache/nm_datasets/imagenette', train: bool = True, dataset_size: sparseml.utils.datasets.imagenette.ImagenetteSize = <ImagenetteSize.s160: 's160'>, image_size: Optional[int] = None, download: bool = True)

Bases: sparseml.tensorflow_v1.datasets.classification.imagefolder.ImageFolderDataset, sparseml.utils.datasets.imagenette.ImagenetteDownloader

Wrapper for the imagenette (10 class) dataset that fastai created. Handles downloading and applying standard transforms.

Parameters

• root – The root folder to find the dataset at, if not found will download here if download=True

• train – True if this is for the training distribution, False for the validation

• dataset_size – The size of the dataset to use and download: See ImagenetteSize for options

• image_size – The image size to output from the dataset

• download – True to download the dataset, False otherwise

name_scope() → str

Returns

the name scope the dataset should be built under in the graph

class sparseml.tensorflow_v1.datasets.classification.imagenette.ImagenetteSize(value)

Bases: enum.Enum

Dataset size for Imagenette / Imagewoof. full does not resize the original dataset at all. s320 resizes the images to 320px. s160 resizes the images to 160px.

full = 'full'

s160 = 's160'

s320 = 's320'

class sparseml.tensorflow_v1.datasets.classification.imagenette.ImagewoofDataset(root: str = '~/.cache/nm_datasets/imagewoof', train: bool = True, dataset_size: sparseml.utils.datasets.imagenette.ImagenetteSize = <ImagenetteSize.s160: 's160'>, image_size: Optional[int] = None, download: bool = True)

Bases: sparseml.tensorflow_v1.datasets.classification.imagefolder.ImageFolderDataset, sparseml.utils.datasets.imagenette.ImagewoofDownloader

Wrapper for the imagewoof (10 class) dataset that fastai created. Handles downloading and applying standard transforms. More info for the dataset can be found here <https://github.com/fastai/imagenette>

Parameters

• root – The root folder to find the dataset at, if not found will download here if download=True

• train – True if this is for the training distribution, False for the validation

• rand_trans – True to apply RandomCrop and RandomHorizontalFlip to the data, False otherwise

• dataset_size – The size of the dataset to use and download: See :py:func ~ImagewoofSize for options

• image_size – The image size to output from the dataset

• download – True to download the dataset, False otherwise

name_scope() → str

Returns

the name scope the dataset should be built under in the graph

Module contents

Datasets related to image classification field in computer vision