sparseml.tensorflow_v1.utils package

Submodules

sparseml.tensorflow_v1.utils.exporter module

Export TensorFlow models to the local device

class sparseml.tensorflow_v1.utils.exporter.GraphExporter(output_dir: str)[source]

Bases: object

An exporter for exporting TensorFlow graphs and sessions into ONNX format as well as numpy arrays for the intput and output tensors

Parameters

output_dir – the directory to save the exports to

property checkpoint_path
export_checkpoint(saver: Optional[tensorflow.python.training.saver.Saver] = None, sess: Optional[tensorflow.python.client.session.Session] = None)[source]

Export a checkpoint for the current TensorFlow graph and session.

Parameters

  • saver – the saver instance to use to save the current session, if not supplied will create a new one using TRAINABLE_VARIABLES

  • sess – the current session to export a checkpoint for, if not supplied will use get_default_session()

export_named_samples(inp_dict: Dict[Union[str, tensorflow.python.framework.ops.Tensor], numpy.ndarray], out_dict: Dict[Union[str, tensorflow.python.framework.ops.Tensor], numpy.ndarray])[source]

Export sample inputs and outputs for the model to the local system.

Parameters

  • inp_dict – the inputs to save

  • out_dict – the outputs to save

export_onnx(inputs: List[Union[str, tensorflow.python.framework.ops.Tensor]], outputs: List[Union[str, tensorflow.python.framework.ops.Tensor]], opset: int = 9, custom_op_handlers=None, extra_opset=None, shape_override: Optional[Dict[str, List]] = None)[source]

Export an ONNX format for the graph from the PB format. Should not be called within an active graph or session.

Parameters

  • inputs – the inputs the graph should be created for, can be either a list of names or a list of tensors

  • outputs – the outputs the graph should be created for, can be either a list of names or a list of tensors

  • opset – ONNX opset

  • custom_op_handlers – dictionary of custom op handlers

  • extra_opset – list of extra opset’s

  • shape_override – new shape to override

export_pb(outputs: List[Union[str, tensorflow.python.framework.ops.Tensor]], graph: Optional[tensorflow.python.framework.ops.Graph] = None, sess: Optional[tensorflow.python.client.session.Session] = None)[source]

Export a serialized pb version of the a graph and session.

Parameters

  • outputs – the list of outputs the graph should be created for (used to determine the scope of the graph to export), can be either a list of names or a list of tensors

  • graph – the graph to export to a pb format, if not supplied will use get_default_graph()

  • sess – the session to export to a pb format, if not supplied will use get_default_session()

export_samples(inp_tensors: List[tensorflow.python.framework.ops.Tensor], inp_vals: List[numpy.ndarray], out_tensors: List[tensorflow.python.framework.ops.Tensor], sess: tensorflow.python.client.session.Session) → List[tensorflow.python.framework.ops.Tensor][source]

Export sample tensors for the model to the local system. Executes the inputs through the model using a session to get the outputs.

Parameters

  • inp_tensors – the input tensors to feed through the model

  • inp_vals – the input values to feed through the model and save

  • out_tensors – the output tensors to load values from the model for saving

  • sess – the session to export to a pb format, if not supplied will use get_default_session()

property onnx_path
property pb_path
static pb_to_onnx(inputs: List[Union[str, tensorflow.python.framework.ops.Tensor]], outputs: List[Union[str, tensorflow.python.framework.ops.Tensor]], pb_path: str, onnx_path: str, opset: int = 9, custom_op_handlers=None, extra_opset=None, shape_override: Optional[Dict[str, List]] = None)[source]

Export an ONNX format for the graph from PB format. Should not be called within an active graph or session.

Parameters

  • inputs – the inputs the graph should be created for, can be either a list of names or a list of tensors

  • outputs – the outputs the graph should be created for, can be either a list of names or a list of tensors

  • pb_path – path to the existing PB file

  • onnx_path – path to the output ONNX file

  • opset – ONNX opset

  • custom_op_handlers – dictionary of custom op handlers

  • extra_opset – list of extra opset’s

  • shape_override – new shape to override

property sample_inputs_path
property sample_outputs_path
property tensorflow_path
sparseml.tensorflow_v1.utils.exporter.default_onnx_opset() → int[source]

sparseml.tensorflow_v1.utils.helpers module

sparseml.tensorflow_v1.utils.helpers.tf_compat_div(x, y, name=None)

Computes Python style division of x by y.

For example:

>>> x = tf.constant([16, 12, 11])
>>> y = tf.constant([4, 6, 2])
>>> tf.divide(x,y)
<tf.Tensor: shape=(3,), dtype=float64,
numpy=array([4. , 2. , 5.5])>
Parameters

  • x – A Tensor

  • y – A Tensor

  • name – A name for the operation (optional).

Returns

A Tensor with same shape as input

sparseml.tensorflow_v1.utils.loss module

sparseml.tensorflow_v1.utils.loss.accuracy(logits: tensorflow.python.framework.ops.Tensor, labels: tensorflow.python.framework.ops.Tensor, index: int = 1) → tensorflow.python.framework.ops.Tensor[source]

Standard evaluation for accuracy.

Parameters

  • logits – the logits from the model to use

  • labels – the labels to compare the logits to

  • index – the index in the tensors to compare against

Returns

the accuracy

sparseml.tensorflow_v1.utils.loss.batch_cross_entropy_loss(logits: tensorflow.python.framework.ops.Tensor, labels: tensorflow.python.framework.ops.Tensor) → tensorflow.python.framework.ops.Tensor[source]

Standard cross entropy loss that reduces across the batch dimension.

Parameters

  • logits – the logits from the model to use

  • labels – the labels to compare the logits to

Returns

the cross entropy loss

sparseml.tensorflow_v1.utils.nets_utils module

Utility functions for working with tensorflow_v1 slim’s nets_factory

sparseml.tensorflow_v1.utils.nets_utils.get_gan_network_fn(name: str, is_training: bool = False)[source]

Returns network_fn for a GAN sub-model

Parameters

  • name – The name of the network.

  • is_training – True if the model is being used for training otherwise False

Return network_fn

Function that will run a gan sub-model

Raises

ValueError – If network name is not recognized.

sparseml.tensorflow_v1.utils.nets_utils.get_model_scope(model_name: str, arg_scope_vars: Optional[Dict] = None)[source]
Parameters

  • model_name – name of the model to create an arg scope for

  • arg_scope_vars –

Returns

arg_scope_vars to be passed to the slim arg_scope

sparseml.tensorflow_v1.utils.nets_utils.get_network_fn(name: str, num_classes: int, weight_decay: float = 0.0, is_training: bool = False, arg_scope_vars: Optional[Dict] = None)[source]

Modified from slim/nets/nets_factory Returns a network_fn such as logits, end_points = network_fn(images).

Parameters

  • name – The name of the network.

  • num_classes – The number of classes to use for classification. If 0 or None, the logits layer is omitted and its input features are returned instead.

  • weight_decay – The l2 coefficient for the model weights.

  • is_training – True if the model is being used for training otherwise False

  • arg_scope_vars – arg_scope_vars to be passed to the slim arg_scope

Return network_fn

A function that applies the model to a batch of images. It has the following signature: net, end_points = network_fn(images) The images input is a tensor of shape [batch_size, height, width, 3 or 1] with height = width = network_fn.default_image_size. (The permissibility and treatment of other sizes depends on the network_fn.) The returned end_points are a dictionary of intermediate activations. The returned net is the topmost layer, depending on num_classes: If num_classes was a non-zero integer, net is a logits tensor of shape [batch_size, num_classes]. If num_classes was 0 or None, net is a tensor with the input to the logits layer of shape [batch_size, 1, 1, num_features] or [batch_size, num_features]. Dropout has not been applied to this (even if the network’s original classification does); it remains for the caller to do this or not.

Raises

ValueError – If network name is not recognized.

sparseml.tensorflow_v1.utils.nets_utils.mobilenet_v1_arg_scope(is_training: bool = True, weight_decay: float = 4e-05, stddev: float = 0.09, regularize_depthwise: bool = False, batch_norm_decay: float = 0.9997, batch_norm_epsilon: float = 0.001, batch_norm_updates_collections: tensorflow.python.framework.ops.GraphKeys = 'update_ops', normalizer_fn: Optional[Callable] = None)[source]

Adapted from slim to allow for Xavier initializer Defines the default MobilenetV1 arg scope.

Parameters

  • is_training – Whether or not we’re training the model. If this is set to None, the parameter is not added to the batch_norm arg_scope.

  • weight_decay – The weight decay to use for regularizing the model.

  • stddev – The standard deviation of the trunctated normal weight initializer.

  • regularize_depthwise – Whether or not apply regularization on depthwise.

  • batch_norm_decay – Decay for batch norm moving average.

  • batch_norm_epsilon – Small float added to variance to avoid dividing by zero in batch norm.

  • batch_norm_updates_collections – Collection for the update ops for batch norm.

  • normalizer_fn – Normalization function to apply after convolution.

Returns

An arg_scope to use for the mobilenet v1 model.

sparseml.tensorflow_v1.utils.summary module

Convenience functions for tensorboard and writing summaries to it

sparseml.tensorflow_v1.utils.summary.write_simple_summary(writer: tensorflow.python.summary.writer.writer.FileWriter, tag: str, val: Any, step: int)[source]

Write a simple value summary to a writer

Parameters

  • writer – the writer to write the summary to

  • tag – the tag to write the value under

  • val – the value to write

  • step – the current global step to write the value at

sparseml.tensorflow_v1.utils.variable module

sparseml.tensorflow_v1.utils.variable.any_str_or_regex_matches_tensor_name(tensor_name: str, name_or_regex_patterns: List[str])[source]
Parameters

  • tensor_name – The name of a tensor

  • name_or_regex_patterns – List of full tensor names to match to the input or regex patterns to match with that should be prefixed with ‘re:’

Returns

True if any given str or regex pattern matches the given name

sparseml.tensorflow_v1.utils.variable.clean_tensor_name(var_tens: Union[str, tensorflow.python.framework.ops.Tensor]) → str[source]
Parameters

var_tens – the tensor to get a variable for

Returns

the cleaned version of the name for a variable tensor (removes read and indices at the end)

sparseml.tensorflow_v1.utils.variable.eval_tensor_density(tens: tensorflow.python.framework.ops.Tensor, sess: Optional[tensorflow.python.client.session.Session] = None) → float[source]

Get the density (fraction of non zero values) in a tensor

Parameters

  • tens – the tensor to get the density for

  • sess – the session to use for evaluating the tensor, if not supplied will use the default session

Returns

the density of the tensor

sparseml.tensorflow_v1.utils.variable.eval_tensor_sparsity(tens: tensorflow.python.framework.ops.Tensor, sess: Optional[tensorflow.python.client.session.Session] = None) → float[source]

Get the sparsity (fraction of zero values) in a tensor

Parameters

  • tens – the tensor to get the sparsity for

  • sess – the session to use for evaluating the tensor, if not supplied will use the default session

Returns

the sparsity of the tensor

sparseml.tensorflow_v1.utils.variable.get_op_input_var(operation: tensorflow.python.framework.ops.Operation, var_index: Union[str, int] = 'from_trainable') → tensorflow.python.framework.ops.Tensor[source]

Get the input variable for an operation. Ex: the weight for a conv operator. See @get_op_var_index for proper values for var_index.

Parameters

  • operation – the operation to get the input variable for

  • var_index – the index to guide which input to grab from the operation

Returns

the tensor input that represents the variable input for the operation

sparseml.tensorflow_v1.utils.variable.get_op_var_index(var_index: Union[str, int], op_inputs: None) → int[source]

Get the index of the variable input to an operation. Ex: getting the index of the weight for a convolutional operator.

There are a few different modes that this can work as for var_index value:
- int given, treats this as the desired index of the weight
- string given equal to VAR_INDEX_FROM_TRAINABLE, picks the most likely input
based on finding the first trainable variable that is an input.
Defaults to the last input (all convs have input as last and most matmuls)
- string given, attempts to match the string in any of the inputs name.
Uses the first one found, raises an exception if one couldn’t be found
Parameters

  • var_index – the index to use for figuring out the proper input

  • op_inputs – inputs to the operator from graph editor

Returns

the integer representing the index of the desired variable

sparseml.tensorflow_v1.utils.variable.get_ops_and_inputs_by_name_or_regex(var_names: List[str], graph: Optional[tensorflow.python.framework.ops.Graph] = None) → List[Tuple[tensorflow.python.framework.ops.Operation, tensorflow.python.framework.ops.Tensor]][source]

Get tuples of operations and the inputs for inputs of operations that match a regex pattern in the list params.

Parameters

  • var_names – List of full names or regex patterns to match variable names by.

  • graph – the graph to get the prunable operations from. If not supplied, then will use the default graph

Returns

a list of (operation, parameter) pairs for parameters that match a regex pattern in var_names. If the wildcards ‘.’ or ‘.*’ are provided as regex patterns, then will match on all prunable layers and return variables using get_op_input_var

sparseml.tensorflow_v1.utils.variable.get_prunable_ops(graph: Optional[tensorflow.python.framework.ops.Graph] = None) → List[Tuple[str, tensorflow.python.framework.ops.Operation]][source]

Get the prunable operations from a TensorFlow graph.

Parameters

graph – the graph to get the prunable operations from. If not supplied, then will use the default graph

Returns

a list containing the names and ops of the prunable operations (MatMul, Conv1D, Conv2D, Conv3D)

sparseml.tensorflow_v1.utils.variable.get_tensor_var(tens: tensorflow.python.framework.ops.Tensor) → tensorflow.python.ops.variables.VariableV1[source]

Get the variable associated with a given tensor. Raises a ValueError if not found

Parameters

tens – the tensor to find a variable for

Returns

the found variable matching the given tensor

sparseml.tensorflow_v1.utils.variable.is_prunable_op(op: tensorflow.python.framework.ops.Operation)[source]

Check whether an op is prunable

Parameters

op – the operation to check

Returns

True if the op is prunable; False otherwise

Module contents

Generic code used as utilities and helpers for TensorFlow