sparsify.workers package

Submodules

sparsify.workers.base module

Code related to the base implementations for job workers

class sparsify.workers.base.JobWorker(job_id: str, project_id: str)

Bases: object

The base job worker instance all job workers must extend

Parameters

• job_id – the id of the job the worker is being run for

• project_id – the id of the project the job belongs to

abstract classmethod format_args(**kwargs) → Dict[str, Any]

Format a given args into proper args to be stored for later use in the constructor for the job worker.

Parameters

kwargs – the args to format

Returns

the formatted args to be stored for later use

classmethod get_type() → str

Returns

the type of job worker

property job_id

the id of the job the worker is being run for

Type

return

property project_id

the id of the project the job belongs to

Type

return

abstract run() → Iterator[Dict[str, Any]]

Perform the work for the job. Must be implemented as an iterator that returns a dictionary containing the progress object on each progress step.

Returns

an iterator containing progress update information

class sparsify.workers.base.JobWorkerRegistry(name, bases, attrs)

Bases: type

Registry class for handling and storing BaseJobWorker sub class instances. All subclasses are added to the the REGISTRY property

REGISTRY = {'BaseProfileJobWorker': <class 'sparsify.workers.projects_profiles.BaseProfileJobWorker'>, 'CreateBenchmarkJobWorker': <class 'sparsify.workers.projects_benchmark.CreateBenchmarkJobWorker'>, 'CreateLossProfileJobWorker': <class 'sparsify.workers.projects_profiles.CreateLossProfileJobWorker'>, 'CreatePerfProfileJobWorker': <class 'sparsify.workers.projects_profiles.CreatePerfProfileJobWorker'>, 'DataFromPathJobWorker': <class 'sparsify.workers.projects_data.DataFromPathJobWorker'>, 'DataFromRepoJobWorker': <class 'sparsify.workers.projects_data.DataFromRepoJobWorker'>, 'JobWorker': <class 'sparsify.workers.base.JobWorker'>, 'ModelFromPathJobWorker': <class 'sparsify.workers.projects_model.ModelFromPathJobWorker'>, 'ModelFromRepoJobWorker': <class 'sparsify.workers.projects_model.ModelFromRepoJobWorker'>, '_DataLoaderJobWorker': <class 'sparsify.workers.projects_data._DataLoaderJobWorker'>, '_ModelLoaderJobWorker': <class 'sparsify.workers.projects_model._ModelLoaderJobWorker'>}

static create_worker(job)

sparsify.workers.manager module

Code related to managing jobs in the server

exception sparsify.workers.manager.JobCancelationFailureError(*args: object)

Bases: Exception

Error raised if a job could not be canceled

exception sparsify.workers.manager.JobNotFoundError(*args: object)

Bases: Exception

Error raised if a job is not found in the database

class sparsify.workers.manager.JobWorkerManager(*args, **kwargs)

Bases: object

Manager class for handling running job workers in the background. Only one job worker can run at once. Once one completes, the next oldest one marked as pending in the db is launched.

Parameters

max_workers – The maximum number of workers to allow the ThreadPoolExecutor to work with in parallel

cancel_job(job_id: str)

Cancel a job with the given job_id so it won’t be run.

Parameters

job_id – the job_id to cancel

Raises

• JobNotFoundError – if the job could not be found in the database

• JobCancelationFailureError – if the job could not be canceled

refresh()

Refresh the available jobs and put any pending ones since last refresh onto the ThreadPoolExecutor.

Otherwise will exit out without doing anything and subsequent jobs will be launched after the current one completes.

shutdown()

Shutdown the JobWorkerManager to stop processing any background jobs

start()

Start the JobWorkerManager to begin processing any background jobs present

sparsify.workers.projects_benchmark module

Code related to the benchmark implementations for job workers

class sparsify.workers.projects_benchmark.CreateBenchmarkJobWorker(job_id: str, project_id: str, model_id: str, benchmark_id: str, core_counts: List[int], batch_sizes: List[int], instruction_sets: List[str], inference_models: List[Dict[str, Optional[str]]], warmup_iterations_per_check: int, iterations_per_check: int)

Bases: sparsify.workers.base.JobWorker

A job worker for running and saving a benchmark for a given project and configuration.

Parameters

• job_id – the id of the job this worker is running under

• project_id – the id of the project the worker is running for

• model_id – id of the model to run the loss profile for

• benchmark_id – the benchmark id that should be updated

• core_counts – list of core count to run on for benchmarking. -1 will use the maximum cores available

• batch_sizes – list of batch sizes to use for benchmarking

• instruction_sets – list of instruction sets

• inference_models – list of inference model to use for comparison with fields inference_engine and inference_model_optimization

• warmup_iterations_per_check – the number of warmup iterations to run for before checking performance / timing

• iterations_per_check – the number of iterations to run for each performance check / timing

Returns

the formatted args to be stored for later use

property batch_sizes

list of batch sizes to use for benchmarking

Type

return

property benchmark_id

id of the benchmark

Type

return

property core_counts

list of core count to run on for benchmarking. -1 will use the maximum cores available

Type

return

classmethod format_args(model_id: str, benchmark_id: str, core_counts: List[int], batch_sizes: List[int], instruction_sets: List[str], inference_models: List[Dict[str, Optional[str]]], warmup_iterations_per_check: int, iterations_per_check: int)

Format a given args into proper args to be stored for later use in the constructor for the job worker.

Parameters

• model_id – id of the model to run the loss profile for

• benchmark_id – the benchmark id that should be updated

• core_counts – list of core count to run on for benchmarking. -1 will use the maximum cores available

• batch_sizes – list of batch sizes to use for benchmarking

• instruction_sets – list of instruction sets

• inference_models – list of inference model to use for comparison with fields inference_engine and inference_model_optimization

• warmup_iterations_per_check – the number of warmup iterations to run for before checking performance / timing

• iterations_per_check – the number of iterations to run for each performance check / timing

Returns

the formatted args to be stored for later use

property inference_models

list of inference model to use for comparison with fields inference_engine and inference_model_optimization

Type

return

property instruction_sets

list of instruction sets

Type

return

property iterations_per_check

the number of iterations to run for each performance check / timing

Type

return

property model_id

id of the model to run the loss profile for

Type

return

run() → Iterator[Dict[str, Any]]

Perform the work for the job. Runs and saves the appropriate benchmark based on the configuration

Returns

an iterator containing progress update information

property warmup_iterations_per_check

the number of warmup iterations to run for before checking performance / timing

Type

return

sparsify.workers.projects_data module

Code related to the project_data implementations for job workers

class sparsify.workers.projects_data.DataFromPathJobWorker(job_id: str, project_id: str, data_id: str, uri: str)

Bases: sparsify.workers.projects_data._DataLoaderJobWorker

A job worker for retrieving .npz data files from a given uri. The uri can be either a local file path or a public url.

Parameters

• job_id – the id of the job this worker is running under

• project_id – the id of the project the worker is running for

• data_id – the id of the data the worker is running for

• uri – the uri to retrieve

run() → Iterator[Dict[str, Any]]

Perform the work for the job. Downloads the data files from a public url if the uri is a public url. Copies the data if the uri is accessible through the local file system. If the uri points to tar file, extract and save any additional data objects

Returns

an iterator containing progress update information

class sparsify.workers.projects_data.DataFromRepoJobWorker(job_id: str, project_id: str, data_id: str, uri: str)

Bases: sparsify.workers.projects_data._DataLoaderJobWorker

A job worker for retrieving .npz data files from a given uri. The uri can be either a local file path or a public url.

Parameters

• job_id – the id of the job this worker is running under

• project_id – the id of the project the worker is running for

• data_id – the id of the data the worker is running for

• uri – the uri to retrieve

run() → Iterator[Dict[str, Any]]

Perform the work for the job.

Returns

an iterator containing progress update information

sparsify.workers.projects_model module

Code related to the base implementations for job workers

class sparsify.workers.projects_model.ModelFromPathJobWorker(job_id: str, project_id: str, model_id: str, uri: str)

Bases: sparsify.workers.projects_model._ModelLoaderJobWorker

A job worker for retrieving a model (currently ONNX) from a given uri. The uri can be either a local file path or a public url.

Parameters

• job_id – the id of the job this worker is running under

• project_id – the id of the project the worker is running for

• model_id – the id of the model the worker is running for

• uri – the uri to retrieve

run() → Iterator[Dict[str, Any]]

Perform the work for the job. Downloads the model from a public url if the uri is a public url. Copies the model if the uri is accessible through the local file system.

Returns

an iterator containing progress update information

class sparsify.workers.projects_model.ModelFromRepoJobWorker(job_id: str, project_id: str, model_id: str, uri: str)

Bases: sparsify.workers.projects_model._ModelLoaderJobWorker

A job worker for retrieving a model (currently ONNX) from a given uri from within the Neural Magic model repo.

Parameters

• job_id – the id of the job this worker is running under

• project_id – the id of the project the worker is running for

• model_id – the id of the model the worker is running for

• uri – the uri to retrieve

run() → Iterator[Dict[str, Any]]

Perform the work for the job. Downloads the model from the model repo.

Returns

an iterator containing progress update information

sparsify.workers.projects_profiles module

Job workers for running profiles within a project

class sparsify.workers.projects_profiles.CreateLossProfileJobWorker(job_id: str, project_id: str, model_id: str, profile_id: str, pruning_estimations: bool, pruning_estimation_type: str, pruning_structure: str, quantized_estimations: bool)

Bases: sparsify.workers.projects_profiles.BaseProfileJobWorker

A job worker for running and saving a loss profile for a given project and configuration.

Parameters

• job_id – the id of the job this worker is running under

• project_id – the id of the project the worker is running for

• model_id – id of the model to run the profile for

• profile_id – the profile id of the profile that should be updated

• pruning_estimations – True to include pruning profile information

• pruning_estimation_type – loss analysis type to run, weight_magnitude or one_shot

• pruning_structure – type of pruning to use, (unstructured, block_4…)

• quantized_estimations – True to include quantized information in the profile

classmethod format_args(model_id: str, profile_id: str, pruning_estimations: bool, pruning_estimation_type: str, pruning_structure: str, quantized_estimations: bool, **kwargs) → Union[None, Dict[str, Any]]

Format a given args into proper args to be stored for later use in the constructor for the job worker.

Parameters

• model_id – id of the model to run the loss profile for

• profile_id – the profile id of the loss profile that should be updated

• pruning_estimations – True to include pruning profile information

• pruning_estimation_type – loss analysis type to run, weight_magnitude or one_shot

• pruning_structure – type of pruning to use, (unstructured, block_4…)

• quantized_estimations – True to include quantized information in the profile, False otherwise

Returns

the formatted args to be stored for later use

property model_id

id of the model to run the loss profile for

Type

return

property profile_id

the profile id of the loss profile that should be updated

Type

return

property pruning_estimation_type

loss analysis type to run, weight_magnitude or one_shot

Type

return

property pruning_estimations

True to include pruning profile information

Type

return

property pruning_structure

type of pruning to use, (unstructured, block_4…)

Type

return

property quantized_estimations

True to include quantized information in the profile, False otherwise

Type

return

run() → Iterator[Dict[str, Any]]

Perform the work for the job. Runs and saves the appropriate loss profile based on the configuration

Returns

an iterator containing progress update information

class sparsify.workers.projects_profiles.CreatePerfProfileJobWorker(job_id: str, project_id: str, model_id: str, profile_id: str, batch_size: int, core_count: int, pruning_estimations: bool, quantized_estimations: bool, iterations_per_check: int, warmup_iterations_per_check: int)

Bases: sparsify.workers.projects_profiles.BaseProfileJobWorker

A job worker for running and saving a perf profile for a given project and configuration.

Parameters

• job_id – the id of the job this worker is running under

• project_id – the id of the project the worker is running for

• model_id – id of the model to run the profile for

• profile_id – the profile id of the profile that should be updated

• batch_size – batch size to use for perf analysis

• core_count – number of cores to run on for perf analysis. -1 will use the maximum cores available

• pruning_estimations – True to include pruning measurements

• quantized_estimations – True to include quantization measurements

• iterations_per_check – number of iterations of the batch size to run for each measurement check

• warmup_iterations_per_check – number of warmup iterations of the batch size to run before each measurement check

property batch_size

batch size to use for perf analysis

Type

return

property core_count

number of cores to run on for perf analysis. -1 will use the maximum cores available

Type

return

classmethod format_args(model_id: str, profile_id: str, batch_size: int, core_count: int, pruning_estimations: bool, quantized_estimations: bool, iterations_per_check: int, warmup_iterations_per_check: int, **kwargs) → Union[None, Dict[str, Any]]

Format a given args into proper args to be stored for later use in the constructor for the job worker.

Parameters

• model_id – id of the model to run the loss profile for

• profile_id – the profile id of the loss profile that should be updated

• batch_size – batch size to use for perf analysis

• core_count – number of cores to run on for perf analysis. -1 will use the maximum cores available

• pruning_estimations – True to include pruning measurements

• quantized_estimations – True to include quantization measurements

• iterations_per_check – number of iterations of the batch size to run for each measurement check

• warmup_iterations_per_check – number of warmup iterations of the batch size to run before each measurement check

Returns

the formatted args to be stored for later use

property iterations_per_check

number of iterations of the batch size to run for each measurement check

Type

return

property pruning_estimations

True to include pruning profile information

Type

return

property quantized_estimations

True to include quantized information in the profile, False otherwise

Type

return

run() → Iterator[Dict[str, Any]]

Perform the work for the job. Runs and saves the appropriate perf profile based on the configuration

Returns

an iterator containing progress update information

property warmup_iterations_per_check

number of warmup iterations of the batch size to run before each measurement check

Type

return

Module contents

Code for running background jobs