sparsify.models package

Submodules

sparsify.models.base module

Base DB model classes for the server

class sparsify.models.base. BaseCreatedModifiedModel ( * args , ** kwargs ) [source]

Bases: sparsify.models.base.BaseModel

Base peewee model that includes created and modified timestamp functionality

DoesNotExist

alias of sparsify.models.base.BaseCreatedModifiedModelDoesNotExist

created = <DateTimeField: BaseCreatedModifiedModel.created>
id = <AutoField: BaseCreatedModifiedModel.id>
modified = <DateTimeField: BaseCreatedModifiedModel.modified>
save ( * args , ** kwargs ) [source]
class sparsify.models.base. BaseModel ( * args , ** kwargs ) [source]

Bases: peewee.Model

Base peewee model all DB models must extend from

DoesNotExist

alias of sparsify.models.base.BaseModelDoesNotExist

id = <AutoField: BaseModel.id>
refresh ( ) [source]

Refresh the data for the model instance from the DB

class sparsify.models.base. CSVField ( null = False , index = False , unique = False , column_name = None , default = None , primary_key = False , constraints = None , sequence = None , collation = None , unindexed = False , choices = None , help_text = None , verbose_name = None , index_type = None , db_column = None , _hidden = False ) [source]

Bases: peewee.TextField

CSV field for handling lists of strings in a peewee database

db_value ( value ) [source]
python_value ( value ) [source]
class sparsify.models.base. CSVFloatField ( null = False , index = False , unique = False , column_name = None , default = None , primary_key = False , constraints = None , sequence = None , collation = None , unindexed = False , choices = None , help_text = None , verbose_name = None , index_type = None , db_column = None , _hidden = False ) [source]

Bases: sparsify.models.base.CSVField

CSV field for handling lists of floats in a peewee database

python_value ( value ) [source]
class sparsify.models.base. CSVIntField ( null = False , index = False , unique = False , column_name = None , default = None , primary_key = False , constraints = None , sequence = None , collation = None , unindexed = False , choices = None , help_text = None , verbose_name = None , index_type = None , db_column = None , _hidden = False ) [source]

Bases: sparsify.models.base.CSVField

CSV field for handling lists of integers in a peewee database

python_value ( value ) [source]
class sparsify.models.base. FileStorage [source]

Bases: object

Class for handling local file storage and the path that is located at. Used for storing large files that would not be good in the DB such as model and data files.

init ( root_path : str ) [source]

Initialize the file storage class for a given path

Parameters

root_path – the root path on the local file system for where to store files

property root_path

the root path on the local file system for where to store files

Type

return

class sparsify.models.base. ListObjField ( json_dumps = None , json_loads = None , ** kwargs ) [source]

Bases: playhouse.sqlite_ext.JSONField

Field for handling lists of objects in a peewee database

db_value ( value ) [source]
python_value ( value ) [source]

sparsify.models.jobs module

DB model classes for jobs

class sparsify.models.jobs. Job ( * args , ** kwargs ) [source]

Bases: sparsify.models.base.BaseModel

DB model for a project’s job.

DoesNotExist

alias of sparsify.models.jobs.JobDoesNotExist

baseprojectprofile_set
created = <DateTimeField: Job.created>
error = <TextField: Job.error>
job_id = <CharField: Job.job_id>
modified = <DateTimeField: Job.modified>
progress = <JSONField: Job.progress>
project_id = <CharField: Job.project_id>
projectbenchmark_set
projectdata_set
projectlossprofile_set
projectmodel_set
projectperfprofile_set
save ( * args , ** kwargs ) [source]

Override for peewee save function to update the modified date

status = <JobStatusField: Job.status>
type_ = <CharField: Job.type_>
worker_ack = <BooleanField: Job.worker_ack>
worker_args = <JSONField: Job.worker_args>
class sparsify.models.jobs. JobStatus ( value ) [source]

Bases: enum.Enum

Enumerator class for tracking the status of jobs

canceled = 'canceled'
canceling = 'canceling'
completed = 'completed'
error = 'error'
pending = 'pending'
started = 'started'
class sparsify.models.jobs. JobStatusField ( null = False , index = False , unique = False , column_name = None , default = None , primary_key = False , constraints = None , sequence = None , collation = None , unindexed = False , choices = None , help_text = None , verbose_name = None , index_type = None , db_column = None , _hidden = False ) [source]

Bases: peewee.Field

peewee DB field for saving and loading JobStatus from the database

db_value ( value : sparsify.models.jobs.JobStatus ) [source]
field_type = 'VARCHAR'
python_value ( value : str ) [source]

sparsify.models.projects module

DB model classes for a project and it’s nested files

class sparsify.models.projects. BaseProjectModel ( * args , ** kwargs ) [source]

Bases: sparsify.models.base.BaseModel

DoesNotExist

alias of sparsify.models.projects.BaseProjectModelDoesNotExist

abstract delete_filesystem ( ) [source]

Delete the state from the local file system

id = <AutoField: BaseProjectModel.id>
abstract setup_filesystem ( ) [source]

Setup the local file system so that it can be used with the data

abstract validate_filesystem ( ) [source]

Validate that the local file system and expected files are correct and exist

class sparsify.models.projects. Project ( * args , ** kwargs ) [source]

Bases: sparsify.models.projects.BaseProjectModel

DB model for a project’s data file. A project may have multiple data files stored in the DB.

DoesNotExist

alias of sparsify.models.projects.ProjectDoesNotExist

benchmarks
created = <DateTimeField: Project.created>
data
delete_filesystem ( ) [source]

Delete the folder from the local file system containing all of the files for the project

description = <TextField: Project.description>
property dir_path

the local directory path for where project’s files are stored

Type

return

property dir_size

Size of the folder on the local file system containing all of the files for the project

models
modified = <DateTimeField: Project.modified>
name = <TextField: Project.name>
optims
profiles_loss
profiles_perf
project_id = <CharField: Project.project_id>
save ( * args , ** kwargs ) [source]

Override for peewee save function to update the modified date

setup_filesystem ( ) [source]

Setup the local file system so that it can be used with the data

training_epochs = <IntegerField: Project.training_epochs>
training_lr_final = <FloatField: Project.training_lr_final>
training_lr_init = <FloatField: Project.training_lr_init>
training_optimizer = <TextField: Project.training_optimizer>
validate_filesystem ( ) [source]

Validate that the local file system and expected files are correct and exist

sparsify.models.projects_benchmark module

DB model classes for project’s benchmark

class sparsify.models.projects_benchmark. ProjectBenchmark ( * args , ** kwargs ) [source]

Bases: sparsify.models.base.BaseCreatedModifiedModel

DB model for a project’s benchmark

DoesNotExist

alias of sparsify.models.projects_benchmark.ProjectBenchmarkDoesNotExist

batch_sizes = <JSONField: ProjectBenchmark.batch_sizes>
benchmark_id = <CharField: ProjectBenchmark.benchmark_id>
core_counts = <JSONField: ProjectBenchmark.core_counts>
created = <DateTimeField: ProjectBenchmark.created>
inference_models = <ListObjField: ProjectBenchmark.inference_models>
instruction_sets = <JSONField: ProjectBenchmark.instruction_sets>
iterations_per_check = <IntegerField: ProjectBenchmark.iterations_per_check>
job = <ForeignKeyField: ProjectBenchmark.job>
job_id = <ForeignKeyField: ProjectBenchmark.job>
modified = <DateTimeField: ProjectBenchmark.modified>
name = <TextField: ProjectBenchmark.name>
project = <ForeignKeyField: ProjectBenchmark.project>
project_id = <ForeignKeyField: ProjectBenchmark.project>
result = <JSONField: ProjectBenchmark.result>
source = <TextField: ProjectBenchmark.source>
warmup_iterations_per_check = <IntegerField: ProjectBenchmark.warmup_iterations_per_check>

sparsify.models.projects_data module

DB model classes for a project’s sample data files

class sparsify.models.projects_data. ProjectData ( * args , ** kwargs ) [source]

Bases: sparsify.models.projects.BaseProjectModel

DB model for a project’s data file. A project may have multiple data files stored in the DB.

DoesNotExist

alias of sparsify.models.projects_data.ProjectDataDoesNotExist

created = <DateTimeField: ProjectData.created>
data_id = <CharField: ProjectData.data_id>
delete_filesystem ( ) [source]

Delete the data file from the local file system

property dir_path

the local directory path for where the data is stored

Type

return

file = <TextField: ProjectData.file>
property file_path

the local file path to the data file

Type

return

job = <ForeignKeyField: ProjectData.job>
job_id = <ForeignKeyField: ProjectData.job>
project = <ForeignKeyField: ProjectData.project>
project_id = <ForeignKeyField: ProjectData.project>
setup_filesystem ( ) [source]

Setup the local file system so that it can be used with the data

source = <TextField: ProjectData.source>
validate_filesystem ( ) [source]

Validate that the local file system and expected files are correct and exist

sparsify.models.projects_model module

DB model classes for a project’s model file

class sparsify.models.projects_model. ProjectModel ( * args , ** kwargs ) [source]

Bases: sparsify.models.projects.BaseProjectModel

DB model for a project’s model file. A project must have only one model file stored in the DB.

DoesNotExist

alias of sparsify.models.projects_model.ProjectModelDoesNotExist

analysis = <JSONField: ProjectModel.analysis>
created = <DateTimeField: ProjectModel.created>
delete_filesystem ( ) [source]

Delete the model file from the local file system

property dir_path

the local directory path for where the model file is stored

Type

return

file = <TextField: ProjectModel.file>
property file_path

the local file path to the data file

Type

return

job = <ForeignKeyField: ProjectModel.job>
job_id = <ForeignKeyField: ProjectModel.job>
model_id = <CharField: ProjectModel.model_id>
project = <ForeignKeyField: ProjectModel.project>
project_id = <ForeignKeyField: ProjectModel.project>
setup_filesystem ( ) [source]

Setup the local file system so that it can be used with the data

source = <TextField: ProjectModel.source>
validate_filesystem ( ) [source]

Validate that the local file system and expected files are correct and exist

sparsify.models.projects_optimizations module

DB model classes for project’s optimizations and modifiers

class sparsify.models.projects_optimizations. ProjectOptimization ( * args , ** kwargs ) [source]

Bases: sparsify.models.base.BaseCreatedModifiedModel

DB model for a project’s optimization (stores modifier settings). A project may have multiple optimizations stored in the DB.

DoesNotExist

alias of sparsify.models.projects_optimizations.ProjectOptimizationDoesNotExist

created = <DateTimeField: ProjectOptimization.created>
end_epoch = <FloatField: ProjectOptimization.end_epoch>
lr_schedule_modifiers
modified = <DateTimeField: ProjectOptimization.modified>
name = <TextField: ProjectOptimization.name>
notes = <TextField: ProjectOptimization.notes>
optim_id = <CharField: ProjectOptimization.optim_id>
profile_loss = <ForeignKeyField: ProjectOptimization.profile_loss>
profile_loss_id = <ForeignKeyField: ProjectOptimization.profile_loss>
profile_perf = <ForeignKeyField: ProjectOptimization.profile_perf>
profile_perf_id = <ForeignKeyField: ProjectOptimization.profile_perf>
project = <ForeignKeyField: ProjectOptimization.project>
project_id = <ForeignKeyField: ProjectOptimization.project>
pruning_modifiers
quantization_modifiers
start_epoch = <FloatField: ProjectOptimization.start_epoch>
trainable_modifiers
class sparsify.models.projects_optimizations. ProjectOptimizationModifierLRSchedule ( * args , ** kwargs ) [source]

Bases: sparsify.models.base.BaseCreatedModifiedModel

DB model for a project’s learning rate schedule modifier.

DoesNotExist

alias of sparsify.models.projects_optimizations.ProjectOptimizationModifierLRScheduleDoesNotExist

created = <DateTimeField: ProjectOptimizationModifierLRSchedule.created>
end_epoch = <FloatField: ProjectOptimizationModifierLRSchedule.end_epoch>
final_lr = <FloatField: ProjectOptimizationModifierLRSchedule.final_lr>
init_lr = <FloatField: ProjectOptimizationModifierLRSchedule.init_lr>
lr_mods = <ListObjField: ProjectOptimizationModifierLRSchedule.lr_mods>
modified = <DateTimeField: ProjectOptimizationModifierLRSchedule.modified>
modifier_id = <CharField: ProjectOptimizationModifierLRSchedule.modifier_id>
optim = <ForeignKeyField: ProjectOptimizationModifierLRSchedule.optim>
optim_id = <ForeignKeyField: ProjectOptimizationModifierLRSchedule.optim>
start_epoch = <FloatField: ProjectOptimizationModifierLRSchedule.start_epoch>
class sparsify.models.projects_optimizations. ProjectOptimizationModifierPruning ( * args , ** kwargs ) [source]

Bases: sparsify.models.base.BaseCreatedModifiedModel

DB model for a project’s optimization pruning modifier.

DoesNotExist

alias of sparsify.models.projects_optimizations.ProjectOptimizationModifierPruningDoesNotExist

balance_perf_loss = <FloatField: ProjectOptimizationModifierPruning.balance_perf_loss>
compression = <FloatField: ProjectOptimizationModifierPruning.compression>
created = <DateTimeField: ProjectOptimizationModifierPruning.created>
end_epoch = <FloatField: ProjectOptimizationModifierPruning.end_epoch>
est_loss_sensitivity = <FloatField: ProjectOptimizationModifierPruning.est_loss_sensitivity>
est_perf_sensitivity = <FloatField: ProjectOptimizationModifierPruning.est_perf_sensitivity>
est_recovery = <FloatField: ProjectOptimizationModifierPruning.est_recovery>
est_time = <FloatField: ProjectOptimizationModifierPruning.est_time>
est_time_baseline = <FloatField: ProjectOptimizationModifierPruning.est_time_baseline>
est_time_gain = <FloatField: ProjectOptimizationModifierPruning.est_time_gain>
filter_min_perf_gain = <FloatField: ProjectOptimizationModifierPruning.filter_min_perf_gain>
filter_min_recovery = <FloatField: ProjectOptimizationModifierPruning.filter_min_recovery>
filter_min_sparsity = <FloatField: ProjectOptimizationModifierPruning.filter_min_sparsity>
flops = <FloatField: ProjectOptimizationModifierPruning.flops>
flops_baseline = <FloatField: ProjectOptimizationModifierPruning.flops_baseline>
flops_gain = <FloatField: ProjectOptimizationModifierPruning.flops_gain>
mask_type = <TextField: ProjectOptimizationModifierPruning.mask_type>
modified = <DateTimeField: ProjectOptimizationModifierPruning.modified>
modifier_id = <CharField: ProjectOptimizationModifierPruning.modifier_id>
nodes = <ListObjField: ProjectOptimizationModifierPruning.nodes>
optim = <ForeignKeyField: ProjectOptimizationModifierPruning.optim>
optim_id = <ForeignKeyField: ProjectOptimizationModifierPruning.optim>
params = <FloatField: ProjectOptimizationModifierPruning.params>
params_baseline = <FloatField: ProjectOptimizationModifierPruning.params_baseline>
sparsity = <FloatField: ProjectOptimizationModifierPruning.sparsity>
start_epoch = <FloatField: ProjectOptimizationModifierPruning.start_epoch>
update_frequency = <FloatField: ProjectOptimizationModifierPruning.update_frequency>
class sparsify.models.projects_optimizations. ProjectOptimizationModifierQuantization ( * args , ** kwargs ) [source]

Bases: sparsify.models.base.BaseCreatedModifiedModel

DB model for a project’s optimization quantization modifier.

DoesNotExist

alias of sparsify.models.projects_optimizations.ProjectOptimizationModifierQuantizationDoesNotExist

balance_perf_loss = <FloatField: ProjectOptimizationModifierQuantization.balance_perf_loss>
compression = <FloatField: ProjectOptimizationModifierQuantization.compression>
created = <DateTimeField: ProjectOptimizationModifierQuantization.created>
end_epoch = <FloatField: ProjectOptimizationModifierQuantization.end_epoch>
est_loss_sensitivity = <FloatField: ProjectOptimizationModifierQuantization.est_loss_sensitivity>
est_perf_sensitivity = <FloatField: ProjectOptimizationModifierQuantization.est_perf_sensitivity>
est_recovery = <FloatField: ProjectOptimizationModifierQuantization.est_recovery>
est_time = <FloatField: ProjectOptimizationModifierQuantization.est_time>
est_time_baseline = <FloatField: ProjectOptimizationModifierQuantization.est_time_baseline>
est_time_gain = <FloatField: ProjectOptimizationModifierQuantization.est_time_gain>
filter_min_perf_gain = <FloatField: ProjectOptimizationModifierQuantization.filter_min_perf_gain>
filter_min_recovery = <FloatField: ProjectOptimizationModifierQuantization.filter_min_recovery>
flops = <FloatField: ProjectOptimizationModifierQuantization.flops>
flops_baseline = <FloatField: ProjectOptimizationModifierQuantization.flops_baseline>
flops_gain = <FloatField: ProjectOptimizationModifierQuantization.flops_gain>
level = <TextField: ProjectOptimizationModifierQuantization.level>
modified = <DateTimeField: ProjectOptimizationModifierQuantization.modified>
modifier_id = <CharField: ProjectOptimizationModifierQuantization.modifier_id>
nodes = <ListObjField: ProjectOptimizationModifierQuantization.nodes>
optim = <ForeignKeyField: ProjectOptimizationModifierQuantization.optim>
optim_id = <ForeignKeyField: ProjectOptimizationModifierQuantization.optim>
params = <FloatField: ProjectOptimizationModifierQuantization.params>
params_baseline = <FloatField: ProjectOptimizationModifierQuantization.params_baseline>
start_epoch = <FloatField: ProjectOptimizationModifierQuantization.start_epoch>
class sparsify.models.projects_optimizations. ProjectOptimizationModifierTrainable ( * args , ** kwargs ) [source]

Bases: sparsify.models.base.BaseCreatedModifiedModel

DB model for a project’s optimization trainable modifier.

DoesNotExist

alias of sparsify.models.projects_optimizations.ProjectOptimizationModifierTrainableDoesNotExist

created = <DateTimeField: ProjectOptimizationModifierTrainable.created>
end_epoch = <FloatField: ProjectOptimizationModifierTrainable.end_epoch>
modified = <DateTimeField: ProjectOptimizationModifierTrainable.modified>
modifier_id = <CharField: ProjectOptimizationModifierTrainable.modifier_id>
nodes = <ListObjField: ProjectOptimizationModifierTrainable.nodes>
optim = <ForeignKeyField: ProjectOptimizationModifierTrainable.optim>
optim_id = <ForeignKeyField: ProjectOptimizationModifierTrainable.optim>
start_epoch = <FloatField: ProjectOptimizationModifierTrainable.start_epoch>

sparsify.models.projects_profiles module

DB model classes for project’s profiles such as performance and loss

class sparsify.models.projects_profiles. BaseProjectProfile ( * args , ** kwargs ) [source]

Bases: sparsify.models.base.BaseModel

Base DB model for project’s profiles such as loss and perf

DoesNotExist

alias of sparsify.models.projects_profiles.BaseProjectProfileDoesNotExist

analysis = <JSONField: BaseProjectProfile.analysis>
created = <DateTimeField: BaseProjectProfile.created>
job = <ForeignKeyField: BaseProjectProfile.job>
job_id = <ForeignKeyField: BaseProjectProfile.job>
name = <TextField: BaseProjectProfile.name>
profile_id = <CharField: BaseProjectProfile.profile_id>
source = <TextField: BaseProjectProfile.source>
class sparsify.models.projects_profiles. ProjectLossProfile ( * args , ** kwargs ) [source]

Bases: sparsify.models.projects_profiles.BaseProjectProfile

DB model for a project’s loss profile. A project may have multiple loss profiles stored in the DB.

DoesNotExist

alias of sparsify.models.projects_profiles.ProjectLossProfileDoesNotExist

analysis = <JSONField: ProjectLossProfile.analysis>
created = <DateTimeField: ProjectLossProfile.created>
job = <ForeignKeyField: ProjectLossProfile.job>
job_id = <ForeignKeyField: ProjectLossProfile.job>
name = <TextField: ProjectLossProfile.name>
profile_id = <CharField: ProjectLossProfile.profile_id>
project = <ForeignKeyField: ProjectLossProfile.project>
project_id = <ForeignKeyField: ProjectLossProfile.project>
projectoptimization_set
pruning_estimation_type = <TextField: ProjectLossProfile.pruning_estimation_type>
pruning_estimations = <BooleanField: ProjectLossProfile.pruning_estimations>
pruning_structure = <TextField: ProjectLossProfile.pruning_structure>
quantized_estimation_type = <TextField: ProjectLossProfile.quantized_estimation_type>
quantized_estimations = <BooleanField: ProjectLossProfile.quantized_estimations>
source = <TextField: ProjectLossProfile.source>
class sparsify.models.projects_profiles. ProjectPerfProfile ( * args , ** kwargs ) [source]

Bases: sparsify.models.projects_profiles.BaseProjectProfile

DB model for a project’s performance profile. A project may have multiple perf profiles stored in the DB

DoesNotExist

alias of sparsify.models.projects_profiles.ProjectPerfProfileDoesNotExist

analysis = <JSONField: ProjectPerfProfile.analysis>
batch_size = <IntegerField: ProjectPerfProfile.batch_size>
core_count = <IntegerField: ProjectPerfProfile.core_count>
created = <DateTimeField: ProjectPerfProfile.created>
instruction_sets = <CSVField: ProjectPerfProfile.instruction_sets>
iterations_per_check = <IntegerField: ProjectPerfProfile.iterations_per_check>
job = <ForeignKeyField: ProjectPerfProfile.job>
job_id = <ForeignKeyField: ProjectPerfProfile.job>
name = <TextField: ProjectPerfProfile.name>
profile_id = <CharField: ProjectPerfProfile.profile_id>
project = <ForeignKeyField: ProjectPerfProfile.project>
project_id = <ForeignKeyField: ProjectPerfProfile.project>
projectoptimization_set
pruning_estimations = <BooleanField: ProjectPerfProfile.pruning_estimations>
quantized_estimations = <BooleanField: ProjectPerfProfile.quantized_estimations>
source = <TextField: ProjectPerfProfile.source>
warmup_iterations_per_check = <IntegerField: ProjectPerfProfile.warmup_iterations_per_check>

sparsify.models.utils module

sparsify.models.utils. database_setup ( working_dir : str , app : Optional [ flask.app.Flask ] = None ) [source]

Module contents

DB models and objects for working with sparsify server