Skip to main content
Version: 1.7.0

Getting Started With Neural Magic

Ready to optimize and deploy deep learning models with greater efficiency? This guide provides the essential steps to get up and running with Neural Magic's powerful software suite. Specifically, you'll learn how to install our key products, deploy an LLM, and create your sparsified LLMs.


Ensure a seamless start by installing Neural Magic's core components and configuring your environment for model optimization and deployment. We'll guide you through managing dependencies and tailoring the setup for your specific use case.

๐Ÿ“„๏ธ Installation

Step-by-step guides for installing NeuralMagic Products such as DeepSparse, SparseML, and SparseZoo.

Deploy a Modelโ€‹

Discover how to deploy pre-sparsified LLMs for accelerated text generation and reduced resource demands. Unlock cost-effective AI applications through the power of CPU-optimized LLM deployment.

๐Ÿ“„๏ธ Deploying LLMs

Deploy large language models (LLMs) for text generation using Neural Magic's DeepSparse. This doc includes code examples, performance benchmarking, and server setup.

Optimize a Modelโ€‹

Learn how to apply state-of-the-art sparsification to significantly reduce the footprint and increase the inference speed of your LLMs quickly utilizing post-training techniques.

๐Ÿ“„๏ธ Optimizing LLMs

Optimize large language models (LLMs) for efficient inference using one-shot pruning and quantization. Learn how to improve model performance and reduce costs without sacrificing accuracy.

Fine-Tune a Sparse Modelโ€‹

Refine your sparsified LLM through fine-tuning to improve its performance and accuracy for your specific use case.

๐Ÿ“„๏ธ Sparse Fine-Tuning With LLMs

Improve the performance of your large language models (LLMs) through fine-tuning with Neural Magic's SparseML. Optimize LLMs for specific tasks while maintaining accuracy.

Transfer a Sparse Modelโ€‹

Transfer a pre-sparsified, foundational LLM to your use case without heavy retraining or model optimization tuning.

๐Ÿ“„๏ธ Sparse Transferring LLMs

Adapt large language models (LLMs) to new domains and tasks using sparse transfer learning with Neural Magic's SparseML. Maintain accuracy while optimizing for efficiency.

๐Ÿ”— Let's Get Started! Choose a task above to begin your Neural Magic journey.

๐Ÿ“š Need Help? Join our Slack community for support.

๐ŸŒŸ Shape the Future: Contribute to our GitHub GitHub repositories.