SDKs overview

Hugging Face provides Software Development Kits (SDKs) and libraries designed to facilitate interaction with its ecosystem, including the Hugging Face Hub, pre-trained models, and associated machine learning tools. These SDKs enable developers to integrate machine learning capabilities into applications, perform inference, fine-tune models, and manage datasets programmatically. The core offerings are centered around the transformers library for Python and transformers.js for JavaScript, which abstract away the complexities of working with various deep learning models and frameworks.

The SDKs are developed to support common machine learning workflows, from natural language processing (NLP) to computer vision and audio tasks. They offer consistent APIs for accessing models hosted on the Hugging Face Hub, which serves as a central repository for open-source machine learning models, datasets, and demo applications called Spaces. Developers can use these libraries to load models, preprocess data, execute inference, and save custom models back to the Hub Hugging Face Hub SDK documentation.

The design philosophy emphasizes ease of use and interoperability with popular deep learning frameworks such as PyTorch, TensorFlow, and JAX Hugging Face Transformers library overview. This compatibility allows developers to leverage their existing knowledge and toolchains while integrating Hugging Face's extensive collection of models. Beyond the primary libraries, the Hugging Face ecosystem also includes specialized libraries like diffusers for diffusion models and peft for parameter-efficient fine-tuning Diffusers library documentation.

Official SDKs by language

Hugging Face maintains official SDKs and libraries primarily for Python and JavaScript, catering to different development environments and application types. These libraries are actively developed and supported by Hugging Face.

Language Package Name Install Command Primary Use Cases Maturity
Python transformers pip install transformers NLP, Computer Vision, Audio tasks; model loading, inference, training, fine-tuning Stable, widely adopted
Python datasets pip install datasets Loading, processing, and sharing datasets for ML models Stable
Python accelerate pip install accelerate Simplifying distributed training across multiple GPUs/TPUs Stable
Python diffusers pip install diffusers Working with diffusion models for image/audio generation Stable
Python peft pip install peft Parameter-Efficient Fine-tuning (LoRA, P-tuning, etc.) Stable
JavaScript @xenova/transformers (transformers.js) npm install @xenova/transformers In-browser/Node.js inference for various ML tasks Stable

Installation

Installation of Hugging Face SDKs typically involves standard package managers for Python and JavaScript environments. Specific installation steps may vary based on the desired deep learning framework (e.g., PyTorch, TensorFlow, JAX) or specific hardware requirements (e.g., GPU support).

Python

To install the core transformers library with CPU support, use pip:

pip install transformers

For specific deep learning framework support, install the necessary dependencies. For example, to use PyTorch:

pip install transformers[torch]

Or for TensorFlow:

pip install transformers[tf]

To install with all optional dependencies for all frameworks:

pip install transformers[all]

The datasets library can be installed via pip:

pip install datasets

Similarly, accelerate and diffusers:

pip install accelerate
pip install diffusers

More detailed installation instructions, including GPU setup and development versions, are available in the Hugging Face Transformers installation guide.

JavaScript

The transformers.js library is installed via npm:

npm install @xenova/transformers

This package supports both Node.js environments and direct use in web browsers transformers.js official documentation.

Quickstart example

This quickstart demonstrates using the Python transformers library to perform sentiment analysis with a pre-trained model. This example loads a pipeline, which abstracts away tokenization, model inference, and post-processing into a single function Hugging Face Transformers quick tour.

Python (Sentiment Analysis)

from transformers import pipeline

# Load the sentiment analysis pipeline
classifier = pipeline('sentiment-analysis')

# Perform inference on a sample text
result = classifier('I love using Hugging Face models!')

# Print the result
print(result)
# Expected output: [{'label': 'POSITIVE', 'score': 0.9998701810836792}]

result_negative = classifier('I hate this movie, it was terrible.')
print(result_negative)
# Expected output: [{'label': 'NEGATIVE', 'score': 0.9997424483299255}]

JavaScript (Text Classification in Node.js)

This example shows how to perform text classification using transformers.js in a Node.js environment.

import { pipeline } from '@xenova/transformers';

async function runSentimentAnalysis() {
  // Allocate a pipeline for sentiment-analysis
  const classifier = await pipeline('sentiment-analysis');

  // Perform inference
  const result = await classifier('We are very happy to show you the Transformers library.');

  // Print the result
  console.log(result);
  // Expected output: [{ label: 'POSITIVE', score: 0.9998179495334625 }]

  const resultNegative = await classifier('This is a terrible experience.');
  console.log(resultNegative);
  // Expected output: [{ label: 'NEGATIVE', score: 0.9996660351753235 }]
}

runSentimentAnalysis();

For browser-based usage, the transformers.js library can be loaded via a CDN or bundled with a web application, allowing on-device inference without server-side calls. This approach leverages WebAssembly and Web Workers for efficient execution Mozilla WebAssembly documentation.

Community libraries

Beyond the official offerings, the Hugging Face ecosystem is supported by a range of community-contributed libraries and tools. These often extend the core functionalities or provide integrations with other platforms and frameworks. While not officially maintained by Hugging Face, many community projects are widely used and contribute significantly to the platform's versatility.

  • optimum: An extension of transformers that provides performance optimizations for training and inference with various hardware accelerators (e.g., ONNX Runtime, OpenVINO, Habana Gaudi). It helps deploy models more efficiently across different target hardware Hugging Face Optimum documentation.
  • evaluate: A library for easily comparing and evaluating machine learning models and datasets. It provides access to a wide range of metrics and evaluation methodologies for various tasks Hugging Face Evaluate library.
  • Gradio: While not exclusively a Hugging Face library, Gradio is commonly used within the Hugging Face ecosystem to build interactive web demos for machine learning models, often hosted on Hugging Face Spaces. It allows developers to create shareable UIs directly from Python code Gradio official documentation.
  • Streamlit: Another popular framework for creating data apps and interactive ML demos. Similar to Gradio, Streamlit applications can be deployed on Hugging Face Spaces to showcase models Streamlit documentation.
  • LangChain / LlamaIndex integrations: Libraries like LangChain and LlamaIndex, which focus on building applications with large language models, often include connectors and integrations for Hugging Face models and the Hugging Face Hub, enabling access to a broad range of open-source LLMs within their frameworks.
  • Awesome Hugging Face: A community-curated list of resources, projects, and integrations related to Hugging Face, including various third-party tools and extensions. This list often highlights emerging community efforts and niche applications.

These community-driven efforts complement the official SDKs by addressing specific use cases, integrating with broader ML ecosystems, and fostering innovation around the Hugging Face platform. Developers are encouraged to explore these resources for specialized needs or to contribute their own tools to the community.