SDKs overview

Archive.org offers various methods for developers to programmatically interact with its extensive digital collections, which include the Wayback Machine, digitized books, audio, video, and software archives. While a single, monolithic SDK covering all aspects of Archive.org does not exist, developers can access different functionalities through specific APIs and libraries. These resources facilitate tasks such as querying the Wayback Machine for historical web pages, accessing metadata for archived items, and performing data analysis on public domain content.

The primary programmatic interfaces provided include the Archive.org Developer Hub, which details APIs for various collections. Developers can utilize these APIs directly or through community-contributed libraries that abstract the HTTP requests into language-specific functions. The focus of these tools is to enable researchers, educators, and application developers to integrate Archive.org's vast repository into their projects, supporting digital preservation and access initiatives.

Official SDKs by language

Archive.org's official developer support primarily revolves around its direct APIs and community-driven tools. While a traditional, monolithic SDK is not provided, the organization supports and often contributes to specific libraries that simplify access to its services. The most prominent and actively maintained 'official' or officially supported library is for Python, particularly for interacting with the Wayback Machine and managing items in the archive via its S3-compatible interface.

The following table outlines key official or strongly supported libraries, focusing on their language, package name, installation method, and general maturity level. These tools are designed to streamline common tasks, such as searching for items, retrieving archived data, and uploading content to the Internet Archive.

Language Package/Tool Install Command Maturity
Python internetarchive pip install internetarchive Stable, actively maintained
Python waybackpy pip install waybackpy Stable, actively maintained for Wayback Machine
Generic (API) Archive.org APIs N/A (HTTP requests) Stable, core infrastructure

Installation

Installation for Archive.org-related libraries typically follows standard package management practices for each programming language. For Python, the popular pip installer is used to fetch packages from the Python Package Index (PyPI). Other community-developed libraries may use their respective language's package managers.

Python

To install the internetarchive Python library, which provides a comprehensive interface for interacting with various Archive.org services, open your terminal or command prompt and execute:

pip install internetarchive

For the waybackpy library, specifically designed for the Wayback Machine API, use:

pip install waybackpy

Ensure you have Python and pip installed on your system before attempting these commands. It is often recommended to use a virtual environment to manage dependencies.

Other Languages (Community)

For community libraries in languages like Java, JavaScript, PHP, or Ruby, installation instructions will vary. Developers should consult the specific project's documentation, often found on platforms like GitHub or package repositories, for detailed installation steps. For example, a Node.js library might be installed via npm, and a Java library via Maven or Gradle dependencies.

Quickstart example

This quickstart example demonstrates how to use the internetarchive Python library to search for items on Archive.org and retrieve basic information. This illustrates a common use case for developers looking to integrate Archive.org data into their applications.

Python Quickstart (internetarchive)

First, ensure you have the internetarchive library installed as described in the Installation section.

import internetarchive as ia

# Search for items related to 'open data'
search_results = ia.search_items('subject:"open data"')

print(f"Found {search_results.num_found} items related to 'open data'.")

# Iterate through the first few results and print their titles and identifiers
print("\nFirst 5 results:")
for i, item in enumerate(search_results):
    if i >= 5:
        break
    print(f"  Title: {item.title}")
    print(f"  Identifier: {item.identifier}")
    print(f"  URL: https://archive.org/details/{item.identifier}\n")

# Example: Accessing the Wayback Machine through internetarchive (requires 'waybackpy' for advanced use)
# If you just want to check if a URL exists in the Wayback Machine or retrieve latest snapshot URL:
from waybackpy import WaybackMachineCDX

# Example URL to check
url_to_check = "https://apispine.com"

# Initialize WaybackMachineCDX for a URL
user_agent = "Mozilla/5.0 (Windows NT 5.1; rv:40.0) Gecko/20100101 Firefox/40.0"
cdx_api = WaybackMachineCDX(url_to_check, user_agent)

try:
    # Get the latest archived snapshot
    latest_snapshot = cdx_api.newest()
    print(f"\nLatest Wayback Machine snapshot for {url_to_check}: {latest_snapshot.archive_url}")
    print(f"Timestamp: {latest_snapshot.timestamp}")
except Exception as e:
    print(f"Could not find a Wayback Machine snapshot for {url_to_check}: {e}")

This script first uses internetarchive to search for content by subject and then demonstrates how to use waybackpy to retrieve the latest snapshot of a given URL from the Wayback Machine. These libraries provide methods for more complex interactions, such as downloading files, uploading items, and managing metadata.

Community libraries

Beyond the officially supported Python tools, the developer community has created a variety of libraries in different programming languages to interact with Archive.org's APIs. These libraries often target specific functionalities or provide language-idiomatic interfaces for developers. While not officially maintained by Archive.org, they can be valuable resources for projects in languages other than Python.

Examples of community-contributed libraries and tools include:

  • Java: Libraries that wrap the Wayback Machine CDX (Capture Data Index) API for querying snapshot metadata. Developers can search for these on Maven Central or GitHub by terms like "internet archive java" or "wayback machine java."
  • JavaScript/Node.js: Modules designed for browser-side or server-side JavaScript applications, often focusing on the Wayback Machine API or direct access to item metadata. These are typically available via npm.
  • PHP: Libraries for integrating Archive.org searches or Wayback Machine lookups into PHP-based web applications.
  • Ruby: Gems that provide an interface to various Archive.org APIs, including item search and metadata retrieval.
  • Go: Packages for programmatic access, often featuring concurrency for efficient data retrieval from the Wayback Machine.

When using community libraries, it is important to check the project's active maintenance status, documentation, and community support. Resources like GitHub repositories and Stack Overflow can provide insights into a library's reliability and continued development. Developers can also refer to the HTTP status code definitions to understand the responses from Archive.org's APIs when building custom HTTP clients for direct API interaction.