Getting started overview

Integrating with Storj involves a sequence of steps to establish connectivity and begin storing data. The core process includes creating a Storj account, generating API credentials, and then using these credentials with an S3-compatible client or one of the provided SDKs to interact with the decentralized network. Storj's S3-compatible Gateway allows developers to use familiar tools and libraries designed for Amazon S3 to interact with Storj's decentralized object storage, reducing the learning curve for those already accustomed to cloud storage APIs.

Before initiating any code-level interaction, access to the Storj Console is required to manage projects, buckets, and API keys. The Console serves as the primary interface for administrative tasks, while programmatically, data operations are conducted via the S3 API or native SDKs.

This guide focuses on the streamlined path to making a first successful request, covering account setup and credential generation, followed by an example using a common S3-compatible client. This approach leverages existing knowledge of S3 operations, simplifying the initial integration for many developers.

Quick-reference setup table

Step What to Do Where
1. Account Creation Sign up for a Storj account. Storj website
2. API Key Generation Create an S3-compatible API key (Access Key, Secret Key) and an Encryption Passphrase. Storj Console (via documentation)
3. Client Setup Configure an S3 client (e.g., MinIO Client, AWS CLI, or an SDK) with your credentials and the Storj S3 Gateway endpoint. Your local development environment
4. First Request Create a bucket and upload an object. Your local development environment

Create an account and get keys

To begin using Storj, account creation is the foundational step. Each account provides access to the Storj Console, where projects are managed, and API credentials are generated. Storj offers a free tier for new users, providing 150 GB of storage and 150 GB of download bandwidth per month, suitable for initial development and testing.

1. Sign up for a Storj account

  1. Navigate to the Storj homepage.
  2. Click the "Sign Up" or "Get Started" button to initiate the registration process.
  3. Provide the required information, typically including email and password.
  4. Verify your email address as prompted to activate your account and gain access to the Storj Console.

2. Generate S3-compatible API credentials

Storj utilizes S3-compatible API keys for programmatic access. These keys come in two parts: an Access Key and a Secret Key. Additionally, an Encryption Passphrase is required for client-side encryption, a core feature of Storj's privacy model.

  1. Log in to the Storj Console.
  2. Navigate to the "Access" section on the left sidebar.
  3. Click "Create S3 API Key".
  4. The Console will generate an Access Key, a Secret Key, and an Encryption Passphrase. It is critical to save these credentials immediately, as the Secret Key and Encryption Passphrase are only displayed once and cannot be retrieved later. Treat these credentials as sensitive and store them securely. Storj documentation provides guidance on understanding access management and key security.

Your first request

After obtaining your API credentials, the next step is to configure an S3-compatible client and make a test request. This example uses the AWS Command Line Interface (CLI), a widely adopted tool for interacting with S3-compatible services. The AWS CLI requires Python, as detailed in the AWS CLI installation guide.

1. Install and configure AWS CLI

If you don't have the AWS CLI installed, follow the official installation instructions for your operating system. For example, using pip:

pip install awscli

Once installed, configure it with your Storj credentials:

aws configure
AWS Access Key ID [None]: YOUR_STORJ_ACCESS_KEY
AWS Secret Access Key [None]: YOUR_STORJ_SECRET_KEY
Default region name [None]: us-east-1  # Storj does not use regions, but S3 clients typically require one.
Default output format [None]: json

Additionally, you must set the AWS_S3_ENDPOINT environment variable to point to the Storj S3 Gateway. The default endpoint for the S3 Gateway is https://gateway.storjshare.io.

export AWS_S3_ENDPOINT=https://gateway.storjshare.io

For persistent configuration, consider adding this export to your shell's profile file (e.g., ~/.bashrc or ~/.zshrc).

2. Set the Encryption Passphrase

The encryption passphrase is set via the STORJ_ENCRYPTION_PASSPHRASE environment variable:

export STORJ_ENCRYPTION_PASSPHRASE=YOUR_STORJ_ENCRYPTION_PASSPHRASE

3. Create a bucket

Buckets are top-level containers for your objects in Storj. Choose a globally unique name.

aws s3api create-bucket --bucket my-first-storj-bucket --endpoint-url https://gateway.storjshare.io

A successful response will look like:

{
    "Location": "/my-first-storj-bucket"
}

4. Upload an object

Create a simple text file, for example, hello.txt, with content "Hello, Storj!". Then upload it to your new bucket.

echo "Hello, Storj!" > hello.txt
aws s3 cp hello.txt s3://my-first-storj-bucket/hello.txt --endpoint-url https://gateway.storjshare.io

The command should complete without output if successful. You can verify the upload in the Storj Console under your project's buckets.

5. Download the object

To confirm the object is stored and retrievable, download it:

aws s3 cp s3://my-first-storj-bucket/hello.txt downloaded_hello.txt --endpoint-url https://gateway.storjshare.io

Check the content of downloaded_hello.txt to ensure it matches the original.

Common next steps

After successfully performing your first object storage operations, consider these common next steps to further integrate Storj into your applications:

  • Explore SDKs: While S3 compatibility is convenient, Storj offers native SDKs for Go, Python, Node.js, Java, Rust, and C#. These SDKs can provide more granular control and potentially optimized performance for specific use cases.
  • Advanced Access Management: Learn about Storj access grants and permissions to manage access to your data securely and efficiently, including creating scoped access keys.
  • Client-Side Encryption: Deepen your understanding of Storj's client-side encryption model. All data uploaded to Storj is client-side encrypted by default, enhancing privacy. The encryption passphrase plays a crucial role in this process.
  • Lifecycle Management: Implement object lifecycle policies for automated deletion or archival of data, optimizing storage costs and compliance. Storj supports S3-compatible lifecycle rules.
  • Integrate with Applications: Begin integrating Storj into your existing applications. Given its S3 compatibility, many applications designed for cloud object storage can be adapted to use Storj by changing the endpoint and credentials.
  • Monitor Usage: Regularly check your usage in the Storj Console to monitor storage consumption and data transfer, especially important for managing costs within and beyond the free tier.
  • Explore Decentralized Application (dApp) Use Cases: For developers building dApps, Storj offers a decentralized backbone for data storage, aligning with principles of Web3.

Troubleshooting the first call

Encountering issues during the initial setup is common. Here are some troubleshooting tips for your first Storj S3-compatible API calls:

  • Incorrect Endpoint URL: Ensure that --endpoint-url https://gateway.storjshare.io is correctly specified in all AWS CLI commands. Forgetting or mistyping this is a frequent cause of connection errors.
  • Missing Environment Variables: Verify that AWS_S3_ENDPOINT and STORJ_ENCRYPTION_PASSPHRASE are set correctly in your shell session. Use echo $AWS_S3_ENDPOINT and echo $STORJ_ENCRYPTION_PASSPHRASE to check their values.
  • Expired or Invalid API Keys: Double-check that the Access Key and Secret Key used in aws configure match the ones generated in the Storj Console. If you suspect an issue, generate a new set of S3 API keys and reconfigure your client. Remember, the Secret Key and Encryption Passphrase are only shown once during creation.
  • Bucket Name Uniqueness: S3-compatible bucket names must be globally unique. If create-bucket fails, try a different, more unique bucket name.
  • Network Connectivity: Ensure your development environment has unrestricted outbound access to gateway.storjshare.io on HTTPS (port 443). Firewall rules or proxy settings can sometimes interfere.
  • AWS CLI Configuration Profile Issues: If you use multiple AWS CLI profiles, ensure you are specifying the correct profile (e.g., --profile storj) or that the default profile is configured for Storj. The AWS CLI documentation provides details on named profiles.
  • Verbose Output: For more detailed error messages when using the AWS CLI, add the --debug flag to your commands. This can provide insights into the underlying HTTP requests and responses.
  • Refer to Storj Documentation: The official Storj documentation provides comprehensive guides and troubleshooting sections for various setup scenarios and client libraries.