Weaviate Authentication: Methods, Credentials, and Security

Authentication overview

Weaviate provides mechanisms to authenticate requests, ensuring that only authorized entities can perform operations on your vector database instance. This is critical for data security and operational integrity, especially when deploying Weaviate in production environments or integrating it with other systems. Authentication methods apply to both Weaviate Cloud instances and self-hosted deployments, though the setup specifics may vary.

The core principle involves verifying the identity of the client making a request before granting access to Weaviate's data and APIs. This typically requires including specific credentials, such as API keys or tokens, in the request headers. Proper authentication helps prevent unauthorized data access, modification, or deletion, and contributes to adherence with compliance standards like SOC 2 Type II and GDPR.

Weaviate's client libraries for Python, TypeScript, Go, Java, Ruby, Rust, and C# are designed to simplify the process of including authentication credentials in your application code. This allows developers to integrate secure access without needing to manage the low-level HTTP header manipulation directly.

Supported authentication methods

Weaviate supports several authentication methods to accommodate different deployment scenarios and security requirements. The choice of method often depends on whether you are using Weaviate Cloud or a self-hosted instance, and the broader identity management infrastructure in place.

API Key Authentication

API key authentication is a common method for securing access to Weaviate. It involves issuing a unique key that must be included in every request to the Weaviate instance. This key acts as a secret token, allowing the Weaviate server to verify the request's origin. API keys are suitable for application-to-application communication where a direct user login is not required.

• How it works: The client sends an API key, typically in an Authorization header (e.g., Authorization: Bearer YOUR_API_KEY) or a custom header (e.g., X-WCS-Api-Key for Weaviate Cloud). Weaviate validates this key against its configured secrets.
• Use cases: Backend services, single-page applications, serverless functions, and local development environments.

OpenID Connect (OIDC) Authentication

OpenID Connect (OIDC) is an identity layer built on top of the OAuth 2.0 framework. It allows clients to verify the identity of the end-user based on the authentication performed by an authorization server, as well as to obtain basic profile information about the end-user in an interoperable and REST-like manner. Weaviate can be configured to integrate with an OIDC provider, delegating user authentication to an external service.

• How it works: Weaviate acts as a Relying Party and integrates with an external OIDC Provider (e.g., Auth0, Okta, Google Identity Platform). Users authenticate with the OIDC Provider, which then issues an ID Token. This token is presented to Weaviate for authorization.
• Use cases: Enterprise environments with existing identity providers, applications requiring user-level authentication and authorization, and scenarios needing centralized identity management.

The following table summarizes Weaviate's authentication methods:

Getting your credentials

Obtaining the necessary authentication credentials depends on your Weaviate deployment model:

Weaviate Cloud

For Weaviate Cloud, API keys are the primary method for authenticating programmatic access. These keys are managed through the Weaviate Cloud Console:

1. Create a Weaviate Cluster: If you haven't already, create a new cluster in the Weaviate Cloud Console.
2. Generate API Key: Navigate to your cluster settings. Weaviate Cloud automatically provisions an API key for your cluster. This key, often labeled as X-WCS-Api-Key, is your primary authentication credential for accessing the cloud instance. You'll typically find it listed under API keys or security settings for your specific cluster.
3. Admin API Key (Weaviate API Key): In addition to the WCS API key for cloud access, you may also need a Weaviate API key for administrative operations or specific data plane interactions, depending on your setup. This is often referred to simply as the Weaviate API Key in the console.
4. OpenID Connect: Weaviate Cloud environments can be configured to use OIDC through enterprise-tier agreements. Details for setting this up are typically provided during the onboarding process for such accounts, involving configuration within the Weaviate Console and your chosen OIDC provider.

Self-Hosted Weaviate

For self-hosted Weaviate instances, you configure authentication directly within your Weaviate server's environment. This offers greater flexibility in choosing and managing your authentication mechanisms:

1. API Key Configuration: To enable API key authentication for a self-hosted instance, you typically set environment variables or configuration files on the server where Weaviate is running. For example, using Docker Compose, you might define AUTHENTICATION_APIKEY_ENABLED=true and provide a list of valid API keys as AUTHENTICATION_APIKEY_USERS in your docker-compose.yml file. More details are available in the Weaviate API key documentation.
2. OpenID Connect Configuration: Implementing OIDC for self-hosted Weaviate requires configuring environment variables that point to your OIDC provider's endpoints (e.g., OIDC_ISSUER, OIDC_CLIENT_ID, OIDC_CLIENT_SECRET). You must also ensure that Weaviate can communicate with your OIDC provider. The Weaviate OpenID Connect documentation provides comprehensive setup instructions.
3. No Authentication: For local development or testing environments, you might choose to disable authentication entirely. This is generally not recommended for production.

Authenticated request example

Integrating authentication into your Weaviate client code is straightforward using the official SDKs. Here's a Python example demonstrating how to initialize a Weaviate client with an API key for a Weaviate Cloud instance:

import weaviate
import os

# For Weaviate Cloud (WCS) instance
WEAVIATE_CLUSTER_URL = os.getenv("WEAVIATE_CLUSTER_URL") # e.g., "https://your-cluster-name.weaviate.network"
WEAVIATE_API_KEY = os.getenv("WEAVIATE_API_KEY") # Your Weaviate Admin API Key
WEAVIATE_WCS_API_KEY = os.getenv("WEAVIATE_WCS_API_KEY") # Your Weaviate Cloud Services API Key (X-WCS-Api-Key)

# Initialize client with WCS API key for cloud authentication
client = weaviate.Client(
    url=WEAVIATE_CLUSTER_URL,
    auth_client_secret=weaviate.AuthApiKey(api_key=WEAVIATE_API_KEY), # Typically the Weaviate Admin API Key
    additional_headers={
        "X-WCS-Api-Key": WEAVIATE_WCS_API_KEY # For Weaviate Cloud access control
    }
)

# Verify connection (optional)
if client.is_ready():
    print("Weaviate client is ready.")
    # Example: Get schema (requires appropriate permissions)
    try:
        schema = client.schema.get()
        print("Schema retrieved successfully.")
        # print(schema) 
    except Exception as e:
        print(f"Error getting schema: {e}")
else:
    print("Weaviate client is not ready.")

# Example for self-hosted instance with API key (assuming AUTHENTICATION_APIKEY_ENABLED=true and a key 'my-secret-key' is configured)
# client_self_hosted = weaviate.Client(
#     url="http://localhost:8080", # Or your self-hosted instance URL
# #    auth_client_secret=weaviate.AuthApiKey(api_key="my-secret-key"), # Only if using Weaviate's built-in API key auth
# )
# if client_self_hosted.is_ready():
#     print("Self-hosted Weaviate client is ready.")

For OIDC, the client initialization would involve passing the OIDC credentials, often handled through environment variables or a configuration object specific to the identity provider. The Weaviate client library abstract these details, allowing you to focus on your application logic.

Security best practices

Adhering to security best practices is essential for protecting your Weaviate instance and the data it contains:

• API Key Management:
  • Environment Variables: Store API keys as environment variables, not hardcoded in your source code. This prevents accidental exposure in version control systems.
  • Secret Management Services: For production, use dedicated secret management services like AWS Secrets Manager, Google Secret Manager, or Azure Key Vault to store and retrieve API keys securely.
  • Rotation: Regularly rotate your API keys. If a key is compromised, changing it limits the duration of potential unauthorized access.
  • Least Privilege: Grant API keys only the minimum necessary permissions. Avoid using highly privileged keys for routine operations.
• OpenID Connect (OIDC) Best Practices:
  • Secure Client Credentials: If your OIDC setup involves client secrets, treat them with the same care as API keys (environment variables, secret managers).
  • Token Expiration and Refresh: Configure short-lived access tokens and use refresh tokens securely. This minimizes the risk if an access token is intercepted.
  • Audience Validation: Ensure your Weaviate instance is configured to validate the aud (audience) claim in OIDC tokens to prevent tokens from being used in unintended contexts.
  • HTTPS Everywhere: Always enforce HTTPS for all communication with Weaviate and your OIDC provider to encrypt data in transit.
• Network Security:
  • Firewall Rules: Restrict network access to your Weaviate instance using firewall rules. Only allow traffic from known IP addresses or networks that host your applications.
  • VPC/Private Endpoints: In cloud environments, deploy Weaviate within a Virtual Private Cloud (VPC) and use private endpoints to ensure communication stays within your private network, bypassing the public internet where possible.
• Monitoring and Auditing:
  • Access Logs: Enable and regularly review access logs for your Weaviate instance to detect unusual activity or potential security breaches.
  • Alerting: Set up alerts for failed authentication attempts or suspicious access patterns.
• Regular Updates: Keep your Weaviate instance and client libraries updated to the latest stable versions to benefit from security patches and improvements.