Authentication overview
Elasticsearch implements a comprehensive security model to control access to its data and APIs. Authentication is the process of verifying the identity of a user or system attempting to interact with an Elasticsearch cluster. This is distinct from authorization, which determines what an authenticated entity is permitted to do.
The security features, including authentication, are part of the Elastic Stack's X-Pack components. These features are enabled by default in Elastic Cloud deployments and can be configured in self-managed clusters. Effective authentication ensures that only legitimate clients, applications, and administrators can perform operations, protecting sensitive data and maintaining the integrity of the search and analytics platform.
Elasticsearch supports a range of authentication methods, allowing administrators to choose the most appropriate strategy based on their security requirements, existing identity infrastructure, and operational context. All communication with Elasticsearch should occur over HTTPS to protect credentials and data in transit, as detailed in the Elasticsearch security settings documentation.
Supported authentication methods
Elasticsearch offers multiple authentication methods, catering to various deployment scenarios and integration needs. The choice of method often depends on the client type (e.g., human user, service account, application), the desired level of security, and integration with existing identity management systems.
Basic authentication
This is the most common and straightforward method, involving a username and password. Elasticsearch manages an internal user database, or it can integrate with external systems. When using basic authentication, credentials are typically sent in the HTTP Authorization header.
API keys
API keys provide a secure and flexible way to grant access to applications and services without exposing user credentials. They are long-lived tokens that can be generated with specific privileges and invalidated when no longer needed. API keys are particularly useful for automated tasks and microservices interactions.
Token-based authentication
Elasticsearch supports various token-based authentication mechanisms, including:
- Bearer tokens: Often used with OpenID Connect or SAML integrations, where an external identity provider issues a token that Elasticsearch then validates.
- Service account tokens: Similar to API keys but often managed within a Kubernetes or cloud environment for service-to-service communication.
External identity providers
For enterprise environments, Elasticsearch can integrate with external identity management systems:
- SAML: Security Assertion Markup Language (SAML) enables single sign-on (SSO) for users managed by a SAML identity provider, such as Okta or Azure AD. This is a common choice for web applications and corporate environments. The Elasticsearch SAML realm documentation provides configuration details.
- OpenID Connect (OIDC): An identity layer built on top of the OAuth 2.0 protocol, OIDC allows Elasticsearch to authenticate users against OIDC providers like Google, Auth0, or Keycloak. It's often preferred for modern web and mobile applications. Further information on this standard can be found in the OpenID Connect specification.
- LDAP/Active Directory: Lightweight Directory Access Protocol (LDAP) and Microsoft Active Directory (AD) realms allow Elasticsearch to authenticate users directly against existing directory services. This is a prevalent method in corporate networks.
- Kerberos: Provides strong authentication for users and services in a network environment, commonly used in large-scale enterprise deployments with Windows or Linux servers.
Table of Authentication Methods
| Method | When to Use | Security Level (General) |
|---|---|---|
| Basic Authentication (Internal Realm) | Small deployments, internal tools, quick setup, initial administration | Moderate (requires secure password management) |
| API Keys | Automated processes, microservices, application-to-Elasticsearch communication | High (granular control, revocable) |
| SAML | Enterprise SSO, web applications, integration with corporate identity providers | High (leveraging IdP security features) |
| OpenID Connect | Modern web/mobile apps, integration with cloud identity providers, developer-friendly SSO | High (leveraging IdP security features, OAuth 2.0 based) |
| LDAP/Active Directory | Corporate networks, existing directory services, centralized user management | High (integrates with established enterprise security) |
| Kerberos | Large enterprise environments, strong network authentication, Windows/Linux server integration | Very High (mutual authentication, ticket-based) |
Getting your credentials
The process for obtaining credentials varies based on the chosen authentication method and whether you are using Elastic Cloud or a self-managed Elasticsearch cluster.
Elastic Cloud
When you create an Elastic Cloud deployment, the system automatically generates a username (elastic) and a strong password for initial access. This information is typically provided in the deployment overview or a download file. For additional users or API keys:
- Users: Navigate to the Kibana Security UI (Stack Management > Security > Users) to create new users and assign roles.
- API Keys: In Kibana, go to Stack Management > Security > API Keys. You can create new API keys, specifying their associated roles and expiration dates. The API key ID and API key string are presented only once upon creation, so they must be stored securely.
- Token management: For advanced scenarios involving SAML or OIDC, configuration is done under Stack Management > Security > Realms, where you define the external identity provider settings.
Self-managed Elasticsearch
For self-managed clusters, you typically start by enabling security features in your elasticsearch.yml configuration file. The initial setup involves running the elasticsearch-setup-passwords utility to set passwords for built-in users (elastic, kibana_system, etc.).
- Users: Use the Elasticsearch Users API (e.g.,
POST /_security/user/) or the Kibana Security UI to create and manage users. - API Keys: API keys can be generated using the Create API Key API (
POST /_security/api_key). The response includes theidandapi_keyfields, which together form the API key. - External Realms: Configuration for LDAP, Active Directory, SAML, or OpenID Connect realms is done directly in the
elasticsearch.ymlfile, specifying the connection details and metadata for your identity provider.
Authenticated request example
Once you have your credentials, you can use them to authenticate requests to Elasticsearch. The following examples demonstrate how to make an authenticated request using Basic Authentication and an API Key.
Basic Authentication (cURL)
This example uses the elastic superuser and its password to retrieve information about the cluster. Replace <your_password> with the actual password and <your_elasticsearch_host> with your cluster's endpoint.
curl -X GET "https://<your_elasticsearch_host>:9243/_cluster/health?pretty" \
-H "Content-Type: application/json" \
-u elastic:<your_password>
The -u flag in cURL automatically constructs the Authorization: Basic header by base64-encoding the username and password.
API Key Authentication (cURL)
To use an API key, you combine the API key ID and the API key string, base64-encode them, and include them in the Authorization: ApiKey header. Replace <api_key_id> and <api_key_string> with your actual key components.
API_KEY_ENCODED=$(echo -n "<api_key_id>:<api_key_string>" | base64)
curl -X GET "https://<your_elasticsearch_host>:9243/_cluster/health?pretty" \
-H "Content-Type: application/json" \
-H "Authorization: ApiKey $API_KEY_ENCODED"
This method is generally preferred for programmatic access due to its revocability and fine-grained permissions.
Python example with Basic Authentication
Using the official Elasticsearch Python client, basic authentication is handled by passing http_auth parameters.
from elasticsearch import Elasticsearch
# Replace with your Elasticsearch host and credentials
ES_HOST = "https://<your_elasticsearch_host>:9243"
ES_USER = "elastic"
ES_PASSWORD = "<your_password>"
# Initialize the client with basic authentication
es = Elasticsearch(
[ES_HOST],
http_auth=(ES_USER, ES_PASSWORD),
verify_certs=True # Or False if using self-signed certs (not recommended for production)
)
# Perform a request
response = es.cluster.health()
print(response)
Security best practices
Implementing strong authentication is a critical component of overall Elasticsearch security. Adhering to best practices helps protect your data from unauthorized access and malicious activities.
Enable TLS/SSL encryption
Always encrypt communication between clients and Elasticsearch, and between nodes within the cluster, using TLS/SSL (HTTPS). This prevents eavesdropping and tampering with data and credentials in transit. Elastic Cloud enforces TLS by default. For self-managed clusters, configure TLS certificates as described in the Elasticsearch TLS configuration guide.
Use strong, unique passwords
For any user accounts, enforce strong password policies, including complexity requirements, minimum length, and regular rotation. Avoid reusing passwords across different systems.
Implement Role-Based Access Control (RBAC)
After authentication, authorization determines what an authenticated user or application can do. Define roles with the principle of least privilege, granting only the necessary permissions for specific tasks. For example, a monitoring tool might only need read access to certain indices, while an application might need write access to its specific data stream. Elasticsearch allows granular control over cluster, index, and document-level permissions. Refer to the Elasticsearch authorization documentation for detailed role management.
Leverage API keys for service accounts
Instead of using user credentials for programmatic access, generate API keys. API keys offer several advantages: they can be scoped to specific roles, have an expiration date, and can be easily revoked without affecting other users or services. This reduces the blast radius in case a credential is compromised.
Integrate with external identity providers
Whenever possible, integrate Elasticsearch with your organization's centralized identity provider (e.g., SAML, OIDC, LDAP, Active Directory). This streamlines user management, enforces consistent authentication policies, and often enables single sign-on (SSO), improving both security and user experience. It also centralizes audit trails for user access.
Regularly audit security logs
Monitor Elasticsearch security logs for unusual authentication attempts, failed logins, or unauthorized access attempts. Elasticsearch's auditing features can log security-related events, providing valuable insights into potential breaches or misconfigurations. The Elasticsearch auditing documentation explains how to enable and configure auditing.
Restrict network access
Implement network-level controls, such as firewalls and security groups, to restrict direct access to Elasticsearch ports (typically 9200 and 9243) from untrusted networks. Only allow connections from authorized applications, services, and administrative jump boxes. Consider using private endpoints or VPC peering in cloud environments.
Keep Elasticsearch updated
Regularly update your Elasticsearch clusters to the latest stable version. Security patches and enhancements are frequently released, addressing newly discovered vulnerabilities. Staying current ensures you benefit from the latest security features and fixes, as highlighted in the Elastic Security Center.
Consider multi-factor authentication (MFA)
While Elasticsearch itself doesn't directly provide MFA for its internal realm, integrating with identity providers that support MFA (like SAML or OIDC providers) allows you to enforce stronger authentication for human users. This adds an extra layer of security beyond just a password.