Loggly

Overview

Loggly, a SolarWinds product, provides cloud-based log management and analytics capabilities. It assists developers, DevOps teams, and IT operations personnel in centralizing log data from various sources, including applications, servers, cloud services, and custom devices (SolarWinds Documentation). The service is engineered to aggregate logs in real time, making them searchable and analyzable through a web interface.

The primary use cases for Loggly include real-time monitoring, troubleshooting application performance issues, and identifying operational anomalies. By collecting logs into a single platform, users can gain visibility across distributed systems without needing to access individual server logs. This centralization is critical for modern microservice architectures and cloud deployments where logs are often fragmented across many hosts and services.

Loggly offers features such as automated parsing of common log formats, custom parsing rules, and the ability to define alerts based on specific log patterns or metrics. Users can create dashboards to visualize log trends and identify recurring issues or performance bottlenecks. Its architecture is designed to handle varying volumes of log data, supporting both small development teams and larger enterprises with extensive logging requirements. The platform integrates with various notification systems, enabling teams to respond to critical events promptly.

The service is well-suited for organizations that prioritize rapid issue identification and resolution. Its focus on real-time analysis aligns with continuous delivery and DevOps practices, where prompt feedback loops on application behavior are essential. For example, during a production incident, engineering teams can use Loggly to quickly filter and search through relevant logs, correlating events across different system components to pinpoint root causes as described in the Loggly documentation. This contrasts with traditional methods that often involve manual log file inspection on individual machines, which can be time-consuming and error-prone.

Loggly supports a range of log ingestion methods, including standard syslog protocols, HTTP/S endpoints, and dedicated agents, providing flexibility in how log data is collected from diverse environments. This adaptability helps ensure that logs from virtually any source can be consolidated for analysis. The system's compliance certifications, such as SOC 2 Type II, GDPR, and HIPAA, address data security and privacy concerns for organizations operating in regulated industries.

Key features

• Real-time Log Aggregation: Collects logs from diverse sources (servers, applications, cloud services) into a centralized, searchable repository instantly.
• Automated Parsing and Indexing: Automatically parses common log formats (e.g., Apache, Nginx, JSON) and provides options for custom parsing rules to structure unstructured logs.
• Advanced Search and Filtering: Offers a robust search syntax and filtering capabilities to quickly isolate specific events, errors, or patterns within large volumes of log data.
• Custom Dashboards: Allows users to create visual dashboards with widgets to monitor key metrics, trends, and anomalies derived from log data.
• Alerting and Notifications: Configurable alerts based on log volume, specific error rates, or custom search criteria, integrated with notification channels like email, PagerDuty, and Slack.
• Root Cause Analysis: Tools for correlating events across different log sources and timeframes, assisting in the rapid identification of the root causes of application and infrastructure issues.
• Integrated Archiving: Options for archiving log data to cloud storage for long-term retention and compliance purposes.
• API for Log Ingestion: Provides API endpoints for programmatically sending logs, allowing for integration with custom applications and services.

Pricing

Loggly offers various plans scaled by data volume and retention period, including a free tier for basic usage.

Detailed pricing information and options for higher data volumes and retention periods are available on the Loggly Pricing page.

Common integrations

• AWS Services: Integrates with Amazon S3, CloudWatch, and other AWS services for collecting cloud infrastructure logs as detailed in the AWS integration guide.
• Azure Services: Supports ingestion of logs from Microsoft Azure resources.
• Google Cloud Platform (GCP): Collects logs from Google Cloud services.
• Heroku: Provides add-ons and syslog drains for logging from Heroku applications.
• Slack: Sends alerts and notifications directly to Slack channels.
• PagerDuty: Integrates for incident management and on-call alerting through the Loggly PagerDuty documentation.
• Jira: Creates tickets or issues in Jira based on log-generated alerts.
• GitHub: Can be configured to integrate with GitHub for project-related logging activities or alerts.
• Common Log Sources: Natively supports ingestion from syslog, HTTP/S, Apache, Nginx, Windows Event Logs, and various programming language loggers.

Alternatives

• Datadog Logs: A component of the Datadog observability platform, offering log management, metrics, and tracing in an integrated view.
• Splunk Cloud Platform: An enterprise-grade platform for machine data, including log management, security information, and event management (SIEM).
• New Relic Logs: Part of the New Relic One observability platform, providing log management capabilities alongside APM and infrastructure monitoring.
• Elastic Stack (Elasticsearch, Logstash, Kibana): An open-source suite commonly deployed for log management, search, and analytics. Elastic's documentation provides an overview.
• Sumo Logic: A cloud-native machine data analytics platform for logs and metrics, focusing on operational intelligence and security.

Getting started

To get started with Loggly, you can send logs using various methods. One common method for Linux-based systems is to configure syslog. Below is an example of sending a test log message using logger, after configuring your system's rsyslog daemon to forward logs to Loggly. Replace YOUR_CUSTOMER_TOKEN with your actual Loggly customer token.

First, ensure rsyslog is configured to send logs to Loggly. Create a new rsyslog configuration file (e.g., /etc/rsyslog.d/22-loggly.conf) with the following content:

# For secure TLS connection
$DefaultNetstreamDriverCAFile /etc/ssl/certs/ca-certificates.crt # Ubuntu/Debian
# $DefaultNetstreamDriverCAFile /etc/pki/tls/certs/ca-bundle.crt # RHEL/CentOS
$ModLoad imuxsock
$ModLoad imjournal # For systemd journal logs

# Send all local messages to Loggly over TLS
*.* @@logs-01.loggly.com:6514;LogglyFormat # If using TLS
*.* @logs-01.loggly.com:514;LogglyFormat   # If using UDP (less secure, not recommended for production)

# Define the LogglyFormat template
$template LogglyFormat,"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag% %YOUR_CUSTOMER_TOKEN% %msg%\n"

After saving the configuration, restart rsyslog:

sudo service rsyslog restart

Now, send a test log message using the logger command:

logger -t myapp "This is a test log message from Loggly setup."

You should then be able to see this message appear in your Loggly dashboard within seconds. For more detailed instructions on specific log sources and ingestion methods, refer to the Loggly documentation.