ELK Stack#

The ELK stack — Elasticsearch, Logstash, Kibana — is a mandatory component of the exam's target architecture for the DevOps-Docker final project 1. It cannot be substituted with alternatives like Loki/Grafana for the primary grading criteria 2.

Role in the Architecture#

The stack handles centralized log management across all services 3:

• Logstash collects and transforms logs from the reverse-proxy and application server
• Elasticsearch indexes those logs
• Kibana provides dashboards and search UI, accessible on port 5601

The data flow is: app/proxy → Logstash → Elasticsearch → Kibana.

Minimum Requirements#

The exam spec mandates 4:

• Logs from both the reverse-proxy and the app server sent to Logstash
• Elasticsearch indexing those logs
• Kibana accessible and showing at least one basic dashboard (volume by service, log table, level filter)
• JSON log format where possible

Docker Compose Integration#

The ELK services must use depends_on / condition: service_healthy to prevent dependent services from starting before ELK is ready 5. Named volumes are required — anonymous volumes are not accepted 6:

volumes:
  elasticsearch_data:
    driver: local

Recommended resource limits 7:

Logstash Configuration#

The canonical input/output pipeline for this architecture 8:

input {
  tcp { port => 5000; codec => json }
}
output {
  elasticsearch {
    hosts => ["elasticsearch:9200"]
    index => "app-logs-%{+YYYY.MM.dd}"
  }
}

For Java apps, use the LogstashTcpSocketAppender (Logback) to ship logs to logstash:5000.

Elasticsearch Healthcheck#

The basic healthcheck in the exam spec uses a plain HTTP call:

healthcheck:
  test: ["CMD-SHELL", "curl -f http://localhost:9200/_cluster/health || exit 1"]
  interval: 30s
  timeout: 10s
  retries: 5
  start_period: 60s

⚠️ ES 8+/9+ Compatibility Warning
Starting with Elasticsearch 8.0, security — including TLS and authentication — is enabled by default (release highlights). This means:

• http://localhost:9200 will return a connection error or redirect; you must use https://
• The elastic user password is auto-generated at first boot
  The plain-HTTP healthcheck above will fail silently against an ES 8+/9+ image without additional configuration. Fix options:

1. Disable security (dev/exam context only): set xpack.security.enabled=false in the ES environment
2. Adapt the healthcheck to HTTPS with credentials:

⧉

test: ["CMD-SHELL", "curl -fs -u elastic:${ELASTIC_PASSWORD} --cacert /usr/share/elasticsearch/config/certs/http_ca.crt https://localhost:9200/_cluster/health || exit 1"]

3. Use the bundled elasticsearch-reset-password / CLI tools to probe readiness instead of curl
   Option 1 is the simplest for local/exam work and pairs with discovery.type=single-node to avoid cluster bootstrap issues.

Common Pitfall: GELF Logging Address#

A documented exam failure mode: configuring the GELF logging driver with localhost instead of the Logstash service name 9. localhost inside a container refers only to that container — use the service name instead:

# ❌ WRONG
gelf-address: "udp://localhost:12201"

# ✅ CORRECT
gelf-address: "udp://logstash:12201"

This error accounts for an average +1.30 point impact on exam scores 10.

Note: Healthchecks are the one place where localhost is correct — they run inside the target container itself 11.

Environment Variables#

The .env.example template includes ELK-specific variables 12:

ELASTIC_PASSWORD=CHANGEME
KIBANA_PASSWORD=CHANGEME

These must never appear in plaintext in compose.yml.

Primary Sources#