host-metrics/docs/VALIDATION.md
2026-09-17 22:31:35 +02:00

42 lines
3.2 KiB
Markdown

# Validation record
Date: 2026-09-17. Development environment: macOS arm64; no Docker daemon, no Debian VM connection supplied.
## Completed locally
- Confirmed stable release metadata and published Docker image tags. Kafka release candidates were excluded.
- Compared Kafka JSON serialization, host input options, JSON v2 parsing, Prometheus output, Starlark metric state and Data Prepper configuration against upstream documentation/source. Used `inputs.system.include` to emit a single numeric system measurement; removed an obsolete network option and used a tag filter for protocol totals. Corrected the process-state field to the documented singular `zombie`.
- Validated the complete Compose model, including the optional profile, with the official standalone **Docker Compose v5.5.1** CLI using a temporary test encryption key. This runs `config --quiet` and does not require/start Docker.
- Parsed all mounted YAML configurations with Ruby's YAML parser; parsed both Telegraf configurations with Python's TOML parser.
- Checked nine shell scripts with `bash -n`.
- Ran four Python test cases covering rate isolation/reset/gap handling, disk busy/load calculations, the saved-object reference graph and non-overlapping dashboard layouts, and the metric configuration/mapping contract.
- Generated 48 saved objects: 4 dashboards, 43 visualizations/navigation panels and one index pattern. NDJSON and pretty JSON representations are equivalent.
The rate test executes the deliberately Python-compatible subset of the Starlark script in Python. It checks mathematics/state behavior; it is **not** validation by the real Starlark interpreter. An official native Telegraf binary download for local runtime validation returned HTTP 403, so that runtime check was not completed.
## Not completed in this environment
- Image startup and full Kafka/Data Prepper/OpenSearch delivery.
- Debian APT installation, systemd sandbox behavior and collection under the real `telegraf` account.
- Real Telegraf configuration parsing, Starlark execution and optional JSON-to-Prometheus conversion.
- Import into a running OpenSearch Dashboards instance and visual browser inspection.
- SQL/Prometheus connector execution against the running pinned stack.
- Throughput, outage durability, recovery and memory/disk capacity benchmarks.
## Run on Debian
```bash
sudo ./scripts/prepare-host.sh
./scripts/start.sh
sudo ./scripts/install-agent.sh
./scripts/smoke-test.sh
./scripts/import-dashboards.sh
./scripts/enable-metric-analytics.sh
# After about one minute:
python3 ./scripts/check-metric-analytics.py
```
The installer performs a real Telegraf `--test` as the service user before starting its dedicated service. The ingestion smoke test requires recent CPU-total, memory, filesystem and disk/network rate data, identity tags, timestamps and expected index field types. Dashboard import checks the API's `success` flag, not just its HTTP status. The optional checker requires both Prometheus samples and a successful federated PPL query.
Finally inspect all four dashboards in the browser and perform the manual acceptance exercise in [OPERATIONS.md](OPERATIONS.md). A complete POC implementation is supplied, but deployment/runtime compatibility is not represented as already proven.