What was covered
- Why LLM apps are different. Inputs are unstructured, outputs are probabilistic, and failures can be silent. Classic observability alone does not show them.
- A division of labour. Langfuse covers LLM semantics, model debugging and cost. The LGTM stack (Loki, Tempo, Mimir, Grafana) covers system performance, infrastructure and alerting.
- One pipe for both. OpenTelemetry is the open, language-agnostic standard for collecting the data. It feeds Alloy, a telemetry collector that sends LLM-specific data to Langfuse and everything else to LGTM.
What came up
- Dashboards and alerts. Plenty of ready-made dashboards exist for both tools. Both can raise alerts and triggers, which helps automation.
- Where to look. Backend errors show up in Grafana. Token usage shows up in Langfuse, and error rates can be seen in either.
- Why it mattered. A project proposal in the room needed exactly this kind of alerting and logging.
Takeaway
Treat the two tools as complementary rather than competing, and let OpenTelemetry be the single entry point.