diff --git a/docs/design/metrics.md b/docs/design/metrics.md index be917c0dc6..0ae4203997 100644 --- a/docs/design/metrics.md +++ b/docs/design/metrics.md @@ -42,7 +42,7 @@ These are documented under [Inferencing and Serving -> Production Metrics](../us ### Grafana Dashboard -vLLM also provides [a reference example](../../examples/online_serving/prometheus_grafana/README.md) for how to collect and store these metrics using Prometheus and visualize them using a Grafana dashboard. +vLLM also provides [a reference example](../../examples/observability/prometheus_grafana/README.md) for how to collect and store these metrics using Prometheus and visualize them using a Grafana dashboard. The subset of metrics exposed in the Grafana dashboard gives us an indication of which metrics are especially important: @@ -657,7 +657,7 @@ vLLM has support for OpenTelemetry tracing: - Added by and reinstated by - Configured with `--oltp-traces-endpoint` and `--collect-detailed-traces` - [OpenTelemetry blog post](https://opentelemetry.io/blog/2024/llm-observability/) -- [User-facing docs](../../examples/online_serving/opentelemetry/README.md) +- [User-facing docs](../../examples/observability/opentelemetry/README.md) - [Blog post](https://medium.com/@ronen.schaffer/follow-the-trail-supercharging-vllm-with-opentelemetry-distributed-tracing-aa655229b46f) - [IBM product docs](https://www.ibm.com/docs/en/instana-observability/current?topic=mgaa-monitoring-large-language-models-llms-vllm-public-preview) diff --git a/examples/online_serving/dashboards/README.md b/examples/observability/dashboards/README.md similarity index 92% rename from examples/online_serving/dashboards/README.md rename to examples/observability/dashboards/README.md index 10b9a864f5..e5f5010a42 100644 --- a/examples/online_serving/dashboards/README.md +++ b/examples/observability/dashboards/README.md @@ -74,8 +74,8 @@ percli apply -f perses/performance_statistics.yaml For detailed deployment instructions and platform-specific options, see: -- **[Grafana Documentation](./grafana)** - JSON dashboards, operator usage, manual import -- **[Perses Documentation](./perses)** - YAML specs, CLI usage, operator wrapping +- **[Grafana Documentation](grafana)** - JSON dashboards, operator usage, manual import +- **[Perses Documentation](perses)** - YAML specs, CLI usage, operator wrapping ## Contributing diff --git a/examples/online_serving/dashboards/grafana/README.md b/examples/observability/dashboards/grafana/README.md similarity index 100% rename from examples/online_serving/dashboards/grafana/README.md rename to examples/observability/dashboards/grafana/README.md diff --git a/examples/online_serving/dashboards/grafana/performance_statistics.json b/examples/observability/dashboards/grafana/performance_statistics.json similarity index 100% rename from examples/online_serving/dashboards/grafana/performance_statistics.json rename to examples/observability/dashboards/grafana/performance_statistics.json diff --git a/examples/online_serving/dashboards/grafana/query_statistics.json b/examples/observability/dashboards/grafana/query_statistics.json similarity index 100% rename from examples/online_serving/dashboards/grafana/query_statistics.json rename to examples/observability/dashboards/grafana/query_statistics.json diff --git a/examples/online_serving/dashboards/perses/README.md b/examples/observability/dashboards/perses/README.md similarity index 100% rename from examples/online_serving/dashboards/perses/README.md rename to examples/observability/dashboards/perses/README.md diff --git a/examples/online_serving/dashboards/perses/performance_statistics.yaml b/examples/observability/dashboards/perses/performance_statistics.yaml similarity index 100% rename from examples/online_serving/dashboards/perses/performance_statistics.yaml rename to examples/observability/dashboards/perses/performance_statistics.yaml diff --git a/examples/online_serving/dashboards/perses/query_statistics.yaml b/examples/observability/dashboards/perses/query_statistics.yaml similarity index 100% rename from examples/online_serving/dashboards/perses/query_statistics.yaml rename to examples/observability/dashboards/perses/query_statistics.yaml diff --git a/examples/offline_inference/metrics.py b/examples/observability/metrics/offline.py similarity index 100% rename from examples/offline_inference/metrics.py rename to examples/observability/metrics/offline.py diff --git a/examples/online_serving/opentelemetry/README.md b/examples/observability/opentelemetry/README.md similarity index 100% rename from examples/online_serving/opentelemetry/README.md rename to examples/observability/opentelemetry/README.md diff --git a/examples/online_serving/opentelemetry/dummy_client.py b/examples/observability/opentelemetry/dummy_client.py similarity index 100% rename from examples/online_serving/opentelemetry/dummy_client.py rename to examples/observability/opentelemetry/dummy_client.py diff --git a/examples/online_serving/prometheus_grafana/README.md b/examples/observability/prometheus_grafana/README.md similarity index 100% rename from examples/online_serving/prometheus_grafana/README.md rename to examples/observability/prometheus_grafana/README.md diff --git a/examples/online_serving/prometheus_grafana/docker-compose.yaml b/examples/observability/prometheus_grafana/docker-compose.yaml similarity index 100% rename from examples/online_serving/prometheus_grafana/docker-compose.yaml rename to examples/observability/prometheus_grafana/docker-compose.yaml diff --git a/examples/online_serving/prometheus_grafana/grafana.json b/examples/observability/prometheus_grafana/grafana.json similarity index 100% rename from examples/online_serving/prometheus_grafana/grafana.json rename to examples/observability/prometheus_grafana/grafana.json diff --git a/examples/online_serving/prometheus_grafana/prometheus.yaml b/examples/observability/prometheus_grafana/prometheus.yaml similarity index 100% rename from examples/online_serving/prometheus_grafana/prometheus.yaml rename to examples/observability/prometheus_grafana/prometheus.yaml