Skip to main content

Datadog

0. Required Metrics​

Container and pod metrics are used to determine recommendations for individual workloads. Node and cluster metrics are used to determine cost and overall cluster health.

If you have custom metric names, please contact us for further assistance.

Expand to see all queries that Flightcrew runs (filtered by cluster)
Metric TypeQuery
CPU Capacitykubernetes_state.node.cpu_capacity.total{*}
CPU Capacitykubernetes_state.node.cpu_capacity{*} by {node}
CPU Limitkubernetes.cpu.limits{*} by {kube_namespace,pod_name,kube_container_name}
CPU Requestkubernetes.cpu.requests{*} by {kube_namespace,pod_name,kube_container_name}
CPU Usagekubernetes.cpu.usage.total{*} by {kube_namespace,pod_name,kube_container_name}
Container Restart Countkubernetes_state.container.restarts{*} by {kube_namespace,pod_name,kube_container_name}
Memory Capacitykubernetes_state.node.memory_capacity.total{*}
Memory Capacitykubernetes_state.node.memory_capacity{*} by {node}
Memory Limitkubernetes.memory.limits{*} by {kube_namespace,pod_name,kube_container_name}
Memory Requestkubernetes.memory.requests{*} by {kube_namespace,pod_name,kube_container_name}
Memory Usagekubernetes.memory.usage{*} by {kube_namespace,pod_name,kube_container_name}
Status Phasekubernetes_state.pod.status_phase{*} by {pod_name,kube_namespace,pod_phase}

1. Set up kube-state-metrics​

Skip this step if: kube-state-metrics is already set up with your Datadog installation. To verify this, check if there's data in the graph for an example metric kubernetes_state.container.ready in the metric Explorer.

Actions: Follow these instructions to set up kube-state-metrics in your Datadog agent. Reload your cluster agent with the updated configuration.

Verify: Use the Explorer link above to ensure data appears in the graph.

2. Ensure Datadog collects cluster names​

Skip this step if: Datadog collects the Kubernetes cluster's name. To verify this, check if there's data in the graph for an example metric system.cpu.user, which uses the kube_cluster_name field.

Actions: If the cluster name is N/A, then it can be added in one of two ways:

  • (Easiest) Ensure your Datadog agent is updated to a version that supports automatic cluster name detection (v6.11 or later), as described here.
  • Or, manually set the clusterName field in the helm chart used to install Datadog:
# For a new installation
helm install datadog-agent -f datadog-values.yaml --set datadog.clusterName=my_cluster_name --set targetSystem=linux datadog/datadog

# For updating an existing installation
helm upgrade datadog-agent -f datadog-values.yaml --set datadog.clusterName=my_cluster_name --set targetSystem=linux datadog/datadog

Verify: Use the Explorer link above to ensure data appears in the graph.

3. Create access keys​

  • Click here to create an API key by hitting the + New Key button.
  • Click here to create an Application key by hitting the + New Key button.
    • Optional: if possible, edit the Scope of this key to only include timeseries_query, since Flightcrew doesn't need access to anything else.