Grafana Agent

This is release v0.35.0 of Grafana Agent.

Upgrading

Read the relevant upgrade guides for specific instructions on upgrading from older versions:

• Static mode upgrade guide
• Static mode Kubernetes operator upgrade guide
• Flow mode upgrade guide

Notable changes:

BREAKING CHANGES: This release has breaking changes. Please read entries carefully and consult the upgrade guide for specific instructions.

Breaking changes

• The algorithm for the "hash" action of otelcol.processor.attributes has changed. The change was made in PR #22831 of opentelemetry-collector-contrib. (@ptodev)

• otelcol.exporter.loki now includes the instrumentation scope in its output. (@ptodev)

• otelcol.extension.jaeger_remote_sampling removes the / HTTP endpoint. The /sampling endpoint is still functional. The change was made in PR #18070 of opentelemetry-collector-contrib. (@ptodev)

• The field version and auth struct block from walk_params in prometheus.exporter.snmp and SNMP integration have been removed. The auth block now can be configured at top level, together with modules (@marctc)

• Rename discovery.file to local.file_match to make it more clear that it discovers file on the local filesystem, and so it doesn't get confused with Prometheus' file discovery. (@rfratto)

• Remove the discovery_target_decode function in favor of using discovery components to better match the behavior of Prometheus' service discovery. (@rfratto)

• In the traces subsystem for Static mode, some metrics are removed and others are renamed. (@ptodev)

  • Removed metrics:

    • "blackbox_exporter_config_last_reload_success_timestamp_seconds" (gauge)
    • "blackbox_exporter_config_last_reload_successful" (gauge)
    • "blackbox_module_unknown_total" (counter)
    • "traces_processor_tail_sampling_count_traces_sampled" (counter)
    • "traces_processor_tail_sampling_new_trace_id_received" (counter)
    • "traces_processor_tail_sampling_sampling_decision_latency" (histogram)
    • "traces_processor_tail_sampling_sampling_decision_timer_latency" (histogram)
    • "traces_processor_tail_sampling_sampling_policy_evaluation_error" (counter)
    • "traces_processor_tail_sampling_sampling_trace_dropped_too_early" (counter)
    • "traces_processor_tail_sampling_sampling_traces_on_memory" (gauge)
    • "traces_receiver_accepted_spans" (counter)
    • "traces_receiver_refused_spans" (counter)
    • "traces_exporter_enqueue_failed_log_records" (counter)
    • "traces_exporter_enqueue_failed_metric_points" (counter)
    • "traces_exporter_enqueue_failed_spans" (counter)
    • "traces_exporter_queue_capacity" (gauge)
    • "traces_exporter_queue_size" (gauge)

  • Renamed metrics:

    • "traces_receiver_refused_spans" is renamed to "traces_receiver_refused_spans_total"
    • "traces_receiver_accepted_spans" is renamed to "traces_receiver_refused_spans_total"
    • "traces_exporter_sent_metric_points" is renamed to "traces_exporter_sent_metric_points_total"

• The remote_sampling block has been removed from otelcol.receiver.jaeger. (@ptodev)

Deprecations

• otelcol.exporter.jaeger has been deprecated and will be removed in Agent v0.38.0. (@ptodev)

Features

• The Pyroscope scrape componen2 hours agot computes and sends delta profiles automatically when required to reduce bandwidth usage. (@cyriltovena)

• Support stage.geoip in loki.process. (@akselleirv)

• Integrations: Introduce the squid integration. (@armstrmi)

• Support custom fields in MMDB file for stage.geoip. (@akselleirv)

• Added json_path function to river stdlib. (@jkroepke)

• Add format, join, tp_lower, replace, split, trim, trim_prefix, trim_suffix, trim_space, to_upper functions to river stdlib. (@jkroepke)

• Flow UI: Add a view for listing the Agent's peers status when clustering is enabled. (@tpaschalis)

• Add a new CLI command grafana-agent convert for converting a river file from supported formats to river. (@erikbaranowski)

• Add support to the grafana-agent run CLI for converting a river file from supported formats to river. (@erikbaranowski)

• Add boringcrypto builds and docker images for Linux arm64 and x64. (@mattdurham)

• New Grafana Agent Flow components:

  • discovery.file discovers scrape targets from files. (@spartan0x117)
  • discovery.kubelet collect scrape targets from the Kubelet API. (@gcampbell12)
  • module.http runs a Grafana Agent Flow module loaded from a remote HTTP endpoint. (@spartan0x117)2 hours ago
  • otelcol.processor.attributes accepts telemetry data from other otelcol components and modifies attributes of a span, log, or metric. (@ptodev)
  • prometheus.exporter.cloudwatch - scrape AWS CloudWatch metrics (@thepalbi)
  • prometheus.exporter.elasticsearch collects metrics from Elasticsearch. (@marctc)
  • prometheus.exporter.kafka collects metrics from Kafka Server. (@oliver-zhang)
  • prometheus.exporter.mongodb collects metrics from MongoDB. (@marctc)
  • prometheus.exporter.squid collects metrics from a squid server. (@armstrmi)
  • prometheus.operator.probes - discovers Probe resources in your Kubernetes cluster and scrape the targets they reference. (@captncraig)
  • pyroscope.ebpf collects system-wide performance profiles from the current host (@korniltsev)
  • otelcol.exporter.loadbalancing - export traces and logs to multiple OTLP gRPC endpoints in a load-balanced way. (@ptodev)

• New Grafana Agent Flow command line utilities:

  • grafana-agent tools prometheus.remote_write holds a collection of remote write-specific tools. These have been ported over from the agentctl command. (@rfratto)

• A new action argument for otelcol.auth.headers. (@ptodev)

• New metadata_keys and metadata_cardinality_limit arguments for otelcol.processor.batch. (@ptodev)

• New boolean_attribute and ottl_condition sampling policies for otelcol.processor.tail_sampling. (@ptodev)

• A new initial_offset argument for otelcol.receiver.kafka. (@ptodev)

Other changes

• Mongodb integration has been re-enabled. (@jcreixell, @marctc)

For a full list of changes, please refer to the CHANGELOG!

Installation

Refer to our installation guides for how to install the variants of Grafana Agent:

• Install static mode
• Install the static mode Kubernetes operator
• Install flow mode