Skip to content

Instantly share code, notes, and snippets.

@ringerc
Last active August 7, 2025 22:50
Show Gist options
  • Save ringerc/a3f98f2fb667d425b38ba0b1ea03b853 to your computer and use it in GitHub Desktop.
Save ringerc/a3f98f2fb667d425b38ba0b1ea03b853 to your computer and use it in GitHub Desktop.
# Tests illustrating a bug where histogram samples are incorrectly discarded as out-of-order
# when a load directive contains a single series that combines float and histogram samples
# ( https://github.com/prometheus/prometheus/pull/17021 )
#
# Place in promql/promqltest/testdata/mix_histogram_over_time.test
#
# Run from prometheus source code root dir with
#
# go test -v -run 'TestEvaluations/testdata/mix_histogram_over_time.test' ./promql
#
#
load 10s
data_histogram{type="mix_histfirst"} {{schema:0 sum:1 count:2}} {{schema:0 sum:2 count:3}} 0 1
# Test result matching for this will fail with a complaint:
#
# error in eval count_over_time(data_histogram[2m]) (line 8): expected 4 for {type="mix_histfirst"} but got 2
#
# because the two histogram samples got discarded silently as out-of-order.
eval instant at 1m count_over_time(data_histogram[2m])
expect no_info
expect no_warn
{type="mix_histfirst"} 4
clear
# To work around this bug in the appender (which probably really only affects
# the test suite in practice), load histogram samples alone as the first
# samples in the series:
load 10s
data_histogram{type="mix_histfirst"} {{schema:0 sum:1 count:2}} {{schema:0 sum:2 count:3}}
# then append some float samples in a separate transaction. This must be a separate load directive
# to ensure the appender is committed, so the histogram samples are not falsely rejected as
# out-of-order vis-a-vis the float samples.
load 10s
data_histogram{type="mix_histfirst"} _ _ 0 1
# As a result, this test will pass
eval instant at 1m count_over_time(data_histogram[2m])
expect no_warn
expect no_info
{type="mix_histfirst"} 4
clear
# output of go test -v -run 'TestEvaluations/testdata/mix_histogram_over_time.test' ./promql
# annotated
=== RUN TestEvaluations
=== RUN TestEvaluations/testdata/mix_histogram_over_time.test
=== RUN TestEvaluations/testdata/mix_histogram_over_time.test/line_13/count_over_time(data_histogram[2m])
test.go:1321:
Error Trace: /home/craig/upm/upm-stack/prometheus/promql/promqltest/test.go:1321
Error: Received unexpected error:
error in eval count_over_time(data_histogram[2m]) (line 13): expected 4 for {type="mix_histfirst"} but got 2
Test: TestEvaluations/testdata/mix_histogram_over_time.test/line_13/count_over_time(data_histogram[2m])
=== RUN TestEvaluations/testdata/mix_histogram_over_time.test/line_35/count_over_time(data_histogram[2m])
--- FAIL: TestEvaluations (0.06s)
--- FAIL: TestEvaluations/testdata/mix_histogram_over_time.test (0.06s)
# **** this fails because of the load dropping the histogram samples ****
--- FAIL: TestEvaluations/testdata/mix_histogram_over_time.test/line_13/count_over_time(data_histogram[2m]) (0.00s)
# **** then the repeat run with the workaround split load passes ****
--- PASS: TestEvaluations/testdata/mix_histogram_over_time.test/line_35/count_over_time(data_histogram[2m]) (0.00s)
FAIL
FAIL github.com/prometheus/prometheus/promql 0.072s
FAIL
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment