On 29 May 2024, at 09:13, Barry Scott barry@barrys-emacs.org wrote:
$ timedatectl Local time: Wed 2024-05-29 09:03:22 BST Universal time: Wed 2024-05-29 08:03:22 UTC RTC time: Thu 2024-05-30 00:09:00 Time zone: Europe/London (BST, +0100) System clock synchronized: yes NTP service: active RTC in local TZ: no
Barry
I just found that this side-effect broke prometheus...
2024-05-29T12:06:11+01:00 prometheus[14814]: ts=2024-05-29T11:06:11.657Z caller=scrape.go:1570 level=warn component="scrape manager" scrape_pool=node_exporter target=http://fender:9100/metrics msg="Error on ingesting samples that are too old or are too far into the future" num_dropped=1552
So all samples will be rejected until the midnight tonight as prometheus uses the time of the last entry in its database for its logic to reject bad values.
Barry