Jan 22 23:49:28 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:49:28 uut-ct0 loki[5077]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:49:28 uut-ct0 loki[5077]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:49:28 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:49:28 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:49:30 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 1.
Jan 22 23:49:30 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:49:30 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:49:30 uut-ct0 loki[5101]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:49:30 uut-ct0 loki[5101]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:49:30 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:49:30 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:49:32 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 2.
Jan 22 23:49:32 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:49:32 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:49:33 uut-ct0 loki[5108]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:49:33 uut-ct0 loki[5108]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:49:33 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:49:33 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:49:35 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 3.
Jan 22 23:49:35 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:49:35 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:49:35 uut-ct0 loki[5116]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:49:35 uut-ct0 loki[5116]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:49:35 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:49:35 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:49:37 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 4.
Jan 22 23:49:37 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:49:37 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:49:37 uut-ct0 loki[5124]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:49:37 uut-ct0 loki[5124]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:49:37 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:49:37 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:49:39 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 5.
Jan 22 23:49:39 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:49:39 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:49:39 uut-ct0 loki[5131]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:49:39 uut-ct0 loki[5131]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:49:39 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:49:39 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:49:41 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 6.
Jan 22 23:49:41 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:49:41 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:49:42 uut-ct0 loki[5138]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:49:42 uut-ct0 loki[5138]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:49:42 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:49:42 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:49:44 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 7.
Jan 22 23:49:44 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:49:44 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:49:44 uut-ct0 loki[5146]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:49:44 uut-ct0 loki[5146]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:49:44 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:49:44 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:49:46 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 8.
Jan 22 23:49:46 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:49:46 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:49:46 uut-ct0 loki[5153]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:49:46 uut-ct0 loki[5153]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:49:46 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:49:46 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:49:48 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 9.
Jan 22 23:49:48 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:49:48 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:49:48 uut-ct0 loki[5160]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:49:48 uut-ct0 loki[5160]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:49:48 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:49:48 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:49:50 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 10.
Jan 22 23:49:50 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:49:50 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:49:51 uut-ct0 loki[5167]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:49:51 uut-ct0 loki[5167]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:49:51 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:49:51 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:49:53 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 11.
Jan 22 23:49:53 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:49:53 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:49:53 uut-ct0 loki[5174]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:49:53 uut-ct0 loki[5174]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:49:53 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:49:53 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:49:55 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 12.
Jan 22 23:49:55 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:49:55 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:49:55 uut-ct0 loki[5181]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:49:55 uut-ct0 loki[5181]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:49:55 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:49:55 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:49:57 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 13.
Jan 22 23:49:57 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:49:57 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:49:57 uut-ct0 loki[5188]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:49:57 uut-ct0 loki[5188]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:49:57 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:49:57 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:49:59 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 14.
Jan 22 23:49:59 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:49:59 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:50:00 uut-ct0 loki[5195]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:50:00 uut-ct0 loki[5195]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:50:00 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:50:00 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:50:02 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 15.
Jan 22 23:50:02 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:50:02 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:50:02 uut-ct0 loki[5202]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:50:02 uut-ct0 loki[5202]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:50:02 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:50:02 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:50:04 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 16.
Jan 22 23:50:04 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:50:04 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:50:04 uut-ct0 loki[5209]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:50:04 uut-ct0 loki[5209]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:50:04 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:50:04 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:50:06 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 17.
Jan 22 23:50:06 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:50:06 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:50:06 uut-ct0 loki[5216]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:50:06 uut-ct0 loki[5216]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:50:06 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:50:06 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:50:08 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 18.
Jan 22 23:50:08 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:50:08 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:50:09 uut-ct0 loki[5223]: failed parsing config: /etc/loki/config.yml: yaml: unmarshal errors:
Jan 22 23:50:09 uut-ct0 loki[5223]: line 44: field enabled not found in type aggregation.Config. Use `-config.expand-env=true` flag if you want to expand environment variables in your config file
Jan 22 23:50:09 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:50:09 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:50:11 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 19.
Jan 22 23:50:11 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:50:11 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:50:11 uut-ct0 loki[5420]: mkdir /var/lib/loki: permission denied
Jan 22 23:50:11 uut-ct0 loki[5420]: error initialising module: ruler-storage
Jan 22 23:50:11 uut-ct0 loki[5420]: github.com/grafana/dskit/modules.(*Manager).initModule
Jan 22 23:50:11 uut-ct0 loki[5420]: /src/loki/vendor/github.com/grafana/dskit/modules/modules.go:138
Jan 22 23:50:11 uut-ct0 loki[5420]: github.com/grafana/dskit/modules.(*Manager).InitModuleServices
Jan 22 23:50:11 uut-ct0 loki[5420]: /src/loki/vendor/github.com/grafana/dskit/modules/modules.go:108
Jan 22 23:50:11 uut-ct0 loki[5420]: github.com/grafana/loki/v3/pkg/loki.(*Loki).Run
Jan 22 23:50:11 uut-ct0 loki[5420]: /src/loki/pkg/loki/loki.go:492
Jan 22 23:50:11 uut-ct0 loki[5420]: main.main
Jan 22 23:50:11 uut-ct0 loki[5420]: /src/loki/cmd/loki/main.go:129
Jan 22 23:50:11 uut-ct0 loki[5420]: runtime.main
Jan 22 23:50:11 uut-ct0 loki[5420]: /usr/local/go/src/runtime/proc.go:272
Jan 22 23:50:11 uut-ct0 loki[5420]: runtime.goexit
Jan 22 23:50:11 uut-ct0 loki[5420]: /usr/local/go/src/runtime/asm_amd64.s:1700
Jan 22 23:50:11 uut-ct0 loki[5420]: level=info ts=2025-01-22T23:50:11.316023904Z caller=main.go:126 msg="Starting Loki" version="(version=3.3.2, branch=release-3.3.x, revision=23b5fc2c)"
Jan 22 23:50:11 uut-ct0 loki[5420]: level=info ts=2025-01-22T23:50:11.316062365Z caller=main.go:127 msg="Loading configuration file" filename=/etc/loki/config.yml
Jan 22 23:50:11 uut-ct0 loki[5420]: level=error ts=2025-01-22T23:50:11.317320239Z caller=log.go:216 msg="error running loki" err="mkdir /var/lib/loki: permission denied\nerror initialising module: ruler-storage\ngithub.com/grafana/dskit/modules.(*Manager).initModule\n\t/src/loki/vendor/github.com/grafana/dskit/modules/modules.go:138\ngithub.com/grafana/dskit/modules.(*Manager).InitModuleServices\n\t/src/loki/vendor/github.com/grafana/dskit/modules/modules.go:108\ngithub.com/grafana/loki/v3/pkg/loki.(*Loki).Run\n\t/src/loki/pkg/loki/loki.go:492\nmain.main\n\t/src/loki/cmd/loki/main.go:129\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:272\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1700"
Jan 22 23:50:11 uut-ct0 systemd[1]: loki.service: Main process exited, code=exited, status=1/FAILURE
Jan 22 23:50:11 uut-ct0 systemd[1]: loki.service: Failed with result 'exit-code'.
Jan 22 23:50:13 uut-ct0 systemd[1]: loki.service: Scheduled restart job, restart counter is at 20.
Jan 22 23:50:13 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:50:13 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.518315838Z caller=main.go:126 msg="Starting Loki" version="(version=3.3.2, branch=release-3.3.x, revision=23b5fc2c)"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.51834897Z caller=main.go:127 msg="Loading configuration file" filename=/etc/loki/config.yml
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.52054567Z caller=server.go:351 msg="server listening on addresses" http=[::]:3100 grpc=[::]:9095
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.529963095Z caller=table_manager.go:136 index-store=tsdb-2024-05-30 msg="uploading tables"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.530071464Z caller=table_manager.go:299 index-store=tsdb-2024-05-30 msg="query readiness setup completed" duration=1.913µs distinct_users_len=0 distinct_users=
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.530097323Z caller=shipper.go:160 index-store=tsdb-2024-05-30 msg="starting index shipper in RW mode"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.530745576Z caller=head_manager.go:308 index-store=tsdb-2024-05-30 component=tsdb-head-manager msg="loaded wals by period" groups=0
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.530793666Z caller=manager.go:86 index-store=tsdb-2024-05-30 component=tsdb-manager msg="loaded leftover local indices" err=null successful=true buckets=0 indices=0 failures=0
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.530826888Z caller=head_manager.go:308 index-store=tsdb-2024-05-30 component=tsdb-head-manager msg="loaded wals by period" groups=0
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.534588135Z caller=worker.go:130 component=querier msg="Starting querier worker using query-scheduler and scheduler ring for addresses"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.535518436Z caller=mapper.go:47 msg="cleaning up mapped rules directory" path=/tmp/loki/rules-temp
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.53960702Z caller=module_service.go:82 msg=starting module=cache-generation-loader
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.539610292Z caller=module_service.go:82 msg=starting module=server
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.539702348Z caller=module_service.go:82 msg=starting module=memberlist-kv
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.53971402Z caller=module_service.go:82 msg=starting module=query-frontend-tripperware
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.539727455Z caller=module_service.go:82 msg=starting module=ring
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.539805621Z caller=ring.go:316 msg="ring doesn't exist in KV store yet"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.539856977Z caller=module_service.go:82 msg=starting module=analytics
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540001277Z caller=module_service.go:82 msg=starting module=query-scheduler-ring
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540030522Z caller=ring.go:316 msg="ring doesn't exist in KV store yet"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540056821Z caller=module_service.go:82 msg=starting module=store
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540071519Z caller=module_service.go:82 msg=starting module=ingester-querier
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540114409Z caller=basic_lifecycler.go:299 msg="instance not found in the ring" instance=uut-ct0 ring=scheduler
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540120971Z caller=basic_lifecycler_delegates.go:63 msg="not loading tokens from file, tokens file path is empty"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540203426Z caller=module_service.go:82 msg=starting module=ruler
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540212693Z caller=ruler.go:533 msg="ruler up and running"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540258118Z caller=module_service.go:82 msg=starting module=ingester
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540268487Z caller=ingester.go:552 component=ingester msg="recovering from checkpoint"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540291901Z caller=recovery.go:41 component=ingester msg="no checkpoint found, treating as no-op"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540320194Z caller=module_service.go:82 msg=starting module=distributor
Jan 22 23:50:13 uut-ct0 loki[5598]: level=error ts=2025-01-22T23:50:13.540342406Z caller=ratestore.go:109 msg="error getting ingester clients" err="empty ring"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540361982Z caller=module_service.go:82 msg=starting module=compactor
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540386027Z caller=ring.go:316 msg="ring doesn't exist in KV store yet"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540453363Z caller=ringmanager.go:186 msg="waiting until scheduler is JOINING in the ring"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540463292Z caller=ringmanager.go:190 msg="scheduler is JOINING in the ring"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540488789Z caller=ringmanager.go:199 msg="waiting until scheduler is ACTIVE in the ring"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.54051599Z caller=basic_lifecycler.go:299 component=distributor msg="instance not found in the ring" instance=uut-ct0 ring=distributor
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.54059616Z caller=ingester.go:568 component=ingester msg="recovered WAL checkpoint recovery finished" elapsed=328.835µs errors=false
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540602322Z caller=ingester.go:574 component=ingester msg="recovering from WAL"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540649896Z caller=basic_lifecycler.go:299 msg="instance not found in the ring" instance=uut-ct0 ring=compactor
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.54065736Z caller=basic_lifecycler_delegates.go:63 msg="not loading tokens from file, tokens file path is empty"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540922641Z caller=ingester.go:590 component=ingester msg="WAL segment recovery finished" elapsed=655.156µs errors=false
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540931187Z caller=ingester.go:538 component=ingester msg="closing recoverer"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.540937769Z caller=ingester.go:546 component=ingester msg="WAL recovery finished" time=668.901µs
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.541107171Z caller=compactor.go:414 msg="waiting until compactor is JOINING in the ring"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.541113523Z caller=compactor.go:418 msg="compactor is JOINING in the ring"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.54114395Z caller=compactor.go:428 msg="waiting until compactor is ACTIVE in the ring"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.541420137Z caller=wal.go:156 msg=started component=wal
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.54148316Z caller=lifecycler.go:675 component=ingester msg="not loading tokens from file, tokens file path is empty"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.541510802Z caller=lifecycler.go:702 component=ingester msg="instance not found in ring, adding with no tokens" ring=ingester
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.541546138Z caller=lifecycler.go:544 component=ingester msg="auto-joining cluster after timeout" ring=ingester
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.541618854Z caller=ingester.go:759 component=ingester msg="sleeping for initial delay before starting periodic flushing" delay=14.530020285s
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.709751013Z caller=ringmanager.go:203 msg="scheduler is ACTIVE in the ring"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.70981346Z caller=module_service.go:82 msg=starting module=query-scheduler
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.709905322Z caller=module_service.go:82 msg=starting module=query-frontend
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.709987986Z caller=module_service.go:82 msg=starting module=querier
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.716175837Z caller=compactor.go:432 msg="compactor is ACTIVE in the ring"
Jan 22 23:50:13 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:13.716215382Z caller=loki.go:542 msg="Loki started" startup_time=208.32523ms
Jan 22 23:50:16 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:16.7103491Z caller=scheduler.go:653 msg="this scheduler is in the ReplicationSet, will now accept requests."
Jan 22 23:50:16 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:16.710369116Z caller=worker.go:231 component=querier msg="adding connection" addr=127.0.0.1:9095
Jan 22 23:50:18 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:18.716569483Z caller=compactor.go:493 msg="this instance has been chosen to run the compactor, starting compactor"
Jan 22 23:50:18 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:18.716624897Z caller=compactor.go:522 msg="waiting 10m0s for ring to stay stable and previous compactions to finish before starting compactor"
Jan 22 23:50:23 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:23.710782172Z caller=frontend_scheduler_worker.go:106 msg="adding connection to scheduler" addr=127.0.0.1:9095
Jan 22 23:50:35 uut-ct0 systemd[1]: Stopping loki.service - Loki service...
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600014556Z caller=signals.go:62 msg="=== received SIGINT/SIGTERM ===\n*** exiting"
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600163399Z caller=manager.go:267 msg="stopping user managers"
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600177064Z caller=manager.go:281 msg="all user managers stopped"
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600194898Z caller=mapper.go:47 msg="cleaning up mapped rules directory" path=/tmp/loki/rules-temp
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600220135Z caller=module_service.go:120 msg="module stopped" module=ruler
Jan 22 23:50:35 uut-ct0 loki[5598]: level=error ts=2025-01-22T23:50:35.600317743Z caller=scheduler_processor.go:111 component=querier msg="error processing requests from scheduler" err="rpc error: code = Canceled desc = context canceled" addr=127.0.0.1:9095
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600348786Z caller=basic_lifecycler.go:240 component=distributor msg="ring lifecycler is shutting down" ring=distributor
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600442887Z caller=wal.go:144 msg=stopped component=wal
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600459849Z caller=lifecycler.go:594 component=ingester msg="lifecycler loop() exited gracefully" ring=ingester
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600475433Z caller=basic_lifecycler.go:405 component=distributor msg="unregistering instance from ring" ring=distributor
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600474236Z caller=lifecycler.go:977 component=ingester msg="changing instance state from" old_state=ACTIVE new_state=LEAVING ring=ingester
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600514957Z caller=basic_lifecycler.go:280 component=distributor msg="instance removed from the ring" ring=distributor
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600565401Z caller=lifecycler.go:1056 component=ingester msg="transfers are disabled"
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600573526Z caller=lifecycler.go:1073 component=ingester msg="lifecycler entering final sleep before shutdown" final_sleep=0s
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600619753Z caller=lifecycler.go:646 component=ingester msg="instance removed from the KV store" ring=ingester
Jan 22 23:50:35 uut-ct0 loki[5598]: level=error ts=2025-01-22T23:50:35.600649914Z caller=scheduler_processor.go:111 component=querier msg="error processing requests from scheduler" err="rpc error: code = Canceled desc = context canceled" addr=127.0.0.1:9095
Jan 22 23:50:35 uut-ct0 loki[5598]: level=error ts=2025-01-22T23:50:35.600701682Z caller=scheduler_processor.go:111 component=querier msg="error processing requests from scheduler" err="rpc error: code = Canceled desc = context canceled" addr=127.0.0.1:9095
Jan 22 23:50:35 uut-ct0 loki[5598]: level=error ts=2025-01-22T23:50:35.600715177Z caller=scheduler_processor.go:111 component=querier msg="error processing requests from scheduler" err="rpc error: code = Canceled desc = context canceled" addr=127.0.0.1:9095
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600725416Z caller=compactor.go:475 msg="compactor exiting"
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600739602Z caller=basic_lifecycler.go:240 msg="ring lifecycler is shutting down" ring=compactor
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600811527Z caller=basic_lifecycler.go:405 msg="unregistering instance from ring" ring=compactor
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600845581Z caller=basic_lifecycler.go:280 msg="instance removed from the ring" ring=compactor
Jan 22 23:50:35 uut-ct0 loki[5598]: level=error ts=2025-01-22T23:50:35.600860799Z caller=compactor.go:534 msg="failed to run compaction" err="context canceled"
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600869035Z caller=compactor.go:592 msg="compactor started"
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600949285Z caller=module_service.go:120 msg="module stopped" module=compactor
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.600964678Z caller=module_service.go:120 msg="module stopped" module=distributor
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.601090223Z caller=module_service.go:120 msg="module stopped" module=ingester
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.601595584Z caller=module_service.go:120 msg="module stopped" module=querier
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.601657651Z caller=module_service.go:120 msg="module stopped" module=ingester-querier
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.602456515Z caller=table_manager.go:77 index-store=tsdb-2024-05-30 msg="stopping table manager"
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.602471063Z caller=table_manager.go:136 index-store=tsdb-2024-05-30 msg="uploading tables"
Jan 22 23:50:35 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:35.60248534Z caller=module_service.go:120 msg="module stopped" module=store
Jan 22 23:50:36 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:36.100905384Z caller=module_service.go:120 msg="module stopped" module=query-frontend
Jan 22 23:50:36 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:36.100970766Z caller=module_service.go:120 msg="module stopped" module=query-frontend-tripperware
Jan 22 23:50:36 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:36.100967281Z caller=module_service.go:120 msg="module stopped" module=query-scheduler
Jan 22 23:50:36 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:36.100987558Z caller=basic_lifecycler.go:240 msg="ring lifecycler is shutting down" ring=scheduler
Jan 22 23:50:36 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:36.101064447Z caller=module_service.go:120 msg="module stopped" module=cache-generation-loader
Jan 22 23:50:36 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:36.101107332Z caller=basic_lifecycler.go:405 msg="unregistering instance from ring" ring=scheduler
Jan 22 23:50:36 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:36.101138961Z caller=basic_lifecycler.go:280 msg="instance removed from the ring" ring=scheduler
Jan 22 23:50:36 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:36.101180629Z caller=module_service.go:120 msg="module stopped" module=query-scheduler-ring
Jan 22 23:50:53 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:53.544108443Z caller=module_service.go:120 msg="module stopped" module=analytics
Jan 22 23:50:53 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:53.55228649Z caller=module_service.go:120 msg="module stopped" module=ring
Jan 22 23:50:53 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:53.552337212Z caller=module_service.go:120 msg="module stopped" module=memberlist-kv
Jan 22 23:50:53 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:53.552733368Z caller=modules.go:1917 msg="server stopped"
Jan 22 23:50:53 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:53.552744609Z caller=module_service.go:120 msg="module stopped" module=server
Jan 22 23:50:53 uut-ct0 loki[5598]: level=info ts=2025-01-22T23:50:53.552758484Z caller=loki.go:542 msg="Loki stopped" running_time=40.044868494s
Jan 22 23:50:53 uut-ct0 systemd[1]: loki.service: Deactivated successfully.
Jan 22 23:50:53 uut-ct0 systemd[1]: Stopped loki.service - Loki service.
Jan 22 23:50:53 uut-ct0 systemd[1]: Started loki.service - Loki service.
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.613365538Z caller=main.go:126 msg="Starting Loki" version="(version=3.3.2, branch=release-3.3.x, revision=23b5fc2c)"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.61339841Z caller=main.go:127 msg="Loading configuration file" filename=/etc/loki/config.yml
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.615745271Z caller=server.go:351 msg="server listening on addresses" http=[::]:3100 grpc=[::]:9095
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.620491423Z caller=table_manager.go:299 index-store=tsdb-2024-05-30 msg="query readiness setup completed" duration=1.333µs distinct_users_len=0 distinct_users=
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.620519706Z caller=shipper.go:160 index-store=tsdb-2024-05-30 msg="starting index shipper in RW mode"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.620785223Z caller=head_manager.go:308 index-store=tsdb-2024-05-30 component=tsdb-head-manager msg="loaded wals by period" groups=0
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.620823895Z caller=manager.go:86 index-store=tsdb-2024-05-30 component=tsdb-manager msg="loaded leftover local indices" err=null successful=true buckets=0 indices=0 failures=0
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.620851988Z caller=head_manager.go:308 index-store=tsdb-2024-05-30 component=tsdb-head-manager msg="loaded wals by period" groups=0
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.622760062Z caller=table_manager.go:136 index-store=tsdb-2024-05-30 msg="uploading tables"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.626205579Z caller=worker.go:130 component=querier msg="Starting querier worker using query-scheduler and scheduler ring for addresses"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.626650151Z caller=mapper.go:47 msg="cleaning up mapped rules directory" path=/tmp/loki/rules-temp
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.630850041Z caller=module_service.go:82 msg=starting module=cache-generation-loader
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.630864132Z caller=module_service.go:82 msg=starting module=server
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.63091822Z caller=module_service.go:82 msg=starting module=query-frontend-tripperware
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.630946962Z caller=module_service.go:82 msg=starting module=memberlist-kv
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.63096162Z caller=module_service.go:82 msg=starting module=ring
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.630993932Z caller=module_service.go:82 msg=starting module=query-scheduler-ring
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631005822Z caller=ring.go:316 msg="ring doesn't exist in KV store yet"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631100359Z caller=module_service.go:82 msg=starting module=store
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631106981Z caller=module_service.go:82 msg=starting module=analytics
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.63124985Z caller=ring.go:316 msg="ring doesn't exist in KV store yet"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631307679Z caller=module_service.go:82 msg=starting module=ingester-querier
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631395273Z caller=basic_lifecycler.go:299 msg="instance not found in the ring" instance=uut-ct0 ring=scheduler
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631405151Z caller=basic_lifecycler_delegates.go:63 msg="not loading tokens from file, tokens file path is empty"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631433237Z caller=module_service.go:82 msg=starting module=ruler
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631437119Z caller=module_service.go:82 msg=starting module=compactor
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.6314456Z caller=ruler.go:533 msg="ruler up and running"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631530614Z caller=module_service.go:82 msg=starting module=ingester
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631574241Z caller=module_service.go:82 msg=starting module=distributor
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631597049Z caller=ingester.go:552 component=ingester msg="recovering from checkpoint"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631624201Z caller=ring.go:316 msg="ring doesn't exist in KV store yet"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631632966Z caller=recovery.go:41 component=ingester msg="no checkpoint found, treating as no-op"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=error ts=2025-01-22T23:50:53.631641827Z caller=ratestore.go:109 msg="error getting ingester clients" err="empty ring"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631681146Z caller=basic_lifecycler.go:299 msg="instance not found in the ring" instance=uut-ct0 ring=compactor
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631685755Z caller=basic_lifecycler_delegates.go:63 msg="not loading tokens from file, tokens file path is empty"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631679725Z caller=ringmanager.go:186 msg="waiting until scheduler is JOINING in the ring"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631693711Z caller=ringmanager.go:190 msg="scheduler is JOINING in the ring"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631694817Z caller=basic_lifecycler.go:299 component=distributor msg="instance not found in the ring" instance=uut-ct0 ring=distributor
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631753643Z caller=ringmanager.go:199 msg="waiting until scheduler is ACTIVE in the ring"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631786325Z caller=compactor.go:414 msg="waiting until compactor is JOINING in the ring"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631789063Z caller=ingester.go:568 component=ingester msg="recovered WAL checkpoint recovery finished" elapsed=193.207µs errors=false
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.631798581Z caller=ingester.go:574 component=ingester msg="recovering from WAL"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.632142284Z caller=ingester.go:590 component=ingester msg="WAL segment recovery finished" elapsed=544.965µs errors=false
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.63215631Z caller=ingester.go:538 component=ingester msg="closing recoverer"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.632162632Z caller=ingester.go:546 component=ingester msg="WAL recovery finished" time=566.706µs
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.632526493Z caller=wal.go:156 msg=started component=wal
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.632593819Z caller=lifecycler.go:675 component=ingester msg="not loading tokens from file, tokens file path is empty"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.632652519Z caller=lifecycler.go:702 component=ingester msg="instance not found in ring, adding with no tokens" ring=ingester
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.632710988Z caller=lifecycler.go:544 component=ingester msg="auto-joining cluster after timeout" ring=ingester
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.632755506Z caller=ingester.go:759 component=ingester msg="sleeping for initial delay before starting periodic flushing" delay=13.308972585s
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.754109411Z caller=compactor.go:418 msg="compactor is JOINING in the ring"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.754246858Z caller=compactor.go:428 msg="waiting until compactor is ACTIVE in the ring"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.788240285Z caller=ringmanager.go:203 msg="scheduler is ACTIVE in the ring"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.78828774Z caller=module_service.go:82 msg=starting module=query-scheduler
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.788380858Z caller=module_service.go:82 msg=starting module=querier
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.7884534Z caller=module_service.go:82 msg=starting module=query-frontend
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.859578866Z caller=compactor.go:432 msg="compactor is ACTIVE in the ring"
Jan 22 23:50:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:53.859622818Z caller=loki.go:542 msg="Loki started" startup_time=259.749494ms
Jan 22 23:50:56 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:56.788512575Z caller=scheduler.go:653 msg="this scheduler is in the ReplicationSet, will now accept requests."
Jan 22 23:50:56 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:56.788530624Z caller=worker.go:231 component=querier msg="adding connection" addr=127.0.0.1:9095
Jan 22 23:50:58 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:58.860108859Z caller=compactor.go:493 msg="this instance has been chosen to run the compactor, starting compactor"
Jan 22 23:50:58 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:50:58.860177046Z caller=compactor.go:522 msg="waiting 10m0s for ring to stay stable and previous compactions to finish before starting compactor"
Jan 22 23:51:03 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:51:03.788655497Z caller=frontend_scheduler_worker.go:106 msg="adding connection to scheduler" addr=127.0.0.1:9095
Jan 22 23:51:23 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:51:23.633430042Z caller=recalculate_owned_streams.go:49 msg="starting recalculate owned streams job"
Jan 22 23:51:23 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:51:23.633458195Z caller=recalculate_owned_streams.go:63 msg="detected ring changes, re-evaluating streams ownership"
Jan 22 23:51:23 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:51:23.633483733Z caller=recalculate_owned_streams.go:52 msg="completed recalculate owned streams job"
Jan 22 23:51:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:51:53.622918338Z caller=table_manager.go:136 index-store=tsdb-2024-05-30 msg="uploading tables"
Jan 22 23:51:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:51:53.633713083Z caller=recalculate_owned_streams.go:49 msg="starting recalculate owned streams job"
Jan 22 23:51:53 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:51:53.633741356Z caller=recalculate_owned_streams.go:52 msg="completed recalculate owned streams job"
Jan 22 23:52:09 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:52:09.115921263Z caller=roundtrip.go:359 org_id=fake traceID=2e0a904678085556 msg="executing query" type=range query="{job=\"loki.source.journal.read\"}" start=2025-01-22T23:47:09.114355633Z end=2025-01-22T23:52:09.114355633Z start_delta=5m0.001563075s end_delta=1.563316ms length=5m0s step=1000 query_hash=1494052604
Jan 22 23:52:09 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:52:09.116831426Z caller=table_manager.go:194 index-store=tsdb-2024-05-30 msg="get or create table" found=false table=index_20110 wait_for_lock=621ns
Jan 22 23:52:09 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:52:09.116848448Z caller=table_manager.go:204 index-store=tsdb-2024-05-30 msg="downloading all files for table" table=index_20110
Jan 22 23:52:09 uut-ct0 loki[6475]: ts=2025-01-22T23:52:09.117088046Z caller=spanlogger.go:111 table-name=index_20110 user=fake caller=log.go:168 level=info msg="init index set" duration=96.02µs success=true
Jan 22 23:52:09 uut-ct0 loki[6475]: ts=2025-01-22T23:52:09.117101521Z caller=spanlogger.go:111 table-name=index_20110 user=fake caller=log.go:168 level=info msg="downloaded index set at query time" duration=96.02µs
Jan 22 23:52:09 uut-ct0 loki[6475]: ts=2025-01-22T23:52:09.117258926Z caller=spanlogger.go:111 table-name=index_20110 user-id=fake user=fake caller=log.go:168 level=info msg="init index set" duration=65.643µs success=true
Jan 22 23:52:09 uut-ct0 loki[6475]: ts=2025-01-22T23:52:09.117268263Z caller=spanlogger.go:111 table-name=index_20110 user-id=fake user=fake caller=log.go:168 level=info msg="downloaded index set at query time" duration=65.643µs
Jan 22 23:52:09 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:52:09.117516498Z caller=metrics.go:409 org_id=fake traceID=2e0a904678085556 latency=fast query_type=stats start=2025-01-22T23:46:39Z end=2025-01-22T23:52:09Z start_delta=5m30.117514454s end_delta=117.514654ms length=5m30s duration=773.989µs status=200 query="{job=\"loki.source.journal.read\"}" query_hash=1494052604 total_entries=1
Jan 22 23:52:09 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:52:09.11940916Z caller=engine.go:263 component=querier org_id=fake traceID=2e0a904678085556 msg="executing query" query="{job=\"loki.source.journal.read\"}" query_hash=1494052604 type=range length=5m0s step=1s
Jan 22 23:52:09 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:52:09.122405407Z caller=table_manager.go:194 index-store=tsdb-2024-05-30 msg="get or create table" found=true table=index_20110 wait_for_lock=1.203µs
Jan 22 23:52:09 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:52:09.122759703Z caller=metrics.go:237 component=querier org_id=fake traceID=2e0a904678085556 latency=fast query="{job=\"loki.source.journal.read\"}" query_hash=1494052604 query_type=limited range_type=range length=5m0s start_delta=5m0.008395093s end_delta=8.395223ms step=1s duration=3.304026ms status=200 limit=2 returned_lines=2 throughput=58MB total_bytes=191kB total_bytes_structured_metadata=14kB lines_per_second=543881 total_lines=1797 post_filter_lines=1797 total_entries=2 store_chunks_download_time=0s queue_time=65.333µs splits=0 shards=0 query_referenced_structured_metadata=false pipeline_wrapper_filtered_lines=0 chunk_refs_fetch_time=152.356µs cache_chunk_req=0 cache_chunk_hit=0 cache_chunk_bytes_stored=0 cache_chunk_bytes_fetched=0 cache_chunk_download_time=0s cache_index_req=0 cache_index_hit=0 cache_index_download_time=0s cache_stats_results_req=0 cache_stats_results_hit=0 cache_stats_results_download_time=0s cache_volume_results_req=0 cache_volume_results_hit=0 cache_volume_results_download_time=0s cache_result_req=0 cache_result_hit=0 cache_result_download_time=0s cache_result_query_length_served=0s cardinality_estimate=0 ingester_chunk_refs=0 ingester_chunk_downloaded=0 ingester_chunk_matches=19 ingester_requests=1 ingester_chunk_head_bytes=191kB ingester_chunk_compressed_bytes=0B ingester_chunk_decompressed_bytes=0B ingester_post_filter_lines=1797 congestion_control_latency=0s index_total_chunks=0 index_post_bloom_filter_chunks=0 index_bloom_filter_ratio=0.00 index_used_bloom_filters=false index_shard_resolver_duration=0s disable_pipeline_wrappers=false has_labelfilter_before_parser=false
Jan 22 23:52:09 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:52:09.124167814Z caller=metrics.go:237 component=frontend org_id=fake traceID=2e0a904678085556 latency=fast query="{job=\"loki.source.journal.read\"}" query_hash=1494052604 query_type=limited range_type=range length=5m0s start_delta=5m0.009803756s end_delta=9.803886ms step=1s duration=8.07995ms status=200 limit=2 returned_lines=0 throughput=24MB total_bytes=191kB total_bytes_structured_metadata=14kB lines_per_second=222402 total_lines=1797 post_filter_lines=1797 total_entries=2 store_chunks_download_time=0s queue_time=64.999µs splits=0 shards=1 query_referenced_structured_metadata=false pipeline_wrapper_filtered_lines=0 chunk_refs_fetch_time=152.356µs cache_chunk_req=0 cache_chunk_hit=0 cache_chunk_bytes_stored=0 cache_chunk_bytes_fetched=0 cache_chunk_download_time=0s cache_index_req=0 cache_index_hit=0 cache_index_download_time=0s cache_stats_results_req=0 cache_stats_results_hit=0 cache_stats_results_download_time=0s cache_volume_results_req=0 cache_volume_results_hit=0 cache_volume_results_download_time=0s cache_result_req=0 cache_result_hit=0 cache_result_download_time=0s cache_result_query_length_served=0s cardinality_estimate=0 ingester_chunk_refs=0 ingester_chunk_downloaded=0 ingester_chunk_matches=19 ingester_requests=1 ingester_chunk_head_bytes=191kB ingester_chunk_compressed_bytes=0B ingester_chunk_decompressed_bytes=0B ingester_post_filter_lines=1797 congestion_control_latency=0s index_total_chunks=0 index_post_bloom_filter_chunks=0 index_bloom_filter_ratio=0.00 index_used_bloom_filters=false index_shard_resolver_duration=0s disable_pipeline_wrappers=false has_labelfilter_before_parser=false
Jan 22 23:52:10 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:52:10.285112732Z caller=roundtrip.go:359 org_id=fake traceID=2b35e3a78bd64ff5 msg="executing query" type=range query="{job=~\"loki.*|systemd-journal|alloy.*\"}" start=2025-01-22T23:47:09Z end=2025-01-22T23:52:10.283819867Z start_delta=5m1.285108976s end_delta=1.289269ms length=5m1.283819867s step=1000 query_hash=678938088
Jan 22 23:52:10 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:52:10.285667951Z caller=table_manager.go:194 index-store=tsdb-2024-05-30 msg="get or create table" found=true table=index_20110 wait_for_lock=742ns
Jan 22 23:52:10 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:52:10.286012476Z caller=metrics.go:409 org_id=fake traceID=2b35e3a78bd64ff5 latency=fast query_type=stats start=2025-01-22T23:46:39Z end=2025-01-22T23:52:10Z start_delta=5m31.286009521s end_delta=286.009761ms length=5m31s duration=417.241µs status=200 query="{job=~\"loki.*|systemd-journal|alloy.*\"}" query_hash=678938088 total_entries=1
Jan 22 23:52:10 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:52:10.286727254Z caller=engine.go:263 component=querier org_id=fake traceID=2b35e3a78bd64ff5 msg="executing query" query="{job=~\"loki.*|systemd-journal|alloy.*\"}" query_hash=678938088 type=range length=5m1.283819867s step=1s
Jan 22 23:52:10 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:52:10.288814473Z caller=table_manager.go:194 index-store=tsdb-2024-05-30 msg="get or create table" found=true table=index_20110 wait_for_lock=631ns
Jan 22 23:52:10 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:52:10.289022383Z caller=metrics.go:237 component=querier org_id=fake traceID=2b35e3a78bd64ff5 latency=fast query="{job=~\"loki.*|systemd-journal|alloy.*\"}" query_hash=678938088 query_type=limited range_type=range length=5m1.283819867s start_delta=5m1.289013706s end_delta=5.19396ms step=1s duration=2.253832ms status=200 limit=10 returned_lines=10 throughput=88MB total_bytes=198kB total_bytes_structured_metadata=14kB lines_per_second=806182 total_lines=1817 post_filter_lines=1817 total_entries=10 store_chunks_download_time=0s queue_time=28.273µs splits=0 shards=0 query_referenced_structured_metadata=false pipeline_wrapper_filtered_lines=0 chunk_refs_fetch_time=72.536µs cache_chunk_req=0 cache_chunk_hit=0 cache_chunk_bytes_stored=0 cache_chunk_bytes_fetched=0 cache_chunk_download_time=0s cache_index_req=0 cache_index_hit=0 cache_index_download_time=0s cache_stats_results_req=0 cache_stats_results_hit=0 cache_stats_results_download_time=0s cache_volume_results_req=0 cache_volume_results_hit=0 cache_volume_results_download_time=0s cache_result_req=0 cache_result_hit=0 cache_result_download_time=0s cache_result_query_length_served=0s cardinality_estimate=0 ingester_chunk_refs=0 ingester_chunk_downloaded=0 ingester_chunk_matches=19 ingester_requests=1 ingester_chunk_head_bytes=198kB ingester_chunk_compressed_bytes=0B ingester_chunk_decompressed_bytes=0B ingester_post_filter_lines=1817 congestion_control_latency=0s index_total_chunks=0 index_post_bloom_filter_chunks=0 index_bloom_filter_ratio=0.00 index_used_bloom_filters=false index_shard_resolver_duration=0s disable_pipeline_wrappers=false has_labelfilter_before_parser=false
Jan 22 23:52:10 uut-ct0 loki[6475]: level=info ts=2025-01-22T23:52:10.289754399Z caller=metrics.go:237 component=frontend org_id=fake traceID=2b35e3a78bd64ff5 latency=fast query="{job=~\"loki.*|systemd-journal|alloy.*\"}" query_hash=678938088 query_type=limited range_type=range length=5m1.283819867s start_delta=5m1.289741835s end_delta=5.922108ms step=1s duration=4.357765ms status=200 limit=10 returned_lines=0 throughput=46MB total_bytes=198kB total_bytes_structured_metadata=14kB lines_per_second=416956 total_lines=1817 post_filter_lines=1817 total_entries=10 store_chunks_download_time=0s queue_time=28µs splits=0 shards=1 query_referenced_structured_metadata=false pipeline_wrapper_filtered_lines=0 chunk_refs_fetch_time=72.536µs cache_chunk_req=0 cache_chunk_hit=0 cache_chunk_bytes_stored=0 cache_chunk_bytes_fetched=0 cache_chunk_download_time=0s cache_index_req=0 cache_index_hit=0 cache_index_download_time=0s cache_stats_results_req=0 cache_stats_results_hit=0 cache_stats_results_download_time=0s cache_volume_results_req=0 cache_volume_results_hit=0 cache_volume_results_download_time=0s cache_result_req=0 cache_result_hit=0 cache_result_download_time=0s cache_result_query_length_served=0s cardinality_estimate=0 ingester_chunk_refs=0 ingester_chunk_downloaded=0 ingester_chunk_matches=19 ingester_requests=1 ingester_chunk_head_bytes=198kB ingester_chunk_compressed_bytes=0B ingester_chunk_decompressed_bytes=0B ingester_post_filter_lines=1817 congestion_control_latency=0s index_total_chunks=0 index_post_bloom_filter_chunks=0 index_bloom_filter_ratio=0.00 index_used_bloom_filters=false index_shard_resolver_duration=0s disable_pipeline_wrappers=false has_labelfilter_before_parser=false