Add metrics and limiter small refactoring #159

leborchuk · 2026-01-16T10:04:33Z

Instrument code and add prometheus metrics to the code. Use standard promauto registry.

Calculate the number of successful and errored requests and the number of processing items. Also measure latencies and sizes for requests:

	"READ":             true,
		"WRITE":            true,
		"LIMIT_READ":       true,
		"LIMIT_WRITE":      true,
		"S3_PUT":           true,
		"S3_GET":           true,
		"CAT":              true,
		"CATV2":            true,
		"PUT":              true,
		"PUTV2":            true,
		"PUTV3":            true,
		"DELETE":           true,
		"LIST":             true,
		"LISTV2":           true,
		"OBJECT META":      true,
		"COPY":             true,
		"COPYV2":           true,
		"GOOL":             true,
		"ERROR":            true,
		"UNTRASHIFY":       true,
		"COLLECT OBSOLETE": true,
		"DELETE OBSOLETE":  true,

It's not the whole list, but I think the essential methods. For measure latency, I delete total_time to data size, so measure processing time of 1 bite. Time measures in nanoseconds.

Default port for gather data is 2112, use curl localhost:2112/metrics

Replace write and limit order for write request. Before we write and then wait rate limit. It is ok for reading requests beacuse we do not know the reading data set size so cannot say how many bites we will consume. But for the writing the situation is reversed. So we should wait limiter if we could write N bites. And after that perform actual write. Because in a case of reverse situation, all processes write their data, it creates a big burst write flow, and then all processes will wait for the limiter. Then again write and so on - the load will be wavelike.

github-advanced-security

check-spelling found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

leborchuk · 2026-01-16T10:06:05Z

The examples of metrics for a single query

test=# select count(*) from public.ydt_100m;
   count
-----------
 100000000

# HELP client_connections The number of client connections to yproxy
# TYPE client_connections gauge
client_connections 0
# HELP external_connections_backup The number of external connections to S3 storage
# TYPE external_connections_backup gauge
external_connections_backup 0
# HELP external_connections_yezzey The number of external connections to S3 storage
# TYPE external_connections_yezzey gauge
external_connections_yezzey 0
# HELP go_gc_duration_seconds A summary of the wall-time pause (stop-the-world) duration in garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 9.0478e-05
go_gc_duration_seconds{quantile="0.25"} 0.000105701
go_gc_duration_seconds{quantile="0.5"} 0.00013717
go_gc_duration_seconds{quantile="0.75"} 0.000170231
go_gc_duration_seconds{quantile="1"} 0.000334833
go_gc_duration_seconds_sum 0.00110412
go_gc_duration_seconds_count 7
# HELP go_gc_gogc_percent Heap size target percentage configured by the user, otherwise 100. This value is set by the GOGC environment variable, and the runtime/debug.SetGCPercent function. Sourced from /gc/gogc:percent.
# TYPE go_gc_gogc_percent gauge
go_gc_gogc_percent 100
# HELP go_gc_gomemlimit_bytes Go runtime memory limit configured by the user, otherwise math.MaxInt64. This value is set by the GOMEMLIMIT environment variable, and the runtime/debug.SetMemoryLimit function. Sourced from /gc/gomemlimit:bytes.
# TYPE go_gc_gomemlimit_bytes gauge
go_gc_gomemlimit_bytes 2.5769803776e+10
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 15
# HELP go_info Information about the Go environment.
# TYPE go_info gauge
go_info{version="go1.24.10"} 1
# HELP go_memstats_alloc_bytes Number of bytes allocated in heap and currently in use. Equals to /memory/classes/heap/objects:bytes.
# TYPE go_memstats_alloc_bytes gauge
go_memstats_alloc_bytes 4.430048e+06
# HELP go_memstats_alloc_bytes_total Total number of bytes allocated in heap until now, even if released already. Equals to /gc/heap/allocs:bytes.
# TYPE go_memstats_alloc_bytes_total counter
go_memstats_alloc_bytes_total 1.4421808e+07
# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table. Equals to /memory/classes/profiling/buckets:bytes.
# TYPE go_memstats_buck_hash_sys_bytes gauge
go_memstats_buck_hash_sys_bytes 1.451385e+06
# HELP go_memstats_frees_total Total number of heap objects frees. Equals to /gc/heap/frees:objects + /gc/heap/tiny/allocs:objects.
# TYPE go_memstats_frees_total counter
go_memstats_frees_total 75601
# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata. Equals to /memory/classes/metadata/other:bytes.
# TYPE go_memstats_gc_sys_bytes gauge
go_memstats_gc_sys_bytes 3.165672e+06
# HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and currently in use, same as go_memstats_alloc_bytes. Equals to /memory/classes/heap/objects:bytes.
# TYPE go_memstats_heap_alloc_bytes gauge
go_memstats_heap_alloc_bytes 4.430048e+06
# HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used. Equals to /memory/classes/heap/released:bytes + /memory/classes/heap/free:bytes.
# TYPE go_memstats_heap_idle_bytes gauge
go_memstats_heap_idle_bytes 8.953856e+06
# HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use. Equals to /memory/classes/heap/objects:bytes + /memory/classes/heap/unused:bytes
# TYPE go_memstats_heap_inuse_bytes gauge
go_memstats_heap_inuse_bytes 6.807552e+06
# HELP go_memstats_heap_objects Number of currently allocated objects. Equals to /gc/heap/objects:objects.
# TYPE go_memstats_heap_objects gauge
go_memstats_heap_objects 22031
# HELP go_memstats_heap_released_bytes Number of heap bytes released to OS. Equals to /memory/classes/heap/released:bytes.
# TYPE go_memstats_heap_released_bytes gauge
go_memstats_heap_released_bytes 6.504448e+06
# HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system. Equals to /memory/classes/heap/objects:bytes + /memory/classes/heap/unused:bytes + /memory/classes/heap/released:bytes + /memory/classes/heap/free:bytes.
# TYPE go_memstats_heap_sys_bytes gauge
go_memstats_heap_sys_bytes 1.5761408e+07
# HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection.
# TYPE go_memstats_last_gc_time_seconds gauge
go_memstats_last_gc_time_seconds 1.7685566443842454e+09
# HELP go_memstats_mallocs_total Total number of heap objects allocated, both live and gc-ed. Semantically a counter version for go_memstats_heap_objects gauge. Equals to /gc/heap/allocs:objects + /gc/heap/tiny/allocs:objects.
# TYPE go_memstats_mallocs_total counter
go_memstats_mallocs_total 97632
# HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures. Equals to /memory/classes/metadata/mcache/inuse:bytes.
# TYPE go_memstats_mcache_inuse_bytes gauge
go_memstats_mcache_inuse_bytes 38656
# HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system. Equals to /memory/classes/metadata/mcache/inuse:bytes + /memory/classes/metadata/mcache/free:bytes.
# TYPE go_memstats_mcache_sys_bytes gauge
go_memstats_mcache_sys_bytes 47112
# HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures. Equals to /memory/classes/metadata/mspan/inuse:bytes.
# TYPE go_memstats_mspan_inuse_bytes gauge
go_memstats_mspan_inuse_bytes 225600
# HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system. Equals to /memory/classes/metadata/mspan/inuse:bytes + /memory/classes/metadata/mspan/free:bytes.
# TYPE go_memstats_mspan_sys_bytes gauge
go_memstats_mspan_sys_bytes 277440
# HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place. Equals to /gc/heap/goal:bytes.
# TYPE go_memstats_next_gc_bytes gauge
go_memstats_next_gc_bytes 7.28113e+06
# HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations. Equals to /memory/classes/other:bytes.
# TYPE go_memstats_other_sys_bytes gauge
go_memstats_other_sys_bytes 3.010015e+06
# HELP go_memstats_stack_inuse_bytes Number of bytes obtained from system for stack allocator in non-CGO environments. Equals to /memory/classes/heap/stacks:bytes.
# TYPE go_memstats_stack_inuse_bytes gauge
go_memstats_stack_inuse_bytes 1.015808e+06
# HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator. Equals to /memory/classes/heap/stacks:bytes + /memory/classes/os-stacks:bytes.
# TYPE go_memstats_stack_sys_bytes gauge
go_memstats_stack_sys_bytes 1.015808e+06
# HELP go_memstats_sys_bytes Number of bytes obtained from system. Equals to /memory/classes/total:byte.
# TYPE go_memstats_sys_bytes gauge
go_memstats_sys_bytes 2.472884e+07
# HELP go_sched_gomaxprocs_threads The current runtime.GOMAXPROCS setting, or the number of operating system threads that can execute user-level Go code simultaneously. Sourced from /sched/gomaxprocs:threads.
# TYPE go_sched_gomaxprocs_threads gauge
go_sched_gomaxprocs_threads 32
# HELP go_threads Number of OS threads created.
# TYPE go_threads gauge
go_threads 19
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 0.98
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 524288
# HELP process_network_receive_bytes_total Number of bytes received by the process over the network.
# TYPE process_network_receive_bytes_total counter
process_network_receive_bytes_total 1.553679757601e+12
# HELP process_network_transmit_bytes_total Number of bytes sent by the process over the network.
# TYPE process_network_transmit_bytes_total counter
process_network_transmit_bytes_total 1.520048543973e+12
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 13
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 2.9097984e+07
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.76855637857e+09
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 2.56823296e+09
# HELP process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.
# TYPE process_virtual_memory_max_bytes gauge
process_virtual_memory_max_bytes 1.8446744073709552e+19
# HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served.
# TYPE promhttp_metric_handler_requests_in_flight gauge
promhttp_metric_handler_requests_in_flight 1
# HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.
# TYPE promhttp_metric_handler_requests_total counter
promhttp_metric_handler_requests_total{code="200"} 2
promhttp_metric_handler_requests_total{code="500"} 0
promhttp_metric_handler_requests_total{code="503"} 0
# HELP read_req_errors_total The total number of errors during reads
# TYPE read_req_errors_total counter
read_req_errors_total 0
# HELP read_req_processed_total The total number of processed reads
# TYPE read_req_processed_total counter
read_req_processed_total 7183
# HELP request_latency Request latency in seconds
# TYPE request_latency histogram
request_latency_bucket{source="CATV2",le="0.001"} 0
request_latency_bucket{source="CATV2",le="0.01"} 0
request_latency_bucket{source="CATV2",le="0.05"} 0
request_latency_bucket{source="CATV2",le="0.1"} 0
request_latency_bucket{source="CATV2",le="0.5"} 0
request_latency_bucket{source="CATV2",le="1"} 0
request_latency_bucket{source="CATV2",le="5"} 0
request_latency_bucket{source="CATV2",le="10"} 0
request_latency_bucket{source="CATV2",le="50"} 4
request_latency_bucket{source="CATV2",le="100"} 4
request_latency_bucket{source="CATV2",le="500"} 4
request_latency_bucket{source="CATV2",le="1000"} 4
request_latency_bucket{source="CATV2",le="+Inf"} 4
request_latency_sum{source="CATV2"} 54.54297553986896
request_latency_count{source="CATV2"} 4
request_latency_bucket{source="READ",le="0.001"} 0
request_latency_bucket{source="READ",le="0.01"} 0
request_latency_bucket{source="READ",le="0.05"} 139
request_latency_bucket{source="READ",le="0.1"} 218
request_latency_bucket{source="READ",le="0.5"} 4537
request_latency_bucket{source="READ",le="1"} 6180
request_latency_bucket{source="READ",le="5"} 6651
request_latency_bucket{source="READ",le="10"} 6704
request_latency_bucket{source="READ",le="50"} 6821
request_latency_bucket{source="READ",le="100"} 6942
request_latency_bucket{source="READ",le="500"} 6978
request_latency_bucket{source="READ",le="1000"} 6984
request_latency_bucket{source="READ",le="+Inf"} 7183
request_latency_sum{source="READ"} 2.4379078546498725e+06
request_latency_count{source="READ"} 7183
request_latency_bucket{source="S3_GET",le="0.001"} 0
request_latency_bucket{source="S3_GET",le="0.01"} 0
request_latency_bucket{source="S3_GET",le="0.05"} 0
request_latency_bucket{source="S3_GET",le="0.1"} 0
request_latency_bucket{source="S3_GET",le="0.5"} 0
request_latency_bucket{source="S3_GET",le="1"} 0
request_latency_bucket{source="S3_GET",le="5"} 4
request_latency_bucket{source="S3_GET",le="10"} 4
request_latency_bucket{source="S3_GET",le="50"} 4
request_latency_bucket{source="S3_GET",le="100"} 4
request_latency_bucket{source="S3_GET",le="500"} 4
request_latency_bucket{source="S3_GET",le="1000"} 4
request_latency_bucket{source="S3_GET",le="+Inf"} 4
request_latency_sum{source="S3_GET"} 11.616734938160908
request_latency_count{source="S3_GET"} 4
# HELP request_size Request latency in seconds
# TYPE request_size histogram
request_size_bucket{source="CATV2",le="1"} 0
request_size_bucket{source="CATV2",le="128"} 0
request_size_bucket{source="CATV2",le="1024"} 0
request_size_bucket{source="CATV2",le="131072"} 0
request_size_bucket{source="CATV2",le="1.048576e+06"} 0
request_size_bucket{source="CATV2",le="2.097152e+06"} 0
request_size_bucket{source="CATV2",le="8.388608e+06"} 0
request_size_bucket{source="CATV2",le="1.6777216e+07"} 0
request_size_bucket{source="CATV2",le="1.34217728e+08"} 4
request_size_bucket{source="CATV2",le="1.073741824e+09"} 4
request_size_bucket{source="CATV2",le="+Inf"} 4
request_size_sum{source="CATV2"} 2.00261268e+08
request_size_count{source="CATV2"} 4
request_size_bucket{source="READ",le="1"} 412
request_size_bucket{source="READ",le="128"} 824
request_size_bucket{source="READ",le="1024"} 850
request_size_bucket{source="READ",le="131072"} 7183
request_size_bucket{source="READ",le="1.048576e+06"} 7183
request_size_bucket{source="READ",le="2.097152e+06"} 7183
request_size_bucket{source="READ",le="8.388608e+06"} 7183
request_size_bucket{source="READ",le="1.6777216e+07"} 7183
request_size_bucket{source="READ",le="1.34217728e+08"} 7183
request_size_bucket{source="READ",le="1.073741824e+09"} 7183
request_size_bucket{source="READ",le="+Inf"} 7183
request_size_sum{source="READ"} 2.00261356e+08
request_size_count{source="READ"} 7183
request_size_bucket{source="S3_GET",le="1"} 0
request_size_bucket{source="S3_GET",le="128"} 0
request_size_bucket{source="S3_GET",le="1024"} 0
request_size_bucket{source="S3_GET",le="131072"} 0
request_size_bucket{source="S3_GET",le="1.048576e+06"} 0
request_size_bucket{source="S3_GET",le="2.097152e+06"} 0
request_size_bucket{source="S3_GET",le="8.388608e+06"} 0
request_size_bucket{source="S3_GET",le="1.6777216e+07"} 0
request_size_bucket{source="S3_GET",le="1.34217728e+08"} 4
request_size_bucket{source="S3_GET",le="1.073741824e+09"} 4
request_size_bucket{source="S3_GET",le="+Inf"} 4
request_size_sum{source="S3_GET"} 2.00261356e+08
request_size_count{source="S3_GET"} 4
# HELP write_req_errors_total The total number of errors during reads
# TYPE write_req_errors_total counter
write_req_errors_total 0
# HELP write_req_processed_total The total number of processed writes
# TYPE write_req_processed_total counter
write_req_processed_total 0
[COMPUTE-PROD]root@rc1b-qfkbl5b84p1r8te6 ~ # curl localhost:2112/metrics
# HELP client_connections The number of client connections to yproxy
# TYPE client_connections gauge
client_connections 0
# HELP external_connections_backup The number of external connections to S3 storage
# TYPE external_connections_backup gauge
external_connections_backup 0
# HELP external_connections_yezzey The number of external connections to S3 storage
# TYPE external_connections_yezzey gauge
external_connections_yezzey 0
# HELP go_gc_duration_seconds A summary of the wall-time pause (stop-the-world) duration in garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 9.0478e-05
go_gc_duration_seconds{quantile="0.25"} 0.000105701
go_gc_duration_seconds{quantile="0.5"} 0.000130359
go_gc_duration_seconds{quantile="0.75"} 0.000147236
go_gc_duration_seconds{quantile="1"} 0.000334833
go_gc_duration_seconds_sum 0.001461997
go_gc_duration_seconds_count 10
# HELP go_gc_gogc_percent Heap size target percentage configured by the user, otherwise 100. This value is set by the GOGC environment variable, and the runtime/debug.SetGCPercent function. Sourced from /gc/gogc:percent.
# TYPE go_gc_gogc_percent gauge
go_gc_gogc_percent 100
# HELP go_gc_gomemlimit_bytes Go runtime memory limit configured by the user, otherwise math.MaxInt64. This value is set by the GOMEMLIMIT environment variable, and the runtime/debug.SetMemoryLimit function. Sourced from /gc/gomemlimit:bytes.
# TYPE go_gc_gomemlimit_bytes gauge
go_gc_gomemlimit_bytes 2.5769803776e+10
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 14
# HELP go_info Information about the Go environment.
# TYPE go_info gauge
go_info{version="go1.24.10"} 1
# HELP go_memstats_alloc_bytes Number of bytes allocated in heap and currently in use. Equals to /memory/classes/heap/objects:bytes.
# TYPE go_memstats_alloc_bytes gauge
go_memstats_alloc_bytes 3.966824e+06
# HELP go_memstats_alloc_bytes_total Total number of bytes allocated in heap until now, even if released already. Equals to /gc/heap/allocs:bytes.
# TYPE go_memstats_alloc_bytes_total counter
go_memstats_alloc_bytes_total 2.0728512e+07
# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table. Equals to /memory/classes/profiling/buckets:bytes.
# TYPE go_memstats_buck_hash_sys_bytes gauge
go_memstats_buck_hash_sys_bytes 1.452297e+06
# HELP go_memstats_frees_total Total number of heap objects frees. Equals to /gc/heap/frees:objects + /gc/heap/tiny/allocs:objects.
# TYPE go_memstats_frees_total counter
go_memstats_frees_total 116797
# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata. Equals to /memory/classes/metadata/other:bytes.
# TYPE go_memstats_gc_sys_bytes gauge
go_memstats_gc_sys_bytes 3.175912e+06
# HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and currently in use, same as go_memstats_alloc_bytes. Equals to /memory/classes/heap/objects:bytes.
# TYPE go_memstats_heap_alloc_bytes gauge
go_memstats_heap_alloc_bytes 3.966824e+06
# HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used. Equals to /memory/classes/heap/released:bytes + /memory/classes/heap/free:bytes.
# TYPE go_memstats_heap_idle_bytes gauge
go_memstats_heap_idle_bytes 9.412608e+06
# HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use. Equals to /memory/classes/heap/objects:bytes + /memory/classes/heap/unused:bytes
# TYPE go_memstats_heap_inuse_bytes gauge
go_memstats_heap_inuse_bytes 6.316032e+06
# HELP go_memstats_heap_objects Number of currently allocated objects. Equals to /gc/heap/objects:objects.
# TYPE go_memstats_heap_objects gauge
go_memstats_heap_objects 19398
# HELP go_memstats_heap_released_bytes Number of heap bytes released to OS. Equals to /memory/classes/heap/released:bytes.
# TYPE go_memstats_heap_released_bytes gauge
go_memstats_heap_released_bytes 6.18496e+06
# HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system. Equals to /memory/classes/heap/objects:bytes + /memory/classes/heap/unused:bytes + /memory/classes/heap/released:bytes + /memory/classes/heap/free:bytes.
# TYPE go_memstats_heap_sys_bytes gauge
go_memstats_heap_sys_bytes 1.572864e+07
# HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection.
# TYPE go_memstats_last_gc_time_seconds gauge
go_memstats_last_gc_time_seconds 1.7685567256385603e+09
# HELP go_memstats_mallocs_total Total number of heap objects allocated, both live and gc-ed. Semantically a counter version for go_memstats_heap_objects gauge. Equals to /gc/heap/allocs:objects + /gc/heap/tiny/allocs:objects.
# TYPE go_memstats_mallocs_total counter
go_memstats_mallocs_total 136195
# HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures. Equals to /memory/classes/metadata/mcache/inuse:bytes.
# TYPE go_memstats_mcache_inuse_bytes gauge
go_memstats_mcache_inuse_bytes 38656
# HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system. Equals to /memory/classes/metadata/mcache/inuse:bytes + /memory/classes/metadata/mcache/free:bytes.
# TYPE go_memstats_mcache_sys_bytes gauge
go_memstats_mcache_sys_bytes 47112
# HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures. Equals to /memory/classes/metadata/mspan/inuse:bytes.
# TYPE go_memstats_mspan_inuse_bytes gauge
go_memstats_mspan_inuse_bytes 221120
# HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system. Equals to /memory/classes/metadata/mspan/inuse:bytes + /memory/classes/metadata/mspan/free:bytes.
# TYPE go_memstats_mspan_sys_bytes gauge
go_memstats_mspan_sys_bytes 277440
# HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place. Equals to /gc/heap/goal:bytes.
# TYPE go_memstats_next_gc_bytes gauge
go_memstats_next_gc_bytes 7.339234e+06
# HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations. Equals to /memory/classes/other:bytes.
# TYPE go_memstats_other_sys_bytes gauge
go_memstats_other_sys_bytes 2.998863e+06
# HELP go_memstats_stack_inuse_bytes Number of bytes obtained from system for stack allocator in non-CGO environments. Equals to /memory/classes/heap/stacks:bytes.
# TYPE go_memstats_stack_inuse_bytes gauge
go_memstats_stack_inuse_bytes 1.048576e+06
# HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator. Equals to /memory/classes/heap/stacks:bytes + /memory/classes/os-stacks:bytes.
# TYPE go_memstats_stack_sys_bytes gauge
go_memstats_stack_sys_bytes 1.048576e+06
# HELP go_memstats_sys_bytes Number of bytes obtained from system. Equals to /memory/classes/total:byte.
# TYPE go_memstats_sys_bytes gauge
go_memstats_sys_bytes 2.472884e+07
# HELP go_sched_gomaxprocs_threads The current runtime.GOMAXPROCS setting, or the number of operating system threads that can execute user-level Go code simultaneously. Sourced from /sched/gomaxprocs:threads.
# TYPE go_sched_gomaxprocs_threads gauge
go_sched_gomaxprocs_threads 32
# HELP go_threads Number of OS threads created.
# TYPE go_threads gauge
go_threads 20
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 1.9
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 524288
# HELP process_network_receive_bytes_total Number of bytes received by the process over the network.
# TYPE process_network_receive_bytes_total counter
process_network_receive_bytes_total 1.553681055589e+12
# HELP process_network_transmit_bytes_total Number of bytes sent by the process over the network.
# TYPE process_network_transmit_bytes_total counter
process_network_transmit_bytes_total 1.520049112935e+12
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 13
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 2.9564928e+07
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.76855637857e+09
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 2.643734528e+09
# HELP process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.
# TYPE process_virtual_memory_max_bytes gauge
process_virtual_memory_max_bytes 1.8446744073709552e+19
# HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served.
# TYPE promhttp_metric_handler_requests_in_flight gauge
promhttp_metric_handler_requests_in_flight 1
# HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.
# TYPE promhttp_metric_handler_requests_total counter
promhttp_metric_handler_requests_total{code="200"} 3
promhttp_metric_handler_requests_total{code="500"} 0
promhttp_metric_handler_requests_total{code="503"} 0
# HELP read_req_errors_total The total number of errors during reads
# TYPE read_req_errors_total counter
read_req_errors_total 0
# HELP read_req_processed_total The total number of processed reads
# TYPE read_req_processed_total counter
read_req_processed_total 14362
# HELP request_latency Request latency in seconds
# TYPE request_latency histogram
request_latency_bucket{source="CATV2",le="0.001"} 0
request_latency_bucket{source="CATV2",le="0.01"} 0
request_latency_bucket{source="CATV2",le="0.05"} 0
request_latency_bucket{source="CATV2",le="0.1"} 0
request_latency_bucket{source="CATV2",le="0.5"} 0
request_latency_bucket{source="CATV2",le="1"} 0
request_latency_bucket{source="CATV2",le="5"} 0
request_latency_bucket{source="CATV2",le="10"} 0
request_latency_bucket{source="CATV2",le="50"} 8
request_latency_bucket{source="CATV2",le="100"} 8
request_latency_bucket{source="CATV2",le="500"} 8
request_latency_bucket{source="CATV2",le="1000"} 8
request_latency_bucket{source="CATV2",le="+Inf"} 8
request_latency_sum{source="CATV2"} 102.52228865142972
request_latency_count{source="CATV2"} 8
request_latency_bucket{source="READ",le="0.001"} 0
request_latency_bucket{source="READ",le="0.01"} 0
request_latency_bucket{source="READ",le="0.05"} 259
request_latency_bucket{source="READ",le="0.1"} 417
request_latency_bucket{source="READ",le="0.5"} 8791
request_latency_bucket{source="READ",le="1"} 12354
request_latency_bucket{source="READ",le="5"} 13293
request_latency_bucket{source="READ",le="10"} 13406
request_latency_bucket{source="READ",le="50"} 13623
request_latency_bucket{source="READ",le="100"} 13885
request_latency_bucket{source="READ",le="500"} 13953
request_latency_bucket{source="READ",le="1000"} 13967
request_latency_bucket{source="READ",le="+Inf"} 14362
request_latency_sum{source="READ"} 4.391004465408538e+06
request_latency_count{source="READ"} 14362
request_latency_bucket{source="S3_GET",le="0.001"} 0
request_latency_bucket{source="S3_GET",le="0.01"} 0
request_latency_bucket{source="S3_GET",le="0.05"} 0
request_latency_bucket{source="S3_GET",le="0.1"} 0
request_latency_bucket{source="S3_GET",le="0.5"} 0
request_latency_bucket{source="S3_GET",le="1"} 1
request_latency_bucket{source="S3_GET",le="5"} 8
request_latency_bucket{source="S3_GET",le="10"} 8
request_latency_bucket{source="S3_GET",le="50"} 8
request_latency_bucket{source="S3_GET",le="100"} 8
request_latency_bucket{source="S3_GET",le="500"} 8
request_latency_bucket{source="S3_GET",le="1000"} 8
request_latency_bucket{source="S3_GET",le="+Inf"} 8
request_latency_sum{source="S3_GET"} 17.627008362700515
request_latency_count{source="S3_GET"} 8
# HELP request_size Request latency in seconds
# TYPE request_size histogram
request_size_bucket{source="CATV2",le="1"} 0
request_size_bucket{source="CATV2",le="128"} 0
request_size_bucket{source="CATV2",le="1024"} 0
request_size_bucket{source="CATV2",le="131072"} 0
request_size_bucket{source="CATV2",le="1.048576e+06"} 0
request_size_bucket{source="CATV2",le="2.097152e+06"} 0
request_size_bucket{source="CATV2",le="8.388608e+06"} 0
request_size_bucket{source="CATV2",le="1.6777216e+07"} 0
request_size_bucket{source="CATV2",le="1.34217728e+08"} 8
request_size_bucket{source="CATV2",le="1.073741824e+09"} 8
request_size_bucket{source="CATV2",le="+Inf"} 8
request_size_sum{source="CATV2"} 4.00522536e+08
request_size_count{source="CATV2"} 8
request_size_bucket{source="READ",le="1"} 824
request_size_bucket{source="READ",le="128"} 1648
request_size_bucket{source="READ",le="1024"} 1691
request_size_bucket{source="READ",le="131072"} 14362
request_size_bucket{source="READ",le="1.048576e+06"} 14362
request_size_bucket{source="READ",le="2.097152e+06"} 14362
request_size_bucket{source="READ",le="8.388608e+06"} 14362
request_size_bucket{source="READ",le="1.6777216e+07"} 14362
request_size_bucket{source="READ",le="1.34217728e+08"} 14362
request_size_bucket{source="READ",le="1.073741824e+09"} 14362
request_size_bucket{source="READ",le="+Inf"} 14362
request_size_sum{source="READ"} 4.00522712e+08
request_size_count{source="READ"} 14362
request_size_bucket{source="S3_GET",le="1"} 0
request_size_bucket{source="S3_GET",le="128"} 0
request_size_bucket{source="S3_GET",le="1024"} 0
request_size_bucket{source="S3_GET",le="131072"} 0
request_size_bucket{source="S3_GET",le="1.048576e+06"} 0
request_size_bucket{source="S3_GET",le="2.097152e+06"} 0
request_size_bucket{source="S3_GET",le="8.388608e+06"} 0
request_size_bucket{source="S3_GET",le="1.6777216e+07"} 0
request_size_bucket{source="S3_GET",le="1.34217728e+08"} 8
request_size_bucket{source="S3_GET",le="1.073741824e+09"} 8
request_size_bucket{source="S3_GET",le="+Inf"} 8
request_size_sum{source="S3_GET"} 4.00522712e+08
request_size_count{source="S3_GET"} 8
# HELP write_req_errors_total The total number of errors during reads
# TYPE write_req_errors_total counter
write_req_errors_total 0
# HELP write_req_processed_total The total number of processed writes
# TYPE write_req_processed_total counter
write_req_processed_total 0

go.mod

pkg/clientpool/clientpool.go

pkg/core/core.go

pkg/metrics/metrics.go

pkg/storage/sessionpool_test.go

pkg/metrics/metrics.go

github-actions · 2026-01-16T14:01:58Z

@check-spelling-bot Report

🔴 Please review

See the 📂 files view, the 📜action log, or 📝 job summary for details.

Unrecognized words (11)

Burstable
COPYV
cpath
LISTV
mws
promauto
promhttp
PUTV
setport
TSTo
Vec

These words are not needed and should be removed

persisnt

To accept these unrecognized words as correct and remove the previously acknowledged and now absent words, you could run the following commands

... in a clone of the git@github.com:open-gpdb/yproxy.git repository
on the AddMetrics branch (ℹ️ how do I use this?):

curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/main/apply.pl' |
perl - 'https://github.com/open-gpdb/yproxy/actions/runs/21069101591/attempts/1' &&
git commit -m 'Update check-spelling metadata'

OR

To have the bot accept them for you, comment in the PR quoting the following line:
@check-spelling-bot apply updates.

Available 📚 dictionaries could cover words (expected and unrecognized) not in the 📘 dictionary

This includes both expected items (238) from .github/actions/spelling/expect.txt and unrecognized words (11)

Dictionary	Entries	Covers	Uniquely
cspell:golang/dict/go.txt	2099	14	4
cspell:node/dict/node.txt	891	14	2
cspell:php/dict/php.txt	1689	13	2
cspell:python/src/python/python-lib.txt	2417	12	1
cspell:dotnet/dict/dotnet.txt	405	6

Consider adding them (in .github/workflows/spelling.yaml) in jobs:/spelling: for uses: check-spelling/check-spelling@main in its with:

      with:
        extra_dictionaries: |
          cspell:golang/dict/go.txt
          cspell:node/dict/node.txt
          cspell:php/dict/php.txt
          cspell:python/src/python/python-lib.txt
          cspell:dotnet/dict/dotnet.txt

To stop checking additional dictionaries, add (in .github/workflows/spelling.yaml) for uses: check-spelling/check-spelling@main in its with:

check_extra_dictionaries: ""

Pattern suggestions ✂️ (9)

You could add these patterns to .github/actions/spelling/patterns.txt:

# Automatically suggested patterns

# hit-count: 13 file-count: 8
# https/http/file urls
(?:\b(?:https?|ftp|file)://)[-A-Za-z0-9+&@#/*%?=~_|!:,.;]+[-A-Za-z0-9+&@#/*%=~_|]

# hit-count: 4 file-count: 4
# uuencoded
[!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_]{40,}

# hit-count: 4 file-count: 2
# C network byte conversions
(?:\d|\bh)to(?!ken)(?=[a-z])|to(?=[adhiklpun]\()

# hit-count: 4 file-count: 1
# container images
image: [-\w./:@]+

# hit-count: 4 file-count: 1
# version suffix <word>v#
(?:(?<=[A-Z]{2})V|(?<=[a-z]{2}|[A-Z]{2})v)\d+(?:\b|(?=[a-zA-Z_]))

# hit-count: 1 file-count: 1
# scala imports
^import (?:[\w.]|\{\w*?(?:,\s*(?:\w*|\*))+\})+

# hit-count: 1 file-count: 1
# Debian changelog severity
[-\w]+ \(.*\) (?:\w+|baseline|unstable|experimental); urgency=(?:low|medium|high|emergency|critical)\b

# hit-count: 1 file-count: 1
# Compiler flags (Unix, Java/Scala)
# Use if you have things like `-Pdocker` and want to treat them as `docker`
(?:^|[\t ,>"'`=(])-(?:(?:J-|)[DPWXY]|[Llf])(?=[A-Z]{2,}|[A-Z][a-z]|[a-z]{2,})

# hit-count: 1 file-count: 1
# Compiler flags (Windows / PowerShell)
# This is a subset of the more general compiler flags pattern.
# It avoids matching `-Path` to prevent it from being treated as `ath`
(?:^|[\t ,"'`=(])-(?:[DPL](?=[A-Z]{2,})|[WXYlf](?=[A-Z]{2,}|[A-Z][a-z]|[a-z]{2,}))

Alternatively, if a pattern suggestion doesn't make sense for this project, add a #
to the beginning of the line in the candidates file with the pattern to stop suggesting it.

Notices ℹ️ (1)

See the 📂 files view, the 📜action log, or 📝 job summary for details.

ℹ️ Notices	Count
ℹ️ candidate-pattern	14

See ℹ️ Event descriptions for more information.

If the flagged items are 🤯 false positives

If items relate to a ...

binary file (or some other file you wouldn't want to check at all).

Please add a file path to the excludes.txt file matching the containing file.

File paths are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your files.

^ refers to the file's path from the root of the repository, so ^README\.md$ would exclude README.md (on whichever branch you're using).
well-formed pattern.

If you can write a pattern that would match it,
try adding it to the patterns.txt file.

Patterns are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your lines.

Note that patterns can't match multiline strings.

🚂 If you're seeing this message and your PR is from a branch that doesn't have check-spelling,
please merge to your PR's base branch to get the version configured for your repository.

leborchuk and others added 9 commits January 16, 2026 12:23

Add serving metric web server

9f94c05

change the sequence of wait/limit for write

b5bf4ea

Perform go mod tidy

b5bcc34

Add metrics to measure write/read requests

7a76287

Use storage name for various pools

ff5c30d

Store latencies info

521d9c5

Use unique name

96eb459

Do not create limiter if no options provided

afef63b

Revert back work with limits

43eae28

github-advanced-security bot found potential problems Jan 16, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

vovik0134 reviewed Jan 16, 2026

View reviewed changes

go.mod Show resolved Hide resolved

Fix spelling

72b2523

github-advanced-security bot found potential problems Jan 16, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

Fix spelling 2

8c9ce58

github-advanced-security bot found potential problems Jan 16, 2026

View reviewed changes

pkg/metrics/metrics.go Dismissed Show dismissed Hide dismissed

pkg/metrics/metrics.go Dismissed Show dismissed Hide dismissed

This comment has been minimized.

Sign in to view

reshke approved these changes Jan 16, 2026

View reviewed changes

Migrate to go 1.25

6fc853e

leborchuk merged commit 9fd6ce2 into master Jan 16, 2026
8 of 9 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add metrics and limiter small refactoring #159

Add metrics and limiter small refactoring #159

Uh oh!

leborchuk commented Jan 16, 2026

Uh oh!

github-advanced-security bot left a comment

Uh oh!

This comment has been minimized.

leborchuk commented Jan 16, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

Uh oh!

Uh oh!

This comment has been minimized.

github-actions bot commented Jan 16, 2026

See the 📂 files view, the 📜action log, or 📝 job summary for details.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Add metrics and limiter small refactoring #159

Add metrics and limiter small refactoring #159

Uh oh!

Conversation

leborchuk commented Jan 16, 2026

Uh oh!

github-advanced-security bot left a comment

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

leborchuk commented Jan 16, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

Uh oh!

Uh oh!

This comment has been minimized.

github-actions bot commented Jan 16, 2026

@check-spelling-bot Report

🔴 Please review

See the 📂 files view, the 📜action log, or 📝 job summary for details.

See the 📂 files view, the 📜action log, or 📝 job summary for details.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants