[2024-10-18 00:55:22.416841] Starting SPDK v24.09.1-pre git sha1 b18e1bd62 / DPDK 23.11.0 initialization... [2024-10-18 00:55:22.416989] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1050226 ] Set seed to 1729205722413310 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-10-18 00:55:22.520676] app.c: 917:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-10-18 00:55:22.562836] app.c: 610:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-10-18 00:55:22.562883] app.c: 611:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1050226' to capture a snapshot of events at runtime. [2024-10-18 00:55:22.562893] app.c: 616:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-10-18 00:55:22.562904] app.c: 617:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-10-18 00:55:22.562912] app.c: 618:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1050226 for offline analysis/debug. [2024-10-18 00:55:22.562997] reactor.c: 990:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Fri Oct 18 00:55:26 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/10/18-00:55:29 ... thread 0: (1000000,1000000) ops and (336131.1,336131.1) ops/second in (2.975030,2.975030) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/6 629.96 MB 25.2 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 348.0 2.53 2.30 12 0.211 0 0 0.0 0.0 L2 3/2 251.78 MB 0.0 0.4 0.2 0.2 0.4 0.2 0.1 2.0 249.9 249.8 1.51 1.46 2 0.756 383K 190 0.0 0.0 L3 9/0 741.25 MB 0.7 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.60 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1181/8 97.00 GB 0.0 0.4 0.2 0.2 1.2 1.0 0.8 1.4 93.4 311.3 4.05 3.76 14 0.289 383K 190 0.0 0.0 Int 0/0 0.00 KB 0.0 0.4 0.2 0.2 1.2 1.0 0.4 1.4 93.4 311.3 4.05 3.76 14 0.289 383K 190 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 0.0 0.0 249.9 249.8 1.51 1.46 2 0.756 383K 190 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 348.0 2.53 2.30 12 0.211 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.1 total, 3.0 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.23 GB write, 400.02 MB/s write, 0.37 GB read, 120.02 MB/s read, 4.0 seconds Interval compaction: 1.23 GB write, 423.22 MB/s write, 0.37 GB read, 126.98 MB/s read, 4.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 4, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 4, interval: 4 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 112632 Average: 1.9591 StdDev: 18.60 Min: 1 Median: 1.2815 Max: 3569 Percentiles: P50: 1.28 P75: 1.71 P99: 4.00 P99.9: 9.25 P99.99: 338.95 ------------------------------------------------------ [ 0, 1 ] 37602 33.385% 33.385% ####### ( 1, 2 ] 66479 59.023% 92.408% ############ ( 2, 3 ] 5631 4.999% 97.407% # ( 3, 4 ] 1794 1.593% 99.000% ( 4, 6 ] 821 0.729% 99.729% ( 6, 10 ] 239 0.212% 99.941% ( 10, 15 ] 26 0.023% 99.964% ( 15, 22 ] 13 0.012% 99.976% ( 22, 34 ] 2 0.002% 99.978% ( 34, 51 ] 1 0.001% 99.979% ( 51, 76 ] 1 0.001% 99.980% ( 76, 110 ] 1 0.001% 99.980% ( 110, 170 ] 7 0.006% 99.987% ( 170, 250 ] 3 0.003% 99.989% ( 250, 380 ] 4 0.004% 99.993% ( 380, 580 ] 2 0.002% 99.995% ( 580, 870 ] 3 0.003% 99.997% ( 870, 1300 ] 1 0.001% 99.998% ( 1900, 2900 ] 3 0.003% 100.001% ( 2900, 4400 ] 1 0.001% 100.002% ** Level 2 read latency histogram (micros): Count: 71309 Average: 2.0160 StdDev: 13.95 Min: 0 Median: 1.3210 Max: 1943 Percentiles: P50: 1.32 P75: 1.73 P99: 4.70 P99.9: 9.21 P99.99: 767.00 ------------------------------------------------------ [ 0, 1 ] 21551 30.222% 30.222% ###### ( 1, 2 ] 43944 61.625% 91.847% ############ ( 2, 3 ] 3474 4.872% 96.719% # ( 3, 4 ] 1374 1.927% 98.645% ( 4, 6 ] 725 1.017% 99.662% ( 6, 10 ] 213 0.299% 99.961% ( 10, 15 ] 8 0.011% 99.972% ( 15, 22 ] 3 0.004% 99.976% ( 22, 34 ] 2 0.003% 99.979% ( 250, 380 ] 5 0.007% 99.986% ( 580, 870 ] 6 0.008% 99.994% ( 870, 1300 ] 3 0.004% 99.999% ( 1300, 1900 ] 1 0.001% 100.000% ( 1900, 2900 ] 1 0.001% 100.001% ** Level 3 read latency histogram (micros): Count: 55 Average: 1472.4909 StdDev: 1123.09 Min: 67 Median: 1151.1538 Max: 4611 Percentiles: P50: 1151.15 P75: 2056.25 P99: 4611.00 P99.9: 4611.00 P99.99: 4611.00 ------------------------------------------------------ ( 51, 76 ] 1 1.818% 1.818% ( 170, 250 ] 3 5.455% 7.273% # ( 250, 380 ] 3 5.455% 12.727% # ( 380, 580 ] 5 9.091% 21.818% ## ( 580, 870 ] 7 12.727% 34.545% ### ( 870, 1300 ] 13 23.636% 58.182% ##### ( 1300, 1900 ] 8 14.545% 72.727% ### ( 1900, 2900 ] 8 14.545% 87.273% ### ( 2900, 4400 ] 5 9.091% 96.364% ## ( 4400, 6600 ] 2 3.636% 100.000% # ** Level 4 read latency histogram (micros): Count: 15 Average: 1690.9333 StdDev: 913.73 Min: 424 Median: 1600.0000 Max: 3305 Percentiles: P50: 1600.00 P75: 2316.67 P99: 3305.00 P99.9: 3305.00 P99.99: 3305.00 ------------------------------------------------------ ( 380, 580 ] 2 13.333% 13.333% ### ( 580, 870 ] 1 6.667% 20.000% # ( 870, 1300 ] 2 13.333% 33.333% ### ( 1300, 1900 ] 5 33.333% 66.667% ####### ( 1900, 2900 ] 3 20.000% 86.667% #### ( 2900, 4400 ] 2 13.333% 100.000% ### ** Level 5 read latency histogram (micros): Count: 20 Average: 547.0000 StdDev: 678.28 Min: 74 Median: 150.0000 Max: 2024 Percentiles: P50: 150.00 P75: 580.00 P99: 2024.00 P99.9: 2024.00 P99.99: 2024.00 ------------------------------------------------------ ( 51, 76 ] 2 10.000% 10.000% ## ( 76, 110 ] 4 20.000% 30.000% #### ( 110, 170 ] 6 30.000% 60.000% ###### ( 380, 580 ] 3 15.000% 75.000% ### ( 870, 1300 ] 2 10.000% 85.000% ## ( 1900, 2900 ] 3 15.000% 100.000% ### ** DB Stats ** Uptime(secs): 3.1 total, 3.0 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 312.51 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.429 H:M:S, 13.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 330.55 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.429 H:M:S, 14.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/18-00:55:32 ... thread 0: (1000000,2000000) ops and (328671.6,332359.5) ops/second in (3.042551,6.017581) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/9 1.23 GB 18.9 0.6 0.0 0.6 2.4 1.8 0.0 1.3 74.9 324.7 7.56 6.98 28 0.270 575K 591 0.0 0.0 L1 1/1 2.90 MB 0.0 0.4 0.4 0.0 0.4 0.4 0.2 1.0 238.5 238.1 1.58 1.54 1 1.585 383K 602 0.0 0.0 L2 2/0 54.70 MB 0.5 0.4 0.2 0.2 0.4 0.2 0.4 2.0 249.9 249.8 1.51 1.46 2 0.756 383K 190 0.0 0.0 L3 38/0 962.72 MB 1.0 0.8 0.6 0.2 0.8 0.6 0.1 1.4 199.4 199.2 3.99 3.51 3 1.331 808K 811 0.0 0.0 L4 148/0 9.57 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1248/10 97.98 GB 0.0 2.1 1.1 1.0 3.9 3.0 2.1 2.1 144.6 273.4 14.65 13.48 34 0.431 2151K 2194 0.0 0.0 Int 0/0 0.00 KB 0.0 2.1 1.1 1.0 3.9 3.0 1.7 2.1 144.6 273.4 14.65 13.48 34 0.431 2151K 2194 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.1 1.1 1.0 2.1 1.1 0.0 0.0 228.9 228.6 9.26 8.64 8 1.157 2151K 2194 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 350.3 5.40 4.85 26 0.208 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.2 total, 6.0 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.91 GB write, 647.10 MB/s write, 2.07 GB read, 342.24 MB/s read, 14.7 seconds Interval compaction: 3.91 GB write, 665.66 MB/s write, 2.07 GB read, 352.05 MB/s read, 14.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 8, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 8, interval: 8 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 415079 Average: 2.1537 StdDev: 10.87 Min: 1 Median: 1.4761 Max: 3569 Percentiles: P50: 1.48 P75: 1.86 P99: 5.47 P99.9: 9.67 P99.99: 17.47 ------------------------------------------------------ [ 0, 1 ] 78616 18.940% 18.940% #### ( 1, 2 ] 270817 65.245% 84.185% ############# ( 2, 3 ] 40913 9.857% 94.041% ## ( 3, 4 ] 14460 3.484% 97.525% # ( 4, 6 ] 8308 2.002% 99.527% ( 6, 10 ] 1692 0.408% 99.934% ( 10, 15 ] 218 0.053% 99.987% ( 15, 22 ] 41 0.010% 99.997% ( 22, 34 ] 12 0.003% 100.000% ( 34, 51 ] 7 0.002% 100.001% ( 51, 76 ] 4 0.001% 100.002% ( 76, 110 ] 1 0.000% 100.002% ( 110, 170 ] 12 0.003% 100.005% ( 170, 250 ] 10 0.002% 100.008% ( 250, 380 ] 11 0.003% 100.010% ( 380, 580 ] 5 0.001% 100.012% ( 580, 870 ] 9 0.002% 100.014% ( 870, 1300 ] 3 0.001% 100.014% ( 1300, 1900 ] 2 0.000% 100.015% ( 1900, 2900 ] 3 0.001% 100.016% ( 2900, 4400 ] 1 0.000% 100.016% ** Level 1 read latency histogram (micros): Count: 81658 Average: 2.4756 StdDev: 5.32 Min: 1 Median: 1.6815 Max: 1037 Percentiles: P50: 1.68 P75: 2.24 P99: 5.93 P99.9: 11.21 P99.99: 14.84 ------------------------------------------------------ [ 0, 1 ] 5182 6.346% 6.346% # ( 1, 2 ] 52305 64.054% 70.400% ############# ( 2, 3 ] 15333 18.777% 89.177% #### ( 3, 4 ] 4590 5.621% 94.798% # ( 4, 6 ] 3557 4.356% 99.154% # ( 6, 10 ] 585 0.716% 99.870% ( 10, 15 ] 101 0.124% 99.994% ( 15, 22 ] 8 0.010% 100.004% ( 22, 34 ] 3 0.004% 100.007% ( 110, 170 ] 1 0.001% 100.009% ( 170, 250 ] 2 0.002% 100.011% ( 250, 380 ] 2 0.002% 100.013% ( 380, 580 ] 1 0.001% 100.015% ( 580, 870 ] 1 0.001% 100.016% ( 870, 1300 ] 1 0.001% 100.017% ** Level 2 read latency histogram (micros): Count: 95439 Average: 2.0753 StdDev: 12.07 Min: 0 Median: 1.3917 Max: 1943 Percentiles: P50: 1.39 P75: 1.79 P99: 5.20 P99.9: 9.46 P99.99: 650.38 ------------------------------------------------------ [ 0, 1 ] 24043 25.192% 25.192% ##### ( 1, 2 ] 60450 63.339% 88.531% ############# ( 2, 3 ] 6673 6.992% 95.523% # ( 3, 4 ] 2457 2.574% 98.097% # ( 4, 6 ] 1433 1.501% 99.599% ( 6, 10 ] 332 0.348% 99.947% ( 10, 15 ] 25 0.026% 99.973% ( 15, 22 ] 8 0.008% 99.981% ( 22, 34 ] 2 0.002% 99.983% ( 250, 380 ] 5 0.005% 99.988% ( 580, 870 ] 6 0.006% 99.995% ( 870, 1300 ] 3 0.003% 99.998% ( 1300, 1900 ] 1 0.001% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 57384 Average: 3.9290 StdDev: 57.48 Min: 0 Median: 1.6828 Max: 4611 Percentiles: P50: 1.68 P75: 2.26 P99: 6.37 P99.9: 164.24 P99.99: 2182.70 ------------------------------------------------------ [ 0, 1 ] 3878 6.758% 6.758% # ( 1, 2 ] 36343 63.333% 70.091% ############# ( 2, 3 ] 10840 18.890% 88.981% #### ( 3, 4 ] 3166 5.517% 94.498% # ( 4, 6 ] 2545 4.435% 98.934% # ( 6, 10 ] 416 0.725% 99.658% ( 10, 15 ] 94 0.164% 99.822% ( 15, 22 ] 7 0.012% 99.834% ( 22, 34 ] 28 0.049% 99.883% ( 34, 51 ] 4 0.007% 99.890% ( 51, 76 ] 2 0.003% 99.894% ( 110, 170 ] 4 0.007% 99.901% ( 170, 250 ] 7 0.012% 99.913% ( 250, 380 ] 6 0.010% 99.923% ( 380, 580 ] 7 0.012% 99.936% ( 580, 870 ] 8 0.014% 99.949% ( 870, 1300 ] 13 0.023% 99.972% ( 1300, 1900 ] 8 0.014% 99.986% ( 1900, 2900 ] 8 0.014% 100.000% ( 2900, 4400 ] 5 0.009% 100.009% ( 4400, 6600 ] 2 0.003% 100.012% ** Level 4 read latency histogram (micros): Count: 15 Average: 1690.9333 StdDev: 913.73 Min: 424 Median: 1600.0000 Max: 3305 Percentiles: P50: 1600.00 P75: 2316.67 P99: 3305.00 P99.9: 3305.00 P99.99: 3305.00 ------------------------------------------------------ ( 380, 580 ] 2 13.333% 13.333% ### ( 580, 870 ] 1 6.667% 20.000% # ( 870, 1300 ] 2 13.333% 33.333% ### ( 1300, 1900 ] 5 33.333% 66.667% ####### ( 1900, 2900 ] 3 20.000% 86.667% #### ( 2900, 4400 ] 2 13.333% 100.000% ### ** Level 5 read latency histogram (micros): Count: 40 Average: 647.9500 StdDev: 750.62 Min: 74 Median: 250.0000 Max: 2612 Percentiles: P50: 250.00 P75: 1042.00 P99: 2612.00 P99.9: 2612.00 P99.99: 2612.00 ------------------------------------------------------ ( 51, 76 ] 2 5.000% 5.000% # ( 76, 110 ] 7 17.500% 22.500% #### ( 110, 170 ] 8 20.000% 42.500% #### ( 170, 250 ] 3 7.500% 50.000% ## ( 250, 380 ] 3 7.500% 57.500% ## ( 380, 580 ] 4 10.000% 67.500% ## ( 580, 870 ] 1 2.500% 70.000% # ( 870, 1300 ] 5 12.500% 82.500% ### ( 1900, 2900 ] 7 17.500% 100.000% #### ** DB Stats ** Uptime(secs): 6.2 total, 3.0 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 317.89 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.829 H:M:S, 13.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 323.46 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.400 H:M:S, 13.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/18-00:55:35 ... thread 0: (1000000,3000000) ops and (330966.4,331893.8) ops/second in (3.021455,9.039036) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/4 1.35 GB 113.3 1.4 0.0 1.4 4.1 2.8 0.0 1.5 104.2 317.1 13.31 12.39 44 0.302 1407K 1432 0.0 0.0 L1 4/0 303.55 MB 30.4 1.2 1.2 0.0 1.2 1.2 0.2 1.0 235.4 234.8 5.09 4.97 2 2.547 1217K 3049 0.0 0.0 L2 3/1 183.47 MB 0.5 0.4 0.2 0.2 0.4 0.2 0.9 2.0 249.9 249.8 1.51 1.46 2 0.756 383K 190 0.0 0.0 L3 20/5 824.27 MB 0.8 0.8 0.6 0.2 0.8 0.6 0.5 1.4 199.4 199.2 3.99 3.51 3 1.331 808K 811 0.0 0.0 L4 161/0 9.40 GB 1.0 0.0 0.0 0.0 0.0 0.0 1.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1056/0 86.86 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1254/10 98.90 GB 0.0 3.7 1.9 1.8 6.4 4.7 4.2 2.3 157.3 275.6 23.90 22.32 51 0.469 3817K 5482 0.0 0.0 Int 0/0 0.00 KB 0.0 3.7 1.9 1.8 6.4 4.7 3.8 2.3 157.3 275.6 23.90 22.32 51 0.469 3817K 5482 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.7 1.9 1.8 3.7 1.9 0.0 0.0 237.6 237.3 15.82 15.06 12 1.318 3817K 5482 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 350.6 8.09 7.26 39 0.207 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.2 total, 9.0 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.43 GB write, 715.10 MB/s write, 3.67 GB read, 408.03 MB/s read, 23.9 seconds Interval compaction: 6.43 GB write, 728.75 MB/s write, 3.67 GB read, 415.82 MB/s read, 23.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 14, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 14, interval: 14 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 765820 Average: 2.1580 StdDev: 10.28 Min: 0 Median: 1.4897 Max: 3569 Percentiles: P50: 1.49 P75: 1.86 P99: 5.40 P99.9: 9.57 P99.99: 14.53 ------------------------------------------------------ [ 0, 1 ] 132653 17.322% 17.322% ### ( 1, 2 ] 511050 66.732% 84.054% ############# ( 2, 3 ] 78303 10.225% 94.279% ## ( 3, 4 ] 26001 3.395% 97.674% # ( 4, 6 ] 14526 1.897% 99.571% ( 6, 10 ] 2822 0.368% 99.939% ( 10, 15 ] 429 0.056% 99.995% ( 15, 22 ] 59 0.008% 100.003% ( 22, 34 ] 18 0.002% 100.005% ( 34, 51 ] 8 0.001% 100.006% ( 51, 76 ] 5 0.001% 100.007% ( 76, 110 ] 1 0.000% 100.007% ( 110, 170 ] 18 0.002% 100.010% ( 170, 250 ] 14 0.002% 100.011% ( 250, 380 ] 19 0.002% 100.014% ( 380, 580 ] 8 0.001% 100.015% ( 580, 870 ] 12 0.002% 100.016% ( 870, 1300 ] 9 0.001% 100.018% ( 1300, 1900 ] 3 0.000% 100.018% ( 1900, 2900 ] 4 0.001% 100.019% ( 2900, 4400 ] 2 0.000% 100.019% ** Level 1 read latency histogram (micros): Count: 112948 Average: 2.5484 StdDev: 9.05 Min: 1 Median: 1.6873 Max: 1523 Percentiles: P50: 1.69 P75: 2.26 P99: 5.95 P99.9: 11.97 P99.99: 28.35 ------------------------------------------------------ [ 0, 1 ] 6979 6.179% 6.179% # ( 1, 2 ] 72015 63.759% 69.938% ############# ( 2, 3 ] 21635 19.155% 89.093% #### ( 3, 4 ] 6198 5.487% 94.581% # ( 4, 6 ] 5125 4.537% 99.118% # ( 6, 10 ] 828 0.733% 99.851% ( 10, 15 ] 140 0.124% 99.975% ( 15, 22 ] 13 0.012% 99.987% ( 22, 34 ] 7 0.006% 99.993% ( 110, 170 ] 1 0.001% 99.994% ( 170, 250 ] 4 0.004% 99.997% ( 250, 380 ] 7 0.006% 100.004% ( 380, 580 ] 2 0.002% 100.005% ( 580, 870 ] 3 0.003% 100.008% ( 870, 1300 ] 4 0.004% 100.012% ( 1300, 1900 ] 1 0.001% 100.012% ** Level 2 read latency histogram (micros): Count: 95439 Average: 2.0753 StdDev: 12.07 Min: 0 Median: 1.3917 Max: 1943 Percentiles: P50: 1.39 P75: 1.79 P99: 5.20 P99.9: 9.46 P99.99: 650.38 ------------------------------------------------------ [ 0, 1 ] 24043 25.192% 25.192% ##### ( 1, 2 ] 60450 63.339% 88.531% ############# ( 2, 3 ] 6673 6.992% 95.523% # ( 3, 4 ] 2457 2.574% 98.097% # ( 4, 6 ] 1433 1.501% 99.599% ( 6, 10 ] 332 0.348% 99.947% ( 10, 15 ] 25 0.026% 99.973% ( 15, 22 ] 8 0.008% 99.981% ( 22, 34 ] 2 0.002% 99.983% ( 250, 380 ] 5 0.005% 99.988% ( 580, 870 ] 6 0.006% 99.995% ( 870, 1300 ] 3 0.003% 99.998% ( 1300, 1900 ] 1 0.001% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 67731 Average: 3.6525 StdDev: 52.92 Min: 0 Median: 1.6558 Max: 4611 Percentiles: P50: 1.66 P75: 2.17 P99: 5.98 P99.9: 33.69 P99.99: 2053.35 ------------------------------------------------------ [ 0, 1 ] 5397 7.968% 7.968% ## ( 1, 2 ] 43413 64.096% 72.064% ############# ( 2, 3 ] 12015 17.739% 89.804% #### ( 3, 4 ] 3532 5.215% 95.019% # ( 4, 6 ] 2721 4.017% 99.036% # ( 6, 10 ] 451 0.666% 99.702% ( 10, 15 ] 100 0.148% 99.849% ( 15, 22 ] 8 0.012% 99.861% ( 22, 34 ] 28 0.041% 99.903% ( 34, 51 ] 4 0.006% 99.908% ( 51, 76 ] 2 0.003% 99.911% ( 110, 170 ] 4 0.006% 99.917% ( 170, 250 ] 7 0.010% 99.928% ( 250, 380 ] 6 0.009% 99.937% ( 380, 580 ] 7 0.010% 99.947% ( 580, 870 ] 8 0.012% 99.959% ( 870, 1300 ] 13 0.019% 99.978% ( 1300, 1900 ] 8 0.012% 99.990% ( 1900, 2900 ] 8 0.012% 100.001% ( 2900, 4400 ] 5 0.007% 100.009% ( 4400, 6600 ] 2 0.003% 100.012% ** Level 4 read latency histogram (micros): Count: 15 Average: 1690.9333 StdDev: 913.73 Min: 424 Median: 1600.0000 Max: 3305 Percentiles: P50: 1600.00 P75: 2316.67 P99: 3305.00 P99.9: 3305.00 P99.99: 3305.00 ------------------------------------------------------ ( 380, 580 ] 2 13.333% 13.333% ### ( 580, 870 ] 1 6.667% 20.000% # ( 870, 1300 ] 2 13.333% 33.333% ### ( 1300, 1900 ] 5 33.333% 66.667% ####### ( 1900, 2900 ] 3 20.000% 86.667% #### ( 2900, 4400 ] 2 13.333% 100.000% ### ** Level 5 read latency histogram (micros): Count: 80 Average: 676.6250 StdDev: 887.27 Min: 74 Median: 210.0000 Max: 3886 Percentiles: P50: 210.00 P75: 923.75 P99: 3886.00 P99.9: 3886.00 P99.99: 3886.00 ------------------------------------------------------ ( 51, 76 ] 2 2.500% 2.500% # ( 76, 110 ] 18 22.500% 25.000% ##### ( 110, 170 ] 18 22.500% 47.500% ##### ( 170, 250 ] 4 5.000% 52.500% # ( 250, 380 ] 5 6.250% 58.750% # ( 380, 580 ] 7 8.750% 67.500% ## ( 580, 870 ] 5 6.250% 73.750% # ( 870, 1300 ] 8 10.000% 83.750% ## ( 1300, 1900 ] 1 1.250% 85.000% ( 1900, 2900 ] 8 10.000% 95.000% ## ( 2900, 4400 ] 4 5.000% 100.000% # ** DB Stats ** Uptime(secs): 9.2 total, 3.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 320.47 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.240 H:M:S, 13.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 325.75 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.411 H:M:S, 13.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/18-00:55:38 ... thread 0: (1000000,4000000) ops and (332314.7,331998.9) ops/second in (3.009196,12.048232) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/12 2.34 GB 62.9 2.2 0.0 2.2 5.9 3.8 0.0 1.6 114.3 313.3 19.29 18.02 61 0.316 2238K 2327 0.0 0.0 L1 4/4 303.55 MB 0.0 1.2 1.2 0.0 1.2 1.2 0.2 1.0 235.4 234.8 5.09 4.97 2 2.547 1217K 3049 0.0 0.0 L2 2/0 54.70 MB 0.5 0.4 0.2 0.2 0.4 0.2 0.9 2.0 249.9 249.8 1.51 1.46 2 0.756 383K 190 0.0 0.0 L3 17/0 952.63 MB 1.0 1.0 0.7 0.3 1.0 0.7 0.5 1.4 204.2 203.9 4.77 4.26 4 1.193 989K 1217 0.0 0.0 L4 161/0 9.40 GB 1.0 0.0 0.0 0.0 0.0 0.0 1.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1056/0 86.86 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1255/16 99.88 GB 0.0 4.6 2.0 2.6 8.4 5.8 4.2 2.2 155.1 280.1 30.67 28.71 69 0.445 4829K 6783 0.0 0.0 Int 0/0 0.00 KB 0.0 4.6 2.0 2.6 8.4 5.8 3.8 2.2 155.1 280.1 30.67 28.71 69 0.445 4829K 6783 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.6 2.0 2.6 4.6 2.0 0.0 0.0 240.8 240.4 19.75 18.91 16 1.235 4829K 6783 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 352.0 10.92 9.80 53 0.206 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.2 total, 12.0 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.39 GB write, 702.98 MB/s write, 4.64 GB read, 389.15 MB/s read, 30.7 seconds Interval compaction: 8.39 GB write, 713.05 MB/s write, 4.64 GB read, 394.73 MB/s read, 30.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 17, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 17, interval: 17 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1106939 Average: 2.1564 StdDev: 11.94 Min: 0 Median: 1.4880 Max: 6664 Percentiles: P50: 1.49 P75: 1.86 P99: 5.39 P99.9: 9.60 P99.99: 14.03 ------------------------------------------------------ [ 0, 1 ] 191726 17.320% 17.320% ### ( 1, 2 ] 741269 66.966% 84.286% ############# ( 2, 3 ] 112091 10.126% 94.412% ## ( 3, 4 ] 36760 3.321% 97.733% # ( 4, 6 ] 20188 1.824% 99.557% ( 6, 10 ] 4219 0.381% 99.938% ( 10, 15 ] 716 0.065% 100.003% ( 15, 22 ] 84 0.008% 100.010% ( 22, 34 ] 23 0.002% 100.012% ( 34, 51 ] 9 0.001% 100.013% ( 51, 76 ] 8 0.001% 100.014% ( 76, 110 ] 1 0.000% 100.014% ( 110, 170 ] 24 0.002% 100.016% ( 170, 250 ] 21 0.002% 100.018% ( 250, 380 ] 24 0.002% 100.020% ( 380, 580 ] 15 0.001% 100.022% ( 580, 870 ] 15 0.001% 100.023% ( 870, 1300 ] 12 0.001% 100.024% ( 1300, 1900 ] 3 0.000% 100.024% ( 1900, 2900 ] 4 0.000% 100.025% ( 2900, 4400 ] 4 0.000% 100.025% ( 6600, 9900 ] 1 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 115131 Average: 2.5406 StdDev: 8.96 Min: 1 Median: 1.6843 Max: 1523 Percentiles: P50: 1.68 P75: 2.25 P99: 5.94 P99.9: 11.89 P99.99: 27.98 ------------------------------------------------------ [ 0, 1 ] 7234 6.283% 6.283% # ( 1, 2 ] 73554 63.887% 70.171% ############# ( 2, 3 ] 21912 19.032% 89.203% #### ( 3, 4 ] 6264 5.441% 94.643% # ( 4, 6 ] 5169 4.490% 99.133% # ( 6, 10 ] 830 0.721% 99.854% ( 10, 15 ] 140 0.122% 99.976% ( 15, 22 ] 13 0.011% 99.987% ( 22, 34 ] 7 0.006% 99.993% ( 110, 170 ] 1 0.001% 99.994% ( 170, 250 ] 4 0.003% 99.997% ( 250, 380 ] 7 0.006% 100.003% ( 380, 580 ] 2 0.002% 100.005% ( 580, 870 ] 3 0.003% 100.008% ( 870, 1300 ] 4 0.003% 100.011% ( 1300, 1900 ] 1 0.001% 100.012% ** Level 2 read latency histogram (micros): Count: 95439 Average: 2.0753 StdDev: 12.07 Min: 0 Median: 1.3917 Max: 1943 Percentiles: P50: 1.39 P75: 1.79 P99: 5.20 P99.9: 9.46 P99.99: 650.38 ------------------------------------------------------ [ 0, 1 ] 24043 25.192% 25.192% ##### ( 1, 2 ] 60450 63.339% 88.531% ############# ( 2, 3 ] 6673 6.992% 95.523% # ( 3, 4 ] 2457 2.574% 98.097% # ( 4, 6 ] 1433 1.501% 99.599% ( 6, 10 ] 332 0.348% 99.947% ( 10, 15 ] 25 0.026% 99.973% ( 15, 22 ] 8 0.008% 99.981% ( 22, 34 ] 2 0.002% 99.983% ( 250, 380 ] 5 0.005% 99.988% ( 580, 870 ] 6 0.006% 99.995% ( 870, 1300 ] 3 0.003% 99.998% ( 1300, 1900 ] 1 0.001% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 69927 Average: 3.6193 StdDev: 52.14 Min: 0 Median: 1.6497 Max: 4611 Percentiles: P50: 1.65 P75: 2.14 P99: 5.97 P99.9: 38.56 P99.99: 2025.91 ------------------------------------------------------ [ 0, 1 ] 5753 8.227% 8.227% ## ( 1, 2 ] 44959 64.294% 72.521% ############# ( 2, 3 ] 12198 17.444% 89.965% ### ( 3, 4 ] 3608 5.160% 95.125% # ( 4, 6 ] 2746 3.927% 99.052% # ( 6, 10 ] 454 0.649% 99.701% ( 10, 15 ] 102 0.146% 99.847% ( 15, 22 ] 8 0.011% 99.858% ( 22, 34 ] 28 0.040% 99.898% ( 34, 51 ] 4 0.006% 99.904% ( 51, 76 ] 2 0.003% 99.907% ( 110, 170 ] 5 0.007% 99.914% ( 170, 250 ] 8 0.011% 99.926% ( 250, 380 ] 7 0.010% 99.936% ( 380, 580 ] 8 0.011% 99.947% ( 580, 870 ] 8 0.011% 99.959% ( 870, 1300 ] 13 0.019% 99.977% ( 1300, 1900 ] 8 0.011% 99.989% ( 1900, 2900 ] 8 0.011% 100.000% ( 2900, 4400 ] 5 0.007% 100.007% ( 4400, 6600 ] 2 0.003% 100.010% ** Level 4 read latency histogram (micros): Count: 15 Average: 1690.9333 StdDev: 913.73 Min: 424 Median: 1600.0000 Max: 3305 Percentiles: P50: 1600.00 P75: 2316.67 P99: 3305.00 P99.9: 3305.00 P99.99: 3305.00 ------------------------------------------------------ ( 380, 580 ] 2 13.333% 13.333% ### ( 580, 870 ] 1 6.667% 20.000% # ( 870, 1300 ] 2 13.333% 33.333% ### ( 1300, 1900 ] 5 33.333% 66.667% ####### ( 1900, 2900 ] 3 20.000% 86.667% #### ( 2900, 4400 ] 2 13.333% 100.000% ### ** Level 5 read latency histogram (micros): Count: 80 Average: 676.6250 StdDev: 887.27 Min: 74 Median: 210.0000 Max: 3886 Percentiles: P50: 210.00 P75: 923.75 P99: 3886.00 P99.9: 3886.00 P99.99: 3886.00 ------------------------------------------------------ ( 51, 76 ] 2 2.500% 2.500% # ( 76, 110 ] 18 22.500% 25.000% ##### ( 110, 170 ] 18 22.500% 47.500% ##### ( 170, 250 ] 4 5.000% 52.500% # ( 250, 380 ] 5 6.250% 58.750% # ( 380, 580 ] 7 8.750% 67.500% ## ( 580, 870 ] 5 6.250% 73.750% # ( 870, 1300 ] 8 10.000% 83.750% ## ( 1300, 1900 ] 1 1.250% 85.000% ( 1900, 2900 ] 8 10.000% 95.000% ## ( 2900, 4400 ] 4 5.000% 100.000% # ** DB Stats ** Uptime(secs): 12.2 total, 3.0 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 322.09 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.665 H:M:S, 13.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 327.06 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.425 H:M:S, 14.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/18-00:55:42 ... thread 0: (1000000,5000000) ops and (309038.2,327137.8) ops/second in (3.235846,15.284078) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/12 3.26 GB 157.3 3.0 0.0 3.0 7.7 4.7 0.0 1.6 121.9 310.9 25.32 23.70 77 0.329 3134K 3325 0.0 0.0 L1 4/4 303.55 MB 0.0 1.2 1.2 0.0 1.2 1.2 0.2 1.0 235.4 234.8 5.09 4.97 2 2.547 1217K 3049 0.0 0.0 L2 2/0 54.70 MB 0.5 0.4 0.2 0.2 0.4 0.2 0.9 2.0 249.9 249.8 1.51 1.46 2 0.756 383K 190 0.0 0.0 L3 17/0 952.63 MB 1.0 1.0 0.7 0.3 1.0 0.7 0.5 1.4 204.2 203.9 4.77 4.26 4 1.193 989K 1217 0.0 0.0 L4 161/0 9.40 GB 1.0 0.0 0.0 0.0 0.0 0.0 1.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1056/0 86.86 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1259/16 100.80 GB 0.0 5.5 2.0 3.5 10.2 6.7 4.2 2.2 153.7 283.9 36.69 34.39 85 0.432 5725K 7781 0.0 0.0 Int 0/0 0.00 KB 0.0 5.5 2.0 3.5 10.2 6.7 3.8 2.2 153.7 283.9 36.69 34.39 85 0.432 5725K 7781 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.5 2.0 3.5 5.5 2.0 0.0 0.0 244.0 243.6 23.11 22.21 19 1.216 5725K 7781 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 352.6 13.58 12.18 66 0.206 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 15.5 total, 15.3 interval Flush(GB): cumulative 4.676, interval 4.676 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.17 GB write, 673.94 MB/s write, 5.51 GB read, 364.74 MB/s read, 36.7 seconds Interval compaction: 10.17 GB write, 681.55 MB/s write, 5.51 GB read, 368.86 MB/s read, 36.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 2, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 2, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 21, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 2, total-stops: 21, interval: 23 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1506636 Average: 2.2136 StdDev: 12.27 Min: 0 Median: 1.5006 Max: 6664 Percentiles: P50: 1.50 P75: 1.87 P99: 5.45 P99.9: 9.77 P99.99: 14.99 ------------------------------------------------------ [ 0, 1 ] 245350 16.285% 16.285% ### ( 1, 2 ] 1014650 67.345% 83.630% ############# ( 2, 3 ] 159713 10.601% 94.231% ## ( 3, 4 ] 51130 3.394% 97.624% # ( 4, 6 ] 28509 1.892% 99.517% ( 6, 10 ] 6131 0.407% 99.923% ( 10, 15 ] 1004 0.067% 99.990% ( 15, 22 ] 114 0.008% 99.998% ( 22, 34 ] 33 0.002% 100.000% ( 34, 51 ] 11 0.001% 100.001% ( 51, 76 ] 10 0.001% 100.001% ( 76, 110 ] 83 0.006% 100.007% ( 110, 170 ] 126 0.008% 100.015% ( 170, 250 ] 67 0.004% 100.020% ( 250, 380 ] 52 0.003% 100.023% ( 380, 580 ] 43 0.003% 100.026% ( 580, 870 ] 29 0.002% 100.028% ( 870, 1300 ] 21 0.001% 100.029% ( 1300, 1900 ] 5 0.000% 100.030% ( 1900, 2900 ] 6 0.000% 100.030% ( 2900, 4400 ] 6 0.000% 100.030% ( 6600, 9900 ] 1 0.000% 100.030% ** Level 1 read latency histogram (micros): Count: 157300 Average: 2.5806 StdDev: 11.28 Min: 1 Median: 1.6452 Max: 2572 Percentiles: P50: 1.65 P75: 2.10 P99: 6.13 P99.9: 16.62 P99.99: 307.09 ------------------------------------------------------ [ 0, 1 ] 12111 7.699% 7.699% ## ( 1, 2 ] 103130 65.563% 73.262% ############# ( 2, 3 ] 26869 17.081% 90.343% ### ( 3, 4 ] 7601 4.832% 95.175% # ( 4, 6 ] 5974 3.798% 98.973% # ( 6, 10 ] 1292 0.821% 99.795% ( 10, 15 ] 162 0.103% 99.898% ( 15, 22 ] 16 0.010% 99.908% ( 22, 34 ] 11 0.007% 99.915% ( 51, 76 ] 6 0.004% 99.919% ( 76, 110 ] 63 0.040% 99.959% ( 110, 170 ] 30 0.019% 99.978% ( 170, 250 ] 14 0.009% 99.987% ( 250, 380 ] 12 0.008% 99.994% ( 380, 580 ] 14 0.009% 100.003% ( 580, 870 ] 3 0.002% 100.005% ( 870, 1300 ] 4 0.003% 100.008% ( 1300, 1900 ] 1 0.001% 100.008% ( 1900, 2900 ] 1 0.001% 100.009% ** Level 2 read latency histogram (micros): Count: 95439 Average: 2.0753 StdDev: 12.07 Min: 0 Median: 1.3917 Max: 1943 Percentiles: P50: 1.39 P75: 1.79 P99: 5.20 P99.9: 9.46 P99.99: 650.38 ------------------------------------------------------ [ 0, 1 ] 24043 25.192% 25.192% ##### ( 1, 2 ] 60450 63.339% 88.531% ############# ( 2, 3 ] 6673 6.992% 95.523% # ( 3, 4 ] 2457 2.574% 98.097% # ( 4, 6 ] 1433 1.501% 99.599% ( 6, 10 ] 332 0.348% 99.947% ( 10, 15 ] 25 0.026% 99.973% ( 15, 22 ] 8 0.008% 99.981% ( 22, 34 ] 2 0.002% 99.983% ( 250, 380 ] 5 0.005% 99.988% ( 580, 870 ] 6 0.006% 99.995% ( 870, 1300 ] 3 0.003% 99.998% ( 1300, 1900 ] 1 0.001% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 69927 Average: 3.6193 StdDev: 52.14 Min: 0 Median: 1.6497 Max: 4611 Percentiles: P50: 1.65 P75: 2.14 P99: 5.97 P99.9: 38.56 P99.99: 2025.91 ------------------------------------------------------ [ 0, 1 ] 5753 8.227% 8.227% ## ( 1, 2 ] 44959 64.294% 72.521% ############# ( 2, 3 ] 12198 17.444% 89.965% ### ( 3, 4 ] 3608 5.160% 95.125% # ( 4, 6 ] 2746 3.927% 99.052% # ( 6, 10 ] 454 0.649% 99.701% ( 10, 15 ] 102 0.146% 99.847% ( 15, 22 ] 8 0.011% 99.858% ( 22, 34 ] 28 0.040% 99.898% ( 34, 51 ] 4 0.006% 99.904% ( 51, 76 ] 2 0.003% 99.907% ( 110, 170 ] 5 0.007% 99.914% ( 170, 250 ] 8 0.011% 99.926% ( 250, 380 ] 7 0.010% 99.936% ( 380, 580 ] 8 0.011% 99.947% ( 580, 870 ] 8 0.011% 99.959% ( 870, 1300 ] 13 0.019% 99.977% ( 1300, 1900 ] 8 0.011% 99.989% ( 1900, 2900 ] 8 0.011% 100.000% ( 2900, 4400 ] 5 0.007% 100.007% ( 4400, 6600 ] 2 0.003% 100.010% ** Level 4 read latency histogram (micros): Count: 15 Average: 1690.9333 StdDev: 913.73 Min: 424 Median: 1600.0000 Max: 3305 Percentiles: P50: 1600.00 P75: 2316.67 P99: 3305.00 P99.9: 3305.00 P99.99: 3305.00 ------------------------------------------------------ ( 380, 580 ] 2 13.333% 13.333% ### ( 580, 870 ] 1 6.667% 20.000% # ( 870, 1300 ] 2 13.333% 33.333% ### ( 1300, 1900 ] 5 33.333% 66.667% ####### ( 1900, 2900 ] 3 20.000% 86.667% #### ( 2900, 4400 ] 2 13.333% 100.000% ### ** Level 5 read latency histogram (micros): Count: 80 Average: 676.6250 StdDev: 887.27 Min: 74 Median: 210.0000 Max: 3886 Percentiles: P50: 210.00 P75: 923.75 P99: 3886.00 P99.9: 3886.00 P99.99: 3886.00 ------------------------------------------------------ ( 51, 76 ] 2 2.500% 2.500% # ( 76, 110 ] 18 22.500% 25.000% ##### ( 110, 170 ] 18 22.500% 47.500% ##### ( 170, 250 ] 4 5.000% 52.500% # ( 250, 380 ] 5 6.250% 58.750% # ( 380, 580 ] 7 8.750% 67.500% ## ( 580, 870 ] 5 6.250% 73.750% # ( 870, 1300 ] 8 10.000% 83.750% ## ( 1300, 1900 ] 1 1.250% 85.000% ( 1900, 2900 ] 8 10.000% 95.000% ## ( 2900, 4400 ] 4 5.000% 100.000% # ** DB Stats ** Uptime(secs): 15.5 total, 3.2 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 318.33 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.300 H:M:S, 14.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 304.15 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.635 H:M:S, 19.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/18-00:55:45 ... thread 0: (1000000,6000000) ops and (293776.5,321061.2) ops/second in (3.403948,18.688026) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/13 2.83 GB 6.3 4.1 0.0 4.1 9.7 5.7 0.0 1.7 129.0 308.6 32.24 30.24 96 0.336 4221K 4426 0.0 0.0 L1 15/15 1.70 GB 0.0 2.9 2.6 0.3 2.9 2.6 0.2 1.1 231.5 230.2 12.75 12.35 3 4.249 2996K 16K 0.0 0.0 L2 2/0 54.70 MB 0.5 0.4 0.2 0.2 0.4 0.2 0.9 2.0 249.9 249.8 1.51 1.46 2 0.756 383K 190 0.0 0.0 L3 17/0 952.63 MB 1.0 1.0 0.7 0.3 1.0 0.7 0.5 1.4 204.2 203.9 4.77 4.26 4 1.193 989K 1217 0.0 0.0 L4 161/0 9.40 GB 1.0 0.0 0.0 0.0 0.0 0.0 1.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1056/0 86.86 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1265/28 101.77 GB 0.0 8.3 3.5 4.8 13.9 9.1 4.2 2.5 165.0 277.6 51.27 48.31 105 0.488 8590K 22K 0.0 0.0 Int 0/0 0.00 KB 0.0 8.3 3.5 4.8 13.9 9.1 3.8 2.5 165.0 277.6 51.27 48.31 105 0.488 8590K 22K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 8.3 3.5 4.8 8.2 3.4 0.0 0.0 242.8 242.2 34.84 33.59 24 1.452 8590K 22K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 352.8 16.43 14.72 81 0.203 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 18.9 total, 18.7 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.90 GB write, 754.59 MB/s write, 8.26 GB read, 448.55 MB/s read, 51.3 seconds Interval compaction: 13.90 GB write, 761.56 MB/s write, 8.26 GB read, 452.69 MB/s read, 51.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 7, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 7, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 23, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 7, total-stops: 23, interval: 30 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1792499 Average: 2.2101 StdDev: 12.95 Min: 0 Median: 1.4870 Max: 6664 Percentiles: P50: 1.49 P75: 1.86 P99: 5.45 P99.9: 9.94 P99.99: 42.89 ------------------------------------------------------ [ 0, 1 ] 312276 17.421% 17.421% ### ( 1, 2 ] 1199095 66.895% 84.316% ############# ( 2, 3 ] 181074 10.102% 94.418% ## ( 3, 4 ] 58917 3.287% 97.705% # ( 4, 6 ] 31932 1.781% 99.486% ( 6, 10 ] 7527 0.420% 99.906% ( 10, 15 ] 1306 0.073% 99.979% ( 15, 22 ] 150 0.008% 99.988% ( 22, 34 ] 37 0.002% 99.990% ( 34, 51 ] 11 0.001% 99.990% ( 51, 76 ] 12 0.001% 99.991% ( 76, 110 ] 169 0.009% 100.000% ( 110, 170 ] 201 0.011% 100.012% ( 170, 250 ] 91 0.005% 100.017% ( 250, 380 ] 68 0.004% 100.020% ( 380, 580 ] 51 0.003% 100.023% ( 580, 870 ] 40 0.002% 100.026% ( 870, 1300 ] 23 0.001% 100.027% ( 1300, 1900 ] 7 0.000% 100.027% ( 1900, 2900 ] 8 0.000% 100.028% ( 2900, 4400 ] 10 0.001% 100.028% ( 6600, 9900 ] 1 0.000% 100.028% ** Level 1 read latency histogram (micros): Count: 193662 Average: 2.7711 StdDev: 15.85 Min: 0 Median: 1.6189 Max: 2572 Percentiles: P50: 1.62 P75: 2.00 P99: 6.92 P99.9: 120.00 P99.99: 559.42 ------------------------------------------------------ [ 0, 1 ] 17900 9.243% 9.243% ## ( 1, 2 ] 127527 65.850% 75.093% ############# ( 2, 3 ] 30353 15.673% 90.766% ### ( 3, 4 ] 8873 4.582% 95.348% # ( 4, 6 ] 6666 3.442% 98.790% # ( 6, 10 ] 1769 0.913% 99.704% ( 10, 15 ] 187 0.097% 99.800% ( 15, 22 ] 20 0.010% 99.810% ( 22, 34 ] 11 0.006% 99.816% ( 34, 51 ] 1 0.001% 99.817% ( 51, 76 ] 7 0.004% 99.820% ( 76, 110 ] 138 0.071% 99.892% ( 110, 170 ] 98 0.051% 99.942% ( 170, 250 ] 40 0.021% 99.963% ( 250, 380 ] 32 0.017% 99.979% ( 380, 580 ] 23 0.012% 99.991% ( 580, 870 ] 15 0.008% 99.999% ( 870, 1300 ] 9 0.005% 100.004% ( 1300, 1900 ] 6 0.003% 100.007% ( 1900, 2900 ] 1 0.001% 100.007% ** Level 2 read latency histogram (micros): Count: 95439 Average: 2.0753 StdDev: 12.07 Min: 0 Median: 1.3917 Max: 1943 Percentiles: P50: 1.39 P75: 1.79 P99: 5.20 P99.9: 9.46 P99.99: 650.38 ------------------------------------------------------ [ 0, 1 ] 24043 25.192% 25.192% ##### ( 1, 2 ] 60450 63.339% 88.531% ############# ( 2, 3 ] 6673 6.992% 95.523% # ( 3, 4 ] 2457 2.574% 98.097% # ( 4, 6 ] 1433 1.501% 99.599% ( 6, 10 ] 332 0.348% 99.947% ( 10, 15 ] 25 0.026% 99.973% ( 15, 22 ] 8 0.008% 99.981% ( 22, 34 ] 2 0.002% 99.983% ( 250, 380 ] 5 0.005% 99.988% ( 580, 870 ] 6 0.006% 99.995% ( 870, 1300 ] 3 0.003% 99.998% ( 1300, 1900 ] 1 0.001% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 69927 Average: 3.6193 StdDev: 52.14 Min: 0 Median: 1.6497 Max: 4611 Percentiles: P50: 1.65 P75: 2.14 P99: 5.97 P99.9: 38.56 P99.99: 2025.91 ------------------------------------------------------ [ 0, 1 ] 5753 8.227% 8.227% ## ( 1, 2 ] 44959 64.294% 72.521% ############# ( 2, 3 ] 12198 17.444% 89.965% ### ( 3, 4 ] 3608 5.160% 95.125% # ( 4, 6 ] 2746 3.927% 99.052% # ( 6, 10 ] 454 0.649% 99.701% ( 10, 15 ] 102 0.146% 99.847% ( 15, 22 ] 8 0.011% 99.858% ( 22, 34 ] 28 0.040% 99.898% ( 34, 51 ] 4 0.006% 99.904% ( 51, 76 ] 2 0.003% 99.907% ( 110, 170 ] 5 0.007% 99.914% ( 170, 250 ] 8 0.011% 99.926% ( 250, 380 ] 7 0.010% 99.936% ( 380, 580 ] 8 0.011% 99.947% ( 580, 870 ] 8 0.011% 99.959% ( 870, 1300 ] 13 0.019% 99.977% ( 1300, 1900 ] 8 0.011% 99.989% ( 1900, 2900 ] 8 0.011% 100.000% ( 2900, 4400 ] 5 0.007% 100.007% ( 4400, 6600 ] 2 0.003% 100.010% ** Level 4 read latency histogram (micros): Count: 15 Average: 1690.9333 StdDev: 913.73 Min: 424 Median: 1600.0000 Max: 3305 Percentiles: P50: 1600.00 P75: 2316.67 P99: 3305.00 P99.9: 3305.00 P99.99: 3305.00 ------------------------------------------------------ ( 380, 580 ] 2 13.333% 13.333% ### ( 580, 870 ] 1 6.667% 20.000% # ( 870, 1300 ] 2 13.333% 33.333% ### ( 1300, 1900 ] 5 33.333% 66.667% ####### ( 1900, 2900 ] 3 20.000% 86.667% #### ( 2900, 4400 ] 2 13.333% 100.000% ### ** Level 5 read latency histogram (micros): Count: 80 Average: 676.6250 StdDev: 887.27 Min: 74 Median: 210.0000 Max: 3886 Percentiles: P50: 210.00 P75: 923.75 P99: 3886.00 P99.9: 3886.00 P99.99: 3886.00 ------------------------------------------------------ ( 51, 76 ] 2 2.500% 2.500% # ( 76, 110 ] 18 22.500% 25.000% ##### ( 110, 170 ] 18 22.500% 47.500% ##### ( 170, 250 ] 4 5.000% 52.500% # ( 250, 380 ] 5 6.250% 58.750% # ( 380, 580 ] 7 8.750% 67.500% ## ( 580, 870 ] 5 6.250% 73.750% # ( 870, 1300 ] 8 10.000% 83.750% ## ( 1300, 1900 ] 1 1.250% 85.000% ( 1900, 2900 ] 8 10.000% 95.000% ## ( 2900, 4400 ] 4 5.000% 100.000% # ** DB Stats ** Uptime(secs): 18.9 total, 3.4 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 313.07 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.147 H:M:S, 16.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 289.13 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.847 H:M:S, 24.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/10/18-00:56:07 ... thread 0: (1000000,7000000) ops and (46084.7,173322.2) ops/second in (21.699172,40.387198) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 7/4 1.04 GB 81.8 4.9 0.0 4.9 11.5 6.6 0.0 1.7 130.3 308.2 38.20 35.84 114 0.335 5053K 5304 0.0 0.0 L1 8/0 748.15 MB 74.8 7.3 5.3 2.0 7.2 5.2 0.2 1.4 219.8 216.8 34.21 31.75 4 8.553 7636K 103K 0.0 0.0 L2 23/17 2.69 GB 7.7 0.4 0.2 0.2 0.4 0.2 4.6 2.0 249.9 249.8 1.51 1.46 2 0.756 383K 190 0.0 0.0 L3 25/18 1.94 GB 0.9 1.0 0.7 0.3 1.0 0.7 1.5 1.4 204.2 203.9 4.77 4.26 4 1.193 989K 1217 0.0 0.0 L4 161/2 9.40 GB 1.0 0.0 0.0 0.0 0.0 0.0 1.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1056/0 86.86 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1280/41 102.67 GB 0.0 13.5 6.2 7.3 20.1 12.8 8.8 3.0 176.0 261.0 78.70 73.31 124 0.635 14M 110K 0.0 0.0 Int 0/0 0.00 KB 0.0 13.5 6.2 7.3 20.1 12.8 8.4 3.0 176.0 261.0 78.70 73.31 124 0.635 14M 110K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.5 6.2 7.3 13.4 6.1 0.0 0.0 233.1 231.2 59.42 56.05 28 2.122 14M 110K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 353.0 19.27 17.26 96 0.201 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 40.6 total, 40.4 interval Flush(GB): cumulative 6.644, interval 6.644 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 20.06 GB write, 506.44 MB/s write, 13.52 GB read, 341.44 MB/s read, 78.7 seconds Interval compaction: 20.06 GB write, 508.61 MB/s write, 13.52 GB read, 342.90 MB/s read, 78.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 25, pending-compaction-bytes-delays: 6, pending-compaction-bytes-stops: 0, total-delays: 19, total-stops: 25, interval: 44 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2734770 Average: 2.6886 StdDev: 19.45 Min: 0 Median: 1.4453 Max: 10230 Percentiles: P50: 1.45 P75: 1.83 P99: 5.86 P99.9: 145.90 P99.99: 336.47 ------------------------------------------------------ [ 0, 1 ] 574254 20.998% 20.998% #### ( 1, 2 ] 1781186 65.131% 86.129% ############# ( 2, 3 ] 229270 8.384% 94.513% ## ( 3, 4 ] 79008 2.889% 97.402% # ( 4, 6 ] 46933 1.716% 99.118% ( 6, 10 ] 10657 0.390% 99.508% ( 10, 15 ] 2404 0.088% 99.596% ( 15, 22 ] 417 0.015% 99.611% ( 22, 34 ] 48 0.002% 99.613% ( 34, 51 ] 37 0.001% 99.614% ( 51, 76 ] 110 0.004% 99.618% ( 76, 110 ] 5416 0.198% 99.816% ( 110, 170 ] 3838 0.140% 99.956% ( 170, 250 ] 559 0.020% 99.977% ( 250, 380 ] 542 0.020% 99.997% ( 380, 580 ] 472 0.017% 100.014% ( 580, 870 ] 151 0.006% 100.019% ( 870, 1300 ] 36 0.001% 100.021% ( 1300, 1900 ] 12 0.000% 100.021% ( 1900, 2900 ] 21 0.001% 100.022% ( 2900, 4400 ] 14 0.001% 100.022% ( 6600, 9900 ] 4 0.000% 100.023% ( 9900, 14000 ] 1 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 667026 Average: 3.3816 StdDev: 55.26 Min: 0 Median: 1.4144 Max: 13107 Percentiles: P50: 1.41 P75: 1.84 P99: 8.37 P99.9: 164.92 P99.99: 728.79 ------------------------------------------------------ [ 0, 1 ] 170030 25.491% 25.491% ##### ( 1, 2 ] 394542 59.149% 84.640% ############ ( 2, 3 ] 51927 7.785% 92.425% ## ( 3, 4 ] 22956 3.442% 95.867% # ( 4, 6 ] 18330 2.748% 98.615% # ( 6, 10 ] 4344 0.651% 99.266% ( 10, 15 ] 879 0.132% 99.398% ( 15, 22 ] 203 0.030% 99.428% ( 22, 34 ] 23 0.003% 99.432% ( 34, 51 ] 1 0.000% 99.432% ( 51, 76 ] 100 0.015% 99.447% ( 76, 110 ] 1948 0.292% 99.739% ( 110, 170 ] 1181 0.177% 99.916% ( 170, 250 ] 233 0.035% 99.951% ( 250, 380 ] 143 0.021% 99.972% ( 380, 580 ] 93 0.014% 99.986% ( 580, 870 ] 61 0.009% 99.995% ( 870, 1300 ] 55 0.008% 100.003% ( 1300, 1900 ] 26 0.004% 100.007% ( 1900, 2900 ] 15 0.002% 100.010% ( 2900, 4400 ] 16 0.002% 100.012% ( 4400, 6600 ] 9 0.001% 100.013% ( 6600, 9900 ] 13 0.002% 100.015% ( 9900, 14000 ] 5 0.001% 100.016% ** Level 2 read latency histogram (micros): Count: 95700 Average: 2.2405 StdDev: 14.33 Min: 0 Median: 1.3929 Max: 1943 Percentiles: P50: 1.39 P75: 1.79 P99: 5.31 P99.9: 13.90 P99.99: 722.10 ------------------------------------------------------ [ 0, 1 ] 24057 25.138% 25.138% ##### ( 1, 2 ] 60555 63.276% 88.414% ############# ( 2, 3 ] 6708 7.009% 95.423% # ( 3, 4 ] 2474 2.585% 98.008% # ( 4, 6 ] 1451 1.516% 99.525% ( 6, 10 ] 339 0.354% 99.879% ( 10, 15 ] 26 0.027% 99.906% ( 15, 22 ] 8 0.008% 99.914% ( 22, 34 ] 2 0.002% 99.916% ( 76, 110 ] 6 0.006% 99.923% ( 110, 170 ] 20 0.021% 99.944% ( 170, 250 ] 17 0.018% 99.961% ( 250, 380 ] 16 0.017% 99.978% ( 380, 580 ] 8 0.008% 99.986% ( 580, 870 ] 7 0.007% 99.994% ( 870, 1300 ] 4 0.004% 99.998% ( 1300, 1900 ] 1 0.001% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 85369 Average: 7.4127 StdDev: 145.87 Min: 0 Median: 1.7051 Max: 13170 Percentiles: P50: 1.71 P75: 2.37 P99: 8.78 P99.9: 786.03 P99.99: 3835.52 ------------------------------------------------------ [ 0, 1 ] 5976 7.000% 7.000% # ( 1, 2 ] 52066 60.989% 67.990% ############ ( 2, 3 ] 16210 18.988% 86.978% #### ( 3, 4 ] 5257 6.158% 93.136% # ( 4, 6 ] 4420 5.178% 98.313% # ( 6, 10 ] 845 0.990% 99.303% ( 10, 15 ] 174 0.204% 99.507% ( 15, 22 ] 18 0.021% 99.528% ( 22, 34 ] 28 0.033% 99.561% ( 34, 51 ] 7 0.008% 99.569% ( 51, 76 ] 3 0.004% 99.572% ( 76, 110 ] 56 0.066% 99.638% ( 110, 170 ] 96 0.112% 99.750% ( 170, 250 ] 46 0.054% 99.804% ( 250, 380 ] 33 0.039% 99.843% ( 380, 580 ] 34 0.040% 99.883% ( 580, 870 ] 22 0.026% 99.909% ( 870, 1300 ] 21 0.025% 99.933% ( 1300, 1900 ] 12 0.014% 99.947% ( 1900, 2900 ] 20 0.023% 99.971% ( 2900, 4400 ] 28 0.033% 100.004% ( 4400, 6600 ] 8 0.009% 100.013% ( 6600, 9900 ] 12 0.014% 100.027% ( 9900, 14000 ] 2 0.002% 100.029% ** Level 4 read latency histogram (micros): Count: 15 Average: 1690.9333 StdDev: 913.73 Min: 424 Median: 1600.0000 Max: 3305 Percentiles: P50: 1600.00 P75: 2316.67 P99: 3305.00 P99.9: 3305.00 P99.99: 3305.00 ------------------------------------------------------ ( 380, 580 ] 2 13.333% 13.333% ### ( 580, 870 ] 1 6.667% 20.000% # ( 870, 1300 ] 2 13.333% 33.333% ### ( 1300, 1900 ] 5 33.333% 66.667% ####### ( 1900, 2900 ] 3 20.000% 86.667% #### ( 2900, 4400 ] 2 13.333% 100.000% ### ** Level 5 read latency histogram (micros): Count: 80 Average: 676.6250 StdDev: 887.27 Min: 74 Median: 210.0000 Max: 3886 Percentiles: P50: 210.00 P75: 923.75 P99: 3886.00 P99.9: 3886.00 P99.99: 3886.00 ------------------------------------------------------ ( 51, 76 ] 2 2.500% 2.500% # ( 76, 110 ] 18 22.500% 25.000% ##### ( 110, 170 ] 18 22.500% 47.500% ##### ( 170, 250 ] 4 5.000% 52.500% # ( 250, 380 ] 5 6.250% 58.750% # ( 380, 580 ] 7 8.750% 67.500% ## ( 580, 870 ] 5 6.250% 73.750% # ( 870, 1300 ] 8 10.000% 83.750% ## ( 1300, 1900 ] 1 1.250% 85.000% ( 1900, 2900 ] 8 10.000% 95.000% ## ( 2900, 4400 ] 4 5.000% 100.000% # ** DB Stats ** Uptime(secs): 40.6 total, 21.7 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 169.85 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:22.107 H:M:S, 54.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 45.36 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:18.960 H:M:S, 87.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 9 num-running-flushes: 1 2024/10/18-00:56:14 ... thread 0: (1000000,8000000) ops and (138551.3,168050.5) ops/second in (7.217542,47.604740) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/9 1.97 GB 50.3 5.5 0.0 5.5 13.0 7.6 0.0 1.7 115.8 275.6 48.44 41.20 126 0.384 5692K 6051 0.0 0.0 L1 8/7 748.15 MB 0.0 7.3 5.3 2.0 7.2 5.2 0.2 1.4 219.8 216.8 34.21 31.75 4 8.553 7636K 103K 0.0 0.0 L2 1/1 128.78 MB 0.0 0.4 0.2 0.2 0.4 0.2 4.6 2.0 249.9 249.8 1.51 1.46 2 0.756 383K 190 0.0 0.0 L3 26/9 1.81 GB 1.2 4.9 3.3 1.7 4.9 3.2 1.5 1.5 145.0 144.4 34.83 23.21 16 2.177 5131K 23K 0.0 0.0 L4 169/6 9.79 GB 1.0 2.6 2.3 0.3 2.5 2.3 1.6 1.1 119.3 118.6 21.99 12.17 28 0.786 2635K 13K 0.0 0.0 L5 1084/0 89.14 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1299/32 103.56 GB 0.0 20.7 11.1 9.6 28.1 18.5 11.5 3.7 150.2 204.1 140.99 109.79 176 0.801 21M 146K 0.0 0.0 Int 0/0 0.00 KB 0.0 20.7 11.1 9.6 28.1 18.5 11.1 3.7 150.2 204.1 140.99 109.79 176 0.801 21M 146K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 20.7 11.1 9.6 20.5 10.9 0.0 0.0 183.9 182.6 115.15 89.79 70 1.645 21M 146K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 299.9 25.84 20.00 106 0.244 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 47.8 total, 47.6 interval Flush(GB): cumulative 7.567, interval 7.567 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 28.11 GB write, 602.37 MB/s write, 20.68 GB read, 443.30 MB/s read, 141.0 seconds Interval compaction: 28.11 GB write, 604.55 MB/s write, 20.68 GB read, 444.91 MB/s read, 141.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 64, pending-compaction-bytes-delays: 6, pending-compaction-bytes-stops: 0, total-delays: 19, total-stops: 64, interval: 83 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3167487 Average: 2.9712 StdDev: 30.96 Min: 0 Median: 1.4994 Max: 14183 Percentiles: P50: 1.50 P75: 1.90 P99: 6.80 P99.9: 147.95 P99.99: 452.02 ------------------------------------------------------ [ 0, 1 ] 584371 18.449% 18.449% #### ( 1, 2 ] 2000987 63.173% 81.622% ############# ( 2, 3 ] 327568 10.342% 91.963% ## ( 3, 4 ] 125443 3.960% 95.924% # ( 4, 6 ] 93375 2.948% 98.872% # ( 6, 10 ] 20409 0.644% 99.516% ( 10, 15 ] 3573 0.113% 99.629% ( 15, 22 ] 526 0.017% 99.645% ( 22, 34 ] 63 0.002% 99.647% ( 34, 51 ] 37 0.001% 99.648% ( 51, 76 ] 111 0.004% 99.652% ( 76, 110 ] 5420 0.171% 99.823% ( 110, 170 ] 3854 0.122% 99.945% ( 170, 250 ] 597 0.019% 99.964% ( 250, 380 ] 614 0.019% 99.983% ( 380, 580 ] 620 0.020% 100.003% ( 580, 870 ] 285 0.009% 100.012% ( 870, 1300 ] 139 0.004% 100.016% ( 1300, 1900 ] 69 0.002% 100.018% ( 1900, 2900 ] 75 0.002% 100.020% ( 2900, 4400 ] 58 0.002% 100.022% ( 4400, 6600 ] 13 0.000% 100.023% ( 6600, 9900 ] 6 0.000% 100.023% ( 9900, 14000 ] 2 0.000% 100.023% ( 14000, 22000 ] 1 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 1495698 Average: 5.9919 StdDev: 98.08 Min: 0 Median: 1.7116 Max: 13676 Percentiles: P50: 1.71 P75: 2.55 P99: 9.19 P99.9: 183.33 P99.99: 622.05 ------------------------------------------------------ [ 0, 1 ] 179637 12.010% 12.010% ## ( 1, 2 ] 798517 53.388% 65.398% ########### ( 2, 3 ] 259204 17.330% 82.728% ### ( 3, 4 ] 115593 7.728% 90.456% ## ( 4, 6 ] 107058 7.158% 97.614% # ( 6, 10 ] 25976 1.737% 99.351% ( 10, 15 ] 4146 0.277% 99.628% ( 15, 22 ] 624 0.042% 99.670% ( 22, 34 ] 51 0.003% 99.673% ( 34, 51 ] 23 0.002% 99.674% ( 51, 76 ] 103 0.007% 99.681% ( 76, 110 ] 1968 0.132% 99.813% ( 110, 170 ] 1242 0.083% 99.896% ( 170, 250 ] 362 0.024% 99.920% ( 250, 380 ] 428 0.029% 99.949% ( 380, 580 ] 539 0.036% 99.985% ( 580, 870 ] 534 0.036% 100.021% ( 870, 1300 ] 420 0.028% 100.049% ( 1300, 1900 ] 270 0.018% 100.067% ( 1900, 2900 ] 341 0.023% 100.089% ( 2900, 4400 ] 385 0.026% 100.115% ( 4400, 6600 ] 98 0.007% 100.122% ( 6600, 9900 ] 38 0.003% 100.124% ( 9900, 14000 ] 10 0.001% 100.125% ** Level 2 read latency histogram (micros): Count: 124205 Average: 4.0516 StdDev: 60.70 Min: 0 Median: 1.5190 Max: 5285 Percentiles: P50: 1.52 P75: 1.94 P99: 7.57 P99.9: 551.14 P99.99: 3543.79 ------------------------------------------------------ [ 0, 1 ] 24210 19.492% 19.492% #### ( 1, 2 ] 73015 58.786% 78.278% ############ ( 2, 3 ] 14337 11.543% 89.821% ## ( 3, 4 ] 5980 4.815% 94.635% # ( 4, 6 ] 4933 3.972% 98.607% # ( 6, 10 ] 1244 1.002% 99.609% ( 10, 15 ] 173 0.139% 99.748% ( 15, 22 ] 36 0.029% 99.777% ( 22, 34 ] 3 0.002% 99.779% ( 34, 51 ] 3 0.002% 99.782% ( 76, 110 ] 6 0.005% 99.787% ( 110, 170 ] 27 0.022% 99.808% ( 170, 250 ] 30 0.024% 99.833% ( 250, 380 ] 47 0.038% 99.870% ( 380, 580 ] 43 0.035% 99.905% ( 580, 870 ] 49 0.039% 99.944% ( 870, 1300 ] 26 0.021% 99.965% ( 1300, 1900 ] 13 0.010% 99.976% ( 1900, 2900 ] 12 0.010% 99.986% ( 2900, 4400 ] 13 0.010% 99.996% ( 4400, 6600 ] 5 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 970788 Average: 10.0252 StdDev: 138.72 Min: 0 Median: 1.9999 Max: 14754 Percentiles: P50: 2.00 P75: 2.99 P99: 9.49 P99.9: 789.30 P99.99: 1183.60 ------------------------------------------------------ [ 0, 1 ] 14914 1.536% 1.536% ( 1, 2 ] 470539 48.470% 50.006% ########## ( 2, 3 ] 244303 25.165% 75.172% ##### ( 3, 4 ] 110483 11.381% 86.552% ## ( 4, 6 ] 104080 10.721% 97.273% ## ( 6, 10 ] 19214 1.979% 99.253% ( 10, 15 ] 3322 0.342% 99.595% ( 15, 22 ] 340 0.035% 99.630% ( 22, 34 ] 59 0.006% 99.636% ( 34, 51 ] 22 0.002% 99.638% ( 51, 76 ] 20 0.002% 99.640% ( 76, 110 ] 73 0.008% 99.648% ( 110, 170 ] 186 0.019% 99.667% ( 170, 250 ] 248 0.026% 99.693% ( 250, 380 ] 471 0.049% 99.741% ( 380, 580 ] 818 0.084% 99.825% ( 580, 870 ] 1009 0.104% 99.929% ( 870, 1300 ] 813 0.084% 100.013% ( 1300, 1900 ] 477 0.049% 100.062% ( 1900, 2900 ] 449 0.046% 100.108% ( 2900, 4400 ] 509 0.052% 100.161% ( 4400, 6600 ] 122 0.013% 100.173% ( 6600, 9900 ] 50 0.005% 100.179% ( 9900, 14000 ] 4 0.000% 100.179% ( 14000, 22000 ] 1 0.000% 100.179% ** Level 4 read latency histogram (micros): Count: 315 Average: 355.1937 StdDev: 835.08 Min: 1 Median: 2.8304 Max: 7525 Percentiles: P50: 2.83 P75: 207.86 P99: 4055.00 P99.9: 7525.00 P99.99: 7525.00 ------------------------------------------------------ [ 0, 1 ] 23 7.302% 7.302% # ( 1, 2 ] 88 27.937% 35.238% ###### ( 2, 3 ] 56 17.778% 53.016% #### ( 3, 4 ] 11 3.492% 56.508% # ( 4, 6 ] 2 0.635% 57.143% ( 6, 10 ] 7 2.222% 59.365% ( 10, 15 ] 5 1.587% 60.952% ( 15, 22 ] 6 1.905% 62.857% ( 22, 34 ] 7 2.222% 65.079% ( 34, 51 ] 8 2.540% 67.619% # ( 51, 76 ] 4 1.270% 68.889% ( 76, 110 ] 5 1.587% 70.476% ( 110, 170 ] 1 0.317% 70.794% ( 170, 250 ] 28 8.889% 79.683% ## ( 250, 380 ] 3 0.952% 80.635% ( 380, 580 ] 3 0.952% 81.587% ( 580, 870 ] 10 3.175% 84.762% # ( 870, 1300 ] 16 5.079% 89.841% # ( 1300, 1900 ] 12 3.810% 93.651% # ( 1900, 2900 ] 13 4.127% 97.778% # ( 2900, 4400 ] 5 1.587% 99.365% ( 4400, 6600 ] 1 0.317% 99.683% ( 6600, 9900 ] 1 0.317% 100.000% ** Level 5 read latency histogram (micros): Count: 220 Average: 1408.7636 StdDev: 1508.99 Min: 74 Median: 745.7143 Max: 6550 Percentiles: P50: 745.71 P75: 2260.00 P99: 6227.69 P99.9: 6550.00 P99.99: 6550.00 ------------------------------------------------------ ( 51, 76 ] 2 0.909% 0.909% ( 76, 110 ] 19 8.636% 9.545% ## ( 110, 170 ] 20 9.091% 18.636% ## ( 170, 250 ] 7 3.182% 21.818% # ( 250, 380 ] 21 9.545% 31.364% ## ( 380, 580 ] 29 13.182% 44.545% ### ( 580, 870 ] 21 9.545% 54.091% ## ( 870, 1300 ] 22 10.000% 64.091% ## ( 1300, 1900 ] 15 6.818% 70.909% # ( 1900, 2900 ] 25 11.364% 82.273% ## ( 2900, 4400 ] 26 11.818% 94.091% ## ( 4400, 6600 ] 13 5.909% 100.000% # ** DB Stats ** Uptime(secs): 47.8 total, 7.2 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 164.79 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:26.288 H:M:S, 55.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 136.36 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.180 H:M:S, 57.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/10/18-00:56:18 ... thread 0: (1000000,9000000) ops and (274970.8,175638.9) ops/second in (3.636750,51.241490) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/9 2.95 GB 163.6 6.5 0.0 6.5 15.1 8.5 0.0 1.8 118.3 273.3 56.45 48.04 142 0.398 6780K 7449 0.0 0.0 L1 8/7 748.15 MB 0.0 7.3 5.3 2.0 7.2 5.2 0.2 1.4 219.8 216.8 34.21 31.75 4 8.553 7636K 103K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 4.6 2.0 249.9 249.8 1.51 1.46 2 0.756 383K 190 0.0 0.0 L3 13/0 968.63 MB 1.0 5.2 3.4 1.9 5.2 3.4 1.5 1.5 146.4 145.7 36.59 24.57 17 2.152 5441K 24K 0.0 0.0 L4 167/0 9.42 GB 1.0 3.7 3.2 0.4 3.6 3.2 1.6 1.1 126.7 126.0 29.62 17.34 41 0.722 3771K 19K 0.0 0.0 L5 1100/0 90.49 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1303/16 104.53 GB 0.0 23.1 12.2 11.0 31.5 20.6 12.9 3.7 149.5 203.8 158.38 123.15 206 0.769 24M 155K 0.0 0.0 Int 0/0 0.00 KB 0.0 23.1 12.2 11.0 31.5 20.6 12.5 3.7 149.5 203.8 158.38 123.15 206 0.769 24M 155K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 23.1 12.2 11.0 23.0 12.0 0.0 0.0 183.4 182.2 129.12 100.55 87 1.484 24M 155K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.6 8.6 0.0 0.0 0.0 299.3 29.26 22.59 119 0.246 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 51.4 total, 51.2 interval Flush(GB): cumulative 8.551, interval 8.551 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 31.53 GB write, 627.92 MB/s write, 23.13 GB read, 460.67 MB/s read, 158.4 seconds Interval compaction: 31.53 GB write, 630.03 MB/s write, 23.13 GB read, 462.22 MB/s read, 158.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 72, pending-compaction-bytes-delays: 6, pending-compaction-bytes-stops: 0, total-delays: 19, total-stops: 72, interval: 91 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3468772 Average: 2.9689 StdDev: 31.02 Min: 0 Median: 1.5132 Max: 14183 Percentiles: P50: 1.51 P75: 1.91 P99: 6.76 P99.9: 144.65 P99.99: 444.87 ------------------------------------------------------ [ 0, 1 ] 610089 17.588% 17.588% #### ( 1, 2 ] 2190660 63.154% 80.742% ############# ( 2, 3 ] 377605 10.886% 91.628% ## ( 3, 4 ] 143470 4.136% 95.764% # ( 4, 6 ] 107867 3.110% 98.873% # ( 6, 10 ] 23194 0.669% 99.542% ( 10, 15 ] 3971 0.114% 99.656% ( 15, 22 ] 559 0.016% 99.673% ( 22, 34 ] 66 0.002% 99.674% ( 34, 51 ] 39 0.001% 99.676% ( 51, 76 ] 113 0.003% 99.679% ( 76, 110 ] 5432 0.157% 99.835% ( 110, 170 ] 3876 0.112% 99.947% ( 170, 250 ] 631 0.018% 99.965% ( 250, 380 ] 641 0.018% 99.984% ( 380, 580 ] 654 0.019% 100.003% ( 580, 870 ] 311 0.009% 100.012% ( 870, 1300 ] 160 0.005% 100.016% ( 1300, 1900 ] 81 0.002% 100.019% ( 1900, 2900 ] 83 0.002% 100.021% ( 2900, 4400 ] 67 0.002% 100.023% ( 4400, 6600 ] 15 0.000% 100.023% ( 6600, 9900 ] 6 0.000% 100.024% ( 9900, 14000 ] 2 0.000% 100.024% ( 14000, 22000 ] 1 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 1672202 Average: 5.7510 StdDev: 94.32 Min: 0 Median: 1.7136 Max: 13676 Percentiles: P50: 1.71 P75: 2.54 P99: 9.15 P99.9: 185.80 P99.99: 644.62 ------------------------------------------------------ [ 0, 1 ] 188754 11.288% 11.288% ## ( 1, 2 ] 907142 54.248% 65.536% ########### ( 2, 3 ] 294631 17.619% 83.155% #### ( 3, 4 ] 125690 7.516% 90.672% ## ( 4, 6 ] 116170 6.947% 97.619% # ( 6, 10 ] 29356 1.756% 99.375% ( 10, 15 ] 4440 0.266% 99.640% ( 15, 22 ] 656 0.039% 99.679% ( 22, 34 ] 59 0.004% 99.683% ( 34, 51 ] 27 0.002% 99.684% ( 51, 76 ] 108 0.006% 99.691% ( 76, 110 ] 2024 0.121% 99.812% ( 110, 170 ] 1393 0.083% 99.895% ( 170, 250 ] 404 0.024% 99.919% ( 250, 380 ] 474 0.028% 99.948% ( 380, 580 ] 584 0.035% 99.983% ( 580, 870 ] 551 0.033% 100.016% ( 870, 1300 ] 435 0.026% 100.042% ( 1300, 1900 ] 281 0.017% 100.058% ( 1900, 2900 ] 352 0.021% 100.079% ( 2900, 4400 ] 392 0.023% 100.103% ( 4400, 6600 ] 101 0.006% 100.109% ( 6600, 9900 ] 40 0.002% 100.111% ( 9900, 14000 ] 10 0.001% 100.112% ** Level 2 read latency histogram (micros): Count: 124205 Average: 4.0516 StdDev: 60.70 Min: 0 Median: 1.5190 Max: 5285 Percentiles: P50: 1.52 P75: 1.94 P99: 7.57 P99.9: 551.14 P99.99: 3543.79 ------------------------------------------------------ [ 0, 1 ] 24210 19.492% 19.492% #### ( 1, 2 ] 73015 58.786% 78.278% ############ ( 2, 3 ] 14337 11.543% 89.821% ## ( 3, 4 ] 5980 4.815% 94.635% # ( 4, 6 ] 4933 3.972% 98.607% # ( 6, 10 ] 1244 1.002% 99.609% ( 10, 15 ] 173 0.139% 99.748% ( 15, 22 ] 36 0.029% 99.777% ( 22, 34 ] 3 0.002% 99.779% ( 34, 51 ] 3 0.002% 99.782% ( 76, 110 ] 6 0.005% 99.787% ( 110, 170 ] 27 0.022% 99.808% ( 170, 250 ] 30 0.024% 99.833% ( 250, 380 ] 47 0.038% 99.870% ( 380, 580 ] 43 0.035% 99.905% ( 580, 870 ] 49 0.039% 99.944% ( 870, 1300 ] 26 0.021% 99.965% ( 1300, 1900 ] 13 0.010% 99.976% ( 1900, 2900 ] 12 0.010% 99.986% ( 2900, 4400 ] 13 0.010% 99.996% ( 4400, 6600 ] 5 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 1206194 Average: 9.3119 StdDev: 131.74 Min: 0 Median: 1.9880 Max: 14754 Percentiles: P50: 1.99 P75: 2.98 P99: 9.28 P99.9: 695.37 P99.99: 1068.68 ------------------------------------------------------ [ 0, 1 ] 18509 1.534% 1.534% ( 1, 2 ] 591687 49.054% 50.589% ########## ( 2, 3 ] 301695 25.012% 75.601% ##### ( 3, 4 ] 136465 11.314% 86.914% ## ( 4, 6 ] 127053 10.533% 97.448% ## ( 6, 10 ] 22832 1.893% 99.341% ( 10, 15 ] 3746 0.311% 99.651% ( 15, 22 ] 373 0.031% 99.682% ( 22, 34 ] 62 0.005% 99.687% ( 34, 51 ] 24 0.002% 99.689% ( 51, 76 ] 24 0.002% 99.691% ( 76, 110 ] 94 0.008% 99.699% ( 110, 170 ] 247 0.020% 99.720% ( 170, 250 ] 322 0.027% 99.746% ( 250, 380 ] 523 0.043% 99.790% ( 380, 580 ] 893 0.074% 99.864% ( 580, 870 ] 1103 0.091% 99.955% ( 870, 1300 ] 912 0.076% 100.031% ( 1300, 1900 ] 541 0.045% 100.076% ( 1900, 2900 ] 498 0.041% 100.117% ( 2900, 4400 ] 571 0.047% 100.164% ( 4400, 6600 ] 144 0.012% 100.176% ( 6600, 9900 ] 55 0.005% 100.181% ( 9900, 14000 ] 4 0.000% 100.181% ( 14000, 22000 ] 1 0.000% 100.181% ** Level 4 read latency histogram (micros): Count: 445 Average: 308.8472 StdDev: 772.72 Min: 1 Median: 2.7319 Max: 7525 Percentiles: P50: 2.73 P75: 195.58 P99: 3965.00 P99.9: 7525.00 P99.99: 7525.00 ------------------------------------------------------ [ 0, 1 ] 43 9.663% 9.663% ## ( 1, 2 ] 129 28.989% 38.652% ###### ( 2, 3 ] 69 15.506% 54.157% ### ( 3, 4 ] 15 3.371% 57.528% # ( 4, 6 ] 2 0.449% 57.978% ( 6, 10 ] 8 1.798% 59.775% ( 10, 15 ] 7 1.573% 61.348% ( 15, 22 ] 8 1.798% 63.146% ( 22, 34 ] 12 2.697% 65.843% # ( 34, 51 ] 10 2.247% 68.090% ( 51, 76 ] 7 1.573% 69.663% ( 76, 110 ] 8 1.798% 71.461% ( 110, 170 ] 2 0.449% 71.910% ( 170, 250 ] 43 9.663% 81.573% ## ( 250, 380 ] 5 1.124% 82.697% ( 380, 580 ] 7 1.573% 84.270% ( 580, 870 ] 12 2.697% 86.966% # ( 870, 1300 ] 21 4.719% 91.685% # ( 1300, 1900 ] 14 3.146% 94.831% # ( 1900, 2900 ] 15 3.371% 98.202% # ( 2900, 4400 ] 5 1.124% 99.326% ( 4400, 6600 ] 2 0.449% 99.775% ( 6600, 9900 ] 1 0.225% 100.000% ** Level 5 read latency histogram (micros): Count: 300 Average: 1330.5733 StdDev: 1479.80 Min: 74 Median: 694.2424 Max: 6550 Percentiles: P50: 694.24 P75: 2033.33 P99: 6187.50 P99.9: 6550.00 P99.99: 6550.00 ------------------------------------------------------ ( 51, 76 ] 2 0.667% 0.667% ( 76, 110 ] 32 10.667% 11.333% ## ( 110, 170 ] 30 10.000% 21.333% ## ( 170, 250 ] 11 3.667% 25.000% # ( 250, 380 ] 28 9.333% 34.333% ## ( 380, 580 ] 34 11.333% 45.667% ## ( 580, 870 ] 33 11.000% 56.667% ## ( 870, 1300 ] 29 9.667% 66.333% ## ( 1300, 1900 ] 22 7.333% 73.667% # ( 1900, 2900 ] 30 10.000% 83.667% ## ( 2900, 4400 ] 33 11.000% 94.667% ## ( 4400, 6600 ] 16 5.333% 100.000% # ** DB Stats ** Uptime(secs): 51.4 total, 3.6 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 172.28 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:27.099 H:M:S, 52.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 270.62 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.812 H:M:S, 22.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/18-00:56:21 ... thread 0: (1000000,10000000) ops and (332150.8,184324.4) ops/second in (3.010681,54.252171) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/14 2.77 GB 25.2 7.0 0.0 7.0 16.5 9.5 0.0 1.7 117.6 276.4 61.05 52.30 157 0.389 7291K 7957 0.0 0.0 L1 21/21 1.82 GB 0.0 9.2 6.5 2.7 9.1 6.3 0.2 1.4 210.1 207.5 44.74 40.09 5 8.948 8786K 103K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 4.6 2.0 249.9 249.8 1.51 1.46 2 0.756 383K 190 0.0 0.0 L3 13/0 968.63 MB 1.0 5.2 3.4 1.9 5.2 3.4 1.5 1.5 146.4 145.7 36.59 24.57 17 2.152 5441K 24K 0.0 0.0 L4 167/0 9.42 GB 1.0 3.7 3.2 0.4 3.6 3.2 1.6 1.1 126.7 126.0 29.62 17.34 41 0.722 3771K 19K 0.0 0.0 L5 1100/0 90.49 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1318/35 105.44 GB 0.0 25.5 13.3 12.2 34.8 22.6 12.9 3.7 150.3 205.2 173.51 135.75 222 0.782 25M 155K 0.0 0.0 Int 0/0 0.00 KB 0.0 25.5 13.3 12.2 34.8 22.6 12.5 3.7 150.3 205.2 173.51 135.75 222 0.782 25M 155K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 25.5 13.3 12.2 25.3 13.1 0.0 0.0 184.2 183.0 141.56 110.75 90 1.573 25M 155K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 9.5 9.5 0.0 0.0 0.0 303.7 31.95 25.00 132 0.242 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 54.4 total, 54.3 interval Flush(GB): cumulative 9.474, interval 9.474 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 34.77 GB write, 654.10 MB/s write, 25.46 GB read, 479.01 MB/s read, 173.5 seconds Interval compaction: 34.77 GB write, 656.18 MB/s write, 25.46 GB read, 480.53 MB/s read, 173.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 76, pending-compaction-bytes-delays: 6, pending-compaction-bytes-stops: 0, total-delays: 19, total-stops: 76, interval: 95 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3684146 Average: 2.9258 StdDev: 30.27 Min: 0 Median: 1.5086 Max: 14183 Percentiles: P50: 1.51 P75: 1.90 P99: 6.55 P99.9: 142.76 P99.99: 439.07 ------------------------------------------------------ [ 0, 1 ] 655356 17.789% 17.789% #### ( 1, 2 ] 2333459 63.338% 81.126% ############# ( 2, 3 ] 396076 10.751% 91.877% ## ( 3, 4 ] 148762 4.038% 95.915% # ( 4, 6 ] 110355 2.995% 98.911% # ( 6, 10 ] 23947 0.650% 99.561% ( 10, 15 ] 4079 0.111% 99.671% ( 15, 22 ] 570 0.015% 99.687% ( 22, 34 ] 68 0.002% 99.689% ( 34, 51 ] 39 0.001% 99.690% ( 51, 76 ] 117 0.003% 99.693% ( 76, 110 ] 5458 0.148% 99.841% ( 110, 170 ] 3985 0.108% 99.949% ( 170, 250 ] 652 0.018% 99.967% ( 250, 380 ] 657 0.018% 99.985% ( 380, 580 ] 669 0.018% 100.003% ( 580, 870 ] 317 0.009% 100.011% ( 870, 1300 ] 165 0.004% 100.016% ( 1300, 1900 ] 85 0.002% 100.018% ( 1900, 2900 ] 83 0.002% 100.020% ( 2900, 4400 ] 68 0.002% 100.022% ( 4400, 6600 ] 15 0.000% 100.023% ( 6600, 9900 ] 6 0.000% 100.023% ( 9900, 14000 ] 2 0.000% 100.023% ( 14000, 22000 ] 1 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 1743585 Average: 5.6526 StdDev: 92.53 Min: 0 Median: 1.7031 Max: 13676 Percentiles: P50: 1.70 P75: 2.50 P99: 9.13 P99.9: 193.25 P99.99: 661.24 ------------------------------------------------------ [ 0, 1 ] 200500 11.499% 11.499% ## ( 1, 2 ] 954721 54.756% 66.256% ########### ( 2, 3 ] 302330 17.340% 83.595% ### ( 3, 4 ] 127502 7.313% 90.908% # ( 4, 6 ] 117363 6.731% 97.639% # ( 6, 10 ] 30320 1.739% 99.378% ( 10, 15 ] 4487 0.257% 99.635% ( 15, 22 ] 662 0.038% 99.673% ( 22, 34 ] 59 0.003% 99.676% ( 34, 51 ] 27 0.002% 99.678% ( 51, 76 ] 115 0.007% 99.685% ( 76, 110 ] 2142 0.123% 99.807% ( 110, 170 ] 1487 0.085% 99.893% ( 170, 250 ] 435 0.025% 99.918% ( 250, 380 ] 502 0.029% 99.946% ( 380, 580 ] 597 0.034% 99.981% ( 580, 870 ] 577 0.033% 100.014% ( 870, 1300 ] 451 0.026% 100.040% ( 1300, 1900 ] 285 0.016% 100.056% ( 1900, 2900 ] 352 0.020% 100.076% ( 2900, 4400 ] 392 0.022% 100.099% ( 4400, 6600 ] 101 0.006% 100.104% ( 6600, 9900 ] 40 0.002% 100.107% ( 9900, 14000 ] 10 0.001% 100.107% ** Level 2 read latency histogram (micros): Count: 124205 Average: 4.0516 StdDev: 60.70 Min: 0 Median: 1.5190 Max: 5285 Percentiles: P50: 1.52 P75: 1.94 P99: 7.57 P99.9: 551.14 P99.99: 3543.79 ------------------------------------------------------ [ 0, 1 ] 24210 19.492% 19.492% #### ( 1, 2 ] 73015 58.786% 78.278% ############ ( 2, 3 ] 14337 11.543% 89.821% ## ( 3, 4 ] 5980 4.815% 94.635% # ( 4, 6 ] 4933 3.972% 98.607% # ( 6, 10 ] 1244 1.002% 99.609% ( 10, 15 ] 173 0.139% 99.748% ( 15, 22 ] 36 0.029% 99.777% ( 22, 34 ] 3 0.002% 99.779% ( 34, 51 ] 3 0.002% 99.782% ( 76, 110 ] 6 0.005% 99.787% ( 110, 170 ] 27 0.022% 99.808% ( 170, 250 ] 30 0.024% 99.833% ( 250, 380 ] 47 0.038% 99.870% ( 380, 580 ] 43 0.035% 99.905% ( 580, 870 ] 49 0.039% 99.944% ( 870, 1300 ] 26 0.021% 99.965% ( 1300, 1900 ] 13 0.010% 99.976% ( 1900, 2900 ] 12 0.010% 99.986% ( 2900, 4400 ] 13 0.010% 99.996% ( 4400, 6600 ] 5 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 1206194 Average: 9.3119 StdDev: 131.74 Min: 0 Median: 1.9880 Max: 14754 Percentiles: P50: 1.99 P75: 2.98 P99: 9.28 P99.9: 695.37 P99.99: 1068.68 ------------------------------------------------------ [ 0, 1 ] 18509 1.534% 1.534% ( 1, 2 ] 591687 49.054% 50.589% ########## ( 2, 3 ] 301695 25.012% 75.601% ##### ( 3, 4 ] 136465 11.314% 86.914% ## ( 4, 6 ] 127053 10.533% 97.448% ## ( 6, 10 ] 22832 1.893% 99.341% ( 10, 15 ] 3746 0.311% 99.651% ( 15, 22 ] 373 0.031% 99.682% ( 22, 34 ] 62 0.005% 99.687% ( 34, 51 ] 24 0.002% 99.689% ( 51, 76 ] 24 0.002% 99.691% ( 76, 110 ] 94 0.008% 99.699% ( 110, 170 ] 247 0.020% 99.720% ( 170, 250 ] 322 0.027% 99.746% ( 250, 380 ] 523 0.043% 99.790% ( 380, 580 ] 893 0.074% 99.864% ( 580, 870 ] 1103 0.091% 99.955% ( 870, 1300 ] 912 0.076% 100.031% ( 1300, 1900 ] 541 0.045% 100.076% ( 1900, 2900 ] 498 0.041% 100.117% ( 2900, 4400 ] 571 0.047% 100.164% ( 4400, 6600 ] 144 0.012% 100.176% ( 6600, 9900 ] 55 0.005% 100.181% ( 9900, 14000 ] 4 0.000% 100.181% ( 14000, 22000 ] 1 0.000% 100.181% ** Level 4 read latency histogram (micros): Count: 445 Average: 308.8472 StdDev: 772.72 Min: 1 Median: 2.7319 Max: 7525 Percentiles: P50: 2.73 P75: 195.58 P99: 3965.00 P99.9: 7525.00 P99.99: 7525.00 ------------------------------------------------------ [ 0, 1 ] 43 9.663% 9.663% ## ( 1, 2 ] 129 28.989% 38.652% ###### ( 2, 3 ] 69 15.506% 54.157% ### ( 3, 4 ] 15 3.371% 57.528% # ( 4, 6 ] 2 0.449% 57.978% ( 6, 10 ] 8 1.798% 59.775% ( 10, 15 ] 7 1.573% 61.348% ( 15, 22 ] 8 1.798% 63.146% ( 22, 34 ] 12 2.697% 65.843% # ( 34, 51 ] 10 2.247% 68.090% ( 51, 76 ] 7 1.573% 69.663% ( 76, 110 ] 8 1.798% 71.461% ( 110, 170 ] 2 0.449% 71.910% ( 170, 250 ] 43 9.663% 81.573% ## ( 250, 380 ] 5 1.124% 82.697% ( 380, 580 ] 7 1.573% 84.270% ( 580, 870 ] 12 2.697% 86.966% # ( 870, 1300 ] 21 4.719% 91.685% # ( 1300, 1900 ] 14 3.146% 94.831% # ( 1900, 2900 ] 15 3.371% 98.202% # ( 2900, 4400 ] 5 1.124% 99.326% ( 4400, 6600 ] 2 0.449% 99.775% ( 6600, 9900 ] 1 0.225% 100.000% ** Level 5 read latency histogram (micros): Count: 300 Average: 1330.5733 StdDev: 1479.80 Min: 74 Median: 694.2424 Max: 6550 Percentiles: P50: 694.24 P75: 2033.33 P99: 6187.50 P99.9: 6550.00 P99.99: 6550.00 ------------------------------------------------------ ( 51, 76 ] 2 0.667% 0.667% ( 76, 110 ] 32 10.667% 11.333% ## ( 110, 170 ] 30 10.000% 21.333% ## ( 170, 250 ] 11 3.667% 25.000% # ( 250, 380 ] 28 9.333% 34.333% ## ( 380, 580 ] 34 11.333% 45.667% ## ( 580, 870 ] 33 11.000% 56.667% ## ( 870, 1300 ] 29 9.667% 66.333% ## ( 1300, 1900 ] 22 7.333% 73.667% # ( 1900, 2900 ] 30 10.000% 83.667% ## ( 2900, 4400 ] 33 11.000% 94.667% ## ( 4400, 6600 ] 16 5.333% 100.000% # ** DB Stats ** Uptime(secs): 54.4 total, 3.0 interval Cumulative writes: 0 writes, 10M keys, 0 commit groups, 0.0 writes per commit group, ingest: 9.61 GB, 180.83 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:27.529 H:M:S, 50.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 326.91 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.430 H:M:S, 14.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/18-00:56:24 ... thread 0: (1000000,11000000) ops and (258135.7,189243.7) ops/second in (3.873932,58.126103) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/14 3.75 GB 125.9 8.1 0.0 8.1 18.6 10.4 0.0 1.8 121.9 278.8 68.21 59.07 175 0.390 8443K 9130 0.0 0.0 L1 21/21 1.82 GB 0.0 9.2 6.5 2.7 9.1 6.3 0.2 1.4 210.1 207.5 44.74 40.09 5 8.948 8786K 103K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 4.6 2.0 249.9 249.8 1.51 1.46 2 0.756 383K 190 0.0 0.0 L3 13/0 968.63 MB 1.0 5.2 3.4 1.9 5.2 3.4 1.5 1.5 146.4 145.7 36.59 24.57 17 2.152 5441K 24K 0.0 0.0 L4 167/0 9.42 GB 1.0 3.7 3.2 0.4 3.6 3.2 1.6 1.1 126.7 126.0 29.62 17.34 41 0.722 3771K 19K 0.0 0.0 L5 1100/0 90.49 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1320/35 106.43 GB 0.0 26.6 13.3 13.3 36.9 23.6 12.9 3.5 150.6 208.9 180.67 142.52 240 0.753 26M 157K 0.0 0.0 Int 0/0 0.00 KB 0.0 26.6 13.3 13.3 36.9 23.6 12.5 3.5 150.6 208.9 180.67 142.52 240 0.753 26M 157K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 26.6 13.3 13.3 26.4 13.1 0.0 0.0 186.6 185.4 145.82 114.93 94 1.551 26M 157K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 10.5 10.5 0.0 0.0 0.0 307.3 34.85 27.58 146 0.239 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 58.3 total, 58.1 interval Flush(GB): cumulative 10.458, interval 10.458 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 36.86 GB write, 647.35 MB/s write, 26.57 GB read, 466.63 MB/s read, 180.7 seconds Interval compaction: 36.86 GB write, 649.27 MB/s write, 26.57 GB read, 468.01 MB/s read, 180.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 17, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 17, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 79, pending-compaction-bytes-delays: 6, pending-compaction-bytes-stops: 0, total-delays: 23, total-stops: 79, interval: 102 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 4063701 Average: 2.8563 StdDev: 28.91 Min: 0 Median: 1.5075 Max: 14183 Percentiles: P50: 1.51 P75: 1.90 P99: 6.23 P99.9: 137.01 P99.99: 408.76 ------------------------------------------------------ [ 0, 1 ] 716475 17.631% 17.631% #### ( 1, 2 ] 2591876 63.781% 81.412% ############# ( 2, 3 ] 436591 10.744% 92.156% ## ( 3, 4 ] 160382 3.947% 96.103% # ( 4, 6 ] 116260 2.861% 98.964% # ( 6, 10 ] 25747 0.634% 99.597% ( 10, 15 ] 4197 0.103% 99.700% ( 15, 22 ] 592 0.015% 99.715% ( 22, 34 ] 71 0.002% 99.717% ( 34, 51 ] 40 0.001% 99.718% ( 51, 76 ] 119 0.003% 99.721% ( 76, 110 ] 5472 0.135% 99.855% ( 110, 170 ] 4035 0.099% 99.955% ( 170, 250 ] 668 0.016% 99.971% ( 250, 380 ] 673 0.017% 99.988% ( 380, 580 ] 679 0.017% 100.004% ( 580, 870 ] 326 0.008% 100.012% ( 870, 1300 ] 169 0.004% 100.017% ( 1300, 1900 ] 88 0.002% 100.019% ( 1900, 2900 ] 83 0.002% 100.021% ( 2900, 4400 ] 68 0.002% 100.022% ( 4400, 6600 ] 15 0.000% 100.023% ( 6600, 9900 ] 6 0.000% 100.023% ( 9900, 14000 ] 2 0.000% 100.023% ( 14000, 22000 ] 1 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 1814835 Average: 5.5385 StdDev: 90.73 Min: 0 Median: 1.6943 Max: 13676 Percentiles: P50: 1.69 P75: 2.47 P99: 9.10 P99.9: 192.43 P99.99: 658.03 ------------------------------------------------------ [ 0, 1 ] 211173 11.636% 11.636% ## ( 1, 2 ] 1002799 55.256% 66.892% ########### ( 2, 3 ] 310244 17.095% 83.986% ### ( 3, 4 ] 129587 7.140% 91.127% # ( 4, 6 ] 118667 6.539% 97.666% # ( 6, 10 ] 31242 1.721% 99.387% ( 10, 15 ] 4502 0.248% 99.635% ( 15, 22 ] 665 0.037% 99.672% ( 22, 34 ] 59 0.003% 99.675% ( 34, 51 ] 27 0.001% 99.677% ( 51, 76 ] 116 0.006% 99.683% ( 76, 110 ] 2245 0.124% 99.807% ( 110, 170 ] 1561 0.086% 99.893% ( 170, 250 ] 475 0.026% 99.919% ( 250, 380 ] 519 0.029% 99.947% ( 380, 580 ] 617 0.034% 99.981% ( 580, 870 ] 578 0.032% 100.013% ( 870, 1300 ] 451 0.025% 100.038% ( 1300, 1900 ] 285 0.016% 100.054% ( 1900, 2900 ] 352 0.019% 100.073% ( 2900, 4400 ] 392 0.022% 100.095% ( 4400, 6600 ] 101 0.006% 100.100% ( 6600, 9900 ] 40 0.002% 100.103% ( 9900, 14000 ] 10 0.001% 100.103% ** Level 2 read latency histogram (micros): Count: 124205 Average: 4.0516 StdDev: 60.70 Min: 0 Median: 1.5190 Max: 5285 Percentiles: P50: 1.52 P75: 1.94 P99: 7.57 P99.9: 551.14 P99.99: 3543.79 ------------------------------------------------------ [ 0, 1 ] 24210 19.492% 19.492% #### ( 1, 2 ] 73015 58.786% 78.278% ############ ( 2, 3 ] 14337 11.543% 89.821% ## ( 3, 4 ] 5980 4.815% 94.635% # ( 4, 6 ] 4933 3.972% 98.607% # ( 6, 10 ] 1244 1.002% 99.609% ( 10, 15 ] 173 0.139% 99.748% ( 15, 22 ] 36 0.029% 99.777% ( 22, 34 ] 3 0.002% 99.779% ( 34, 51 ] 3 0.002% 99.782% ( 76, 110 ] 6 0.005% 99.787% ( 110, 170 ] 27 0.022% 99.808% ( 170, 250 ] 30 0.024% 99.833% ( 250, 380 ] 47 0.038% 99.870% ( 380, 580 ] 43 0.035% 99.905% ( 580, 870 ] 49 0.039% 99.944% ( 870, 1300 ] 26 0.021% 99.965% ( 1300, 1900 ] 13 0.010% 99.976% ( 1900, 2900 ] 12 0.010% 99.986% ( 2900, 4400 ] 13 0.010% 99.996% ( 4400, 6600 ] 5 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 1206194 Average: 9.3119 StdDev: 131.74 Min: 0 Median: 1.9880 Max: 14754 Percentiles: P50: 1.99 P75: 2.98 P99: 9.28 P99.9: 695.37 P99.99: 1068.68 ------------------------------------------------------ [ 0, 1 ] 18509 1.534% 1.534% ( 1, 2 ] 591687 49.054% 50.589% ########## ( 2, 3 ] 301695 25.012% 75.601% ##### ( 3, 4 ] 136465 11.314% 86.914% ## ( 4, 6 ] 127053 10.533% 97.448% ## ( 6, 10 ] 22832 1.893% 99.341% ( 10, 15 ] 3746 0.311% 99.651% ( 15, 22 ] 373 0.031% 99.682% ( 22, 34 ] 62 0.005% 99.687% ( 34, 51 ] 24 0.002% 99.689% ( 51, 76 ] 24 0.002% 99.691% ( 76, 110 ] 94 0.008% 99.699% ( 110, 170 ] 247 0.020% 99.720% ( 170, 250 ] 322 0.027% 99.746% ( 250, 380 ] 523 0.043% 99.790% ( 380, 580 ] 893 0.074% 99.864% ( 580, 870 ] 1103 0.091% 99.955% ( 870, 1300 ] 912 0.076% 100.031% ( 1300, 1900 ] 541 0.045% 100.076% ( 1900, 2900 ] 498 0.041% 100.117% ( 2900, 4400 ] 571 0.047% 100.164% ( 4400, 6600 ] 144 0.012% 100.176% ( 6600, 9900 ] 55 0.005% 100.181% ( 9900, 14000 ] 4 0.000% 100.181% ( 14000, 22000 ] 1 0.000% 100.181% ** Level 4 read latency histogram (micros): Count: 445 Average: 308.8472 StdDev: 772.72 Min: 1 Median: 2.7319 Max: 7525 Percentiles: P50: 2.73 P75: 195.58 P99: 3965.00 P99.9: 7525.00 P99.99: 7525.00 ------------------------------------------------------ [ 0, 1 ] 43 9.663% 9.663% ## ( 1, 2 ] 129 28.989% 38.652% ###### ( 2, 3 ] 69 15.506% 54.157% ### ( 3, 4 ] 15 3.371% 57.528% # ( 4, 6 ] 2 0.449% 57.978% ( 6, 10 ] 8 1.798% 59.775% ( 10, 15 ] 7 1.573% 61.348% ( 15, 22 ] 8 1.798% 63.146% ( 22, 34 ] 12 2.697% 65.843% # ( 34, 51 ] 10 2.247% 68.090% ( 51, 76 ] 7 1.573% 69.663% ( 76, 110 ] 8 1.798% 71.461% ( 110, 170 ] 2 0.449% 71.910% ( 170, 250 ] 43 9.663% 81.573% ## ( 250, 380 ] 5 1.124% 82.697% ( 380, 580 ] 7 1.573% 84.270% ( 580, 870 ] 12 2.697% 86.966% # ( 870, 1300 ] 21 4.719% 91.685% # ( 1300, 1900 ] 14 3.146% 94.831% # ( 1900, 2900 ] 15 3.371% 98.202% # ( 2900, 4400 ] 5 1.124% 99.326% ( 4400, 6600 ] 2 0.449% 99.775% ( 6600, 9900 ] 1 0.225% 100.000% ** Level 5 read latency histogram (micros): Count: 300 Average: 1330.5733 StdDev: 1479.80 Min: 74 Median: 694.2424 Max: 6550 Percentiles: P50: 694.24 P75: 2033.33 P99: 6187.50 P99.9: 6550.00 P99.99: 6550.00 ------------------------------------------------------ ( 51, 76 ] 2 0.667% 0.667% ( 76, 110 ] 32 10.667% 11.333% ## ( 110, 170 ] 30 10.000% 21.333% ## ( 170, 250 ] 11 3.667% 25.000% # ( 250, 380 ] 28 9.333% 34.333% ## ( 380, 580 ] 34 11.333% 45.667% ## ( 580, 870 ] 33 11.000% 56.667% ## ( 870, 1300 ] 29 9.667% 66.333% ## ( 1300, 1900 ] 22 7.333% 73.667% # ( 1900, 2900 ] 30 10.000% 83.667% ## ( 2900, 4400 ] 33 11.000% 94.667% ## ( 4400, 6600 ] 16 5.333% 100.000% # ** DB Stats ** Uptime(secs): 58.3 total, 3.9 interval Cumulative writes: 0 writes, 11M keys, 0 commit groups, 0.0 writes per commit group, ingest: 10.57 GB, 185.69 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:28.748 H:M:S, 49.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 254.05 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.219 H:M:S, 31.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 5.354 micros/op 186789 ops/sec 60.073 seconds 11220999 operations; 181.0 MB/s Microseconds per write: Count: 11220999 Average: 5.3536 StdDev: 570.79 Min: 0 Median: 1.7160 Max: 799878 Percentiles: P50: 1.72 P75: 2.68 P99: 6.42 P99.9: 1045.02 P99.99: 2735.37 ------------------------------------------------------ [ 0, 1 ] 1623791 14.471% 14.471% ### ( 1, 2 ] 5568207 49.623% 64.094% ########## ( 2, 3 ] 1797107 16.016% 80.110% ### ( 3, 4 ] 1544788 13.767% 93.877% ### ( 4, 6 ] 567705 5.059% 98.936% # ( 6, 10 ] 68322 0.609% 99.545% ( 10, 15 ] 16765 0.149% 99.694% ( 15, 22 ] 6404 0.057% 99.751% ( 22, 34 ] 1620 0.014% 99.766% ( 34, 51 ] 197 0.002% 99.767% ( 51, 76 ] 50 0.000% 99.768% ( 76, 110 ] 7751 0.069% 99.837% ( 110, 170 ] 3204 0.029% 99.866% ( 170, 250 ] 347 0.003% 99.869% ( 250, 380 ] 71 0.001% 99.869% ( 380, 580 ] 28 0.000% 99.870% ( 580, 870 ] 39 0.000% 99.870% ( 870, 1300 ] 8309 0.074% 99.944% ( 1300, 1900 ] 21 0.000% 99.944% ( 1900, 2900 ] 6166 0.055% 99.999% ( 2900, 4400 ] 52 0.000% 100.000% ( 4400, 6600 ] 12 0.000% 100.000% ( 6600, 9900 ] 6 0.000% 100.000% ( 9900, 14000 ] 2 0.000% 100.000% ( 14000, 22000 ] 4 0.000% 100.000% ( 22000, 33000 ] 1 0.000% 100.000% ( 50000, 75000 ] 3 0.000% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% ( 170000, 250000 ] 21 0.000% 100.000% ( 570000, 860000 ] 5 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 197613 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 11220999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 11580070968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 569 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 30058258 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 30669433902 rocksdb.compact.write.bytes COUNT : 30469935486 rocksdb.flush.write.bytes COUNT : 11560021212 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 67 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 321 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 11466871808 rocksdb.memtable.garbage.bytes.at.flush COUNT : 4489216 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 30778128878 rocksdb.non.last.level.read.count COUNT : 7431333 rocksdb.block.checksum.compute.count COUNT : 7430262 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.359114 P95 : 3.799018 P99 : 5.827743 P100 : 799876.000000 COUNT : 11220999 SUM : 55093631 rocksdb.compaction.times.micros P50 : 989200.000000 P95 : 5216666.666667 P99 : 14400000.000000 P100 : 21464996.000000 COUNT : 95 SUM : 146772900 rocksdb.compaction.times.cpu_micros P50 : 896060.606061 P95 : 3687500.000000 P99 : 14400000.000000 P100 : 19398697.000000 COUNT : 95 SUM : 115869319 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 713.967391 P95 : 15800.000000 P99 : 209200.000000 P100 : 262431.000000 COUNT : 151 SUM : 813320 rocksdb.compaction.outfile.sync.micros P50 : 1253.090909 P95 : 128900.000000 P99 : 182266.666667 P100 : 500942.000000 COUNT : 354 SUM : 8325381 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 103.445783 P95 : 1724.285714 P99 : 3945.000000 P100 : 4580.000000 COUNT : 282 SUM : 109167 rocksdb.table.open.io.micros P50 : 1631.168831 P95 : 8786.250000 P99 : 13797.928571 P100 : 21180.000000 COUNT : 569 SUM : 1372369 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.696555 P95 : 7.874363 P99 : 11.062125 P100 : 14762.000000 COUNT : 7428724 SUM : 49893330 rocksdb.read.block.get.micros P50 : 512.075472 P95 : 3711.904762 P99 : 6228.200000 P100 : 16699.000000 COUNT : 1138 SUM : 1116758 rocksdb.write.raw.block.micros P50 : 0.513649 P95 : 0.975932 P99 : 2.821612 P100 : 16441.000000 COUNT : 10179876 SUM : 22257104 rocksdb.numfiles.in.singlecompaction P50 : 3.177632 P95 : 5.300000 P99 : 25.060000 P100 : 31.000000 COUNT : 149 SUM : 557 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.557923 P95 : 1258.078110 P99 : 2678.890339 P100 : 799687.000000 COUNT : 139696 SUM : 30064778 rocksdb.sst.read.micros P50 : 1.615303 P95 : 4.679430 P99 : 8.439637 P100 : 14754.000000 COUNT : 7431701 SUM : 34598366 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 11220999 SUM : 11580070968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 215656.565657 P95 : 473100.000000 P99 : 893104.000000 P100 : 893104.000000 COUNT : 151 SUM : 35726819 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 347 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0