[2024-10-14 19:58:23.289424] Starting SPDK v25.01-pre git sha1 3a02df0b1 / DPDK 22.11.4 initialization... [2024-10-14 19:58:23.289571] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3790353 ] Set seed to 1728928703285855 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-10-14 19:58:23.379310] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-10-14 19:58:23.402141] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-10-14 19:58:23.402187] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3790353' to capture a snapshot of events at runtime. [2024-10-14 19:58:23.402198] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-10-14 19:58:23.402208] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-10-14 19:58:23.402216] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3790353 for offline analysis/debug. [2024-10-14 19:58:23.402751] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Oct 14 19:58:27 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/10/14-19:58:30 ... thread 0: (1000000,1000000) ops and (335717.8,335717.8) ops/second in (2.978692,2.978692) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/9 692.98 MB 12.6 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 352.5 2.50 2.26 12 0.208 0 0 0.0 0.0 L2 3/2 237.07 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.0 1.8 256.6 256.4 1.36 1.31 2 0.679 353K 196 0.0 0.0 L3 8/0 692.94 MB 0.7 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.88 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1181/11 97.00 GB 0.0 0.3 0.2 0.2 1.2 1.0 0.0 1.4 90.3 318.7 3.86 3.57 14 0.276 353K 196 0.0 0.0 Int 0/0 0.00 KB 0.0 0.3 0.2 0.2 1.2 1.0 0.0 1.4 90.3 318.7 3.86 3.57 14 0.276 353K 196 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 0.0 0.0 256.6 256.4 1.36 1.31 2 0.679 353K 196 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 352.5 2.50 2.26 12 0.208 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.1 total, 3.0 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.20 GB write, 391.50 MB/s write, 0.34 GB read, 110.91 MB/s read, 3.9 seconds Interval compaction: 1.20 GB write, 412.78 MB/s write, 0.34 GB read, 116.94 MB/s read, 3.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 3, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 3, interval: 3 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 148044 Average: 2.0404 StdDev: 14.16 Min: 0 Median: 1.3727 Max: 3088 Percentiles: P50: 1.37 P75: 1.78 P99: 5.32 P99.9: 9.66 P99.99: 18.03 ------------------------------------------------------ [ 0, 1 ] 40374 27.272% 27.272% ##### ( 1, 2 ] 90275 60.978% 88.250% ############ ( 2, 3 ] 10192 6.884% 95.135% # ( 3, 4 ] 4054 2.738% 97.873% # ( 4, 6 ] 2535 1.712% 99.585% ( 6, 10 ] 510 0.344% 99.930% ( 10, 15 ] 85 0.057% 99.987% ( 15, 22 ] 12 0.008% 99.995% ( 22, 34 ] 2 0.001% 99.997% ( 51, 76 ] 1 0.001% 99.997% ( 110, 170 ] 7 0.005% 100.002% ( 170, 250 ] 2 0.001% 100.003% ( 250, 380 ] 8 0.005% 100.009% ( 380, 580 ] 1 0.001% 100.009% ( 580, 870 ] 2 0.001% 100.011% ( 870, 1300 ] 1 0.001% 100.011% ( 1900, 2900 ] 1 0.001% 100.012% ( 2900, 4400 ] 2 0.001% 100.014% ** Level 2 read latency histogram (micros): Count: 80777 Average: 2.1920 StdDev: 15.87 Min: 0 Median: 1.3825 Max: 2750 Percentiles: P50: 1.38 P75: 1.79 P99: 5.62 P99.9: 36.69 P99.99: 536.89 ------------------------------------------------------ [ 0, 1 ] 21270 26.332% 26.332% ##### ( 1, 2 ] 49986 61.881% 88.213% ############ ( 2, 3 ] 5452 6.749% 94.963% # ( 3, 4 ] 2104 2.605% 97.567% # ( 4, 6 ] 1428 1.768% 99.335% ( 6, 10 ] 386 0.478% 99.813% ( 10, 15 ] 51 0.063% 99.876% ( 15, 22 ] 11 0.014% 99.890% ( 22, 34 ] 3 0.004% 99.894% ( 34, 51 ] 33 0.041% 99.934% ( 51, 76 ] 18 0.022% 99.957% ( 76, 110 ] 5 0.006% 99.963% ( 110, 170 ] 5 0.006% 99.969% ( 170, 250 ] 9 0.011% 99.980% ( 250, 380 ] 4 0.005% 99.985% ( 380, 580 ] 5 0.006% 99.991% ( 580, 870 ] 1 0.001% 99.993% ( 870, 1300 ] 3 0.004% 99.996% ( 1300, 1900 ] 2 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 40 Average: 1635.4000 StdDev: 1280.98 Min: 68 Median: 1192.5000 Max: 4945 Percentiles: P50: 1192.50 P75: 2400.00 P99: 4945.00 P99.9: 4945.00 P99.99: 4945.00 ------------------------------------------------------ ( 51, 76 ] 1 2.500% 2.500% # ( 170, 250 ] 1 2.500% 5.000% # ( 250, 380 ] 2 5.000% 10.000% # ( 380, 580 ] 4 10.000% 20.000% ## ( 580, 870 ] 6 15.000% 35.000% ### ( 870, 1300 ] 8 20.000% 55.000% #### ( 1300, 1900 ] 5 12.500% 67.500% ### ( 1900, 2900 ] 6 15.000% 82.500% ### ( 2900, 4400 ] 5 12.500% 95.000% ### ( 4400, 6600 ] 2 5.000% 100.000% # ** Level 4 read latency histogram (micros): Count: 35 Average: 1726.2000 StdDev: 1407.97 Min: 135 Median: 1165.6250 Max: 5081 Percentiles: P50: 1165.62 P75: 2650.00 P99: 5081.00 P99.9: 5081.00 P99.99: 5081.00 ------------------------------------------------------ ( 110, 170 ] 2 5.714% 5.714% # ( 170, 250 ] 2 5.714% 11.429% # ( 250, 380 ] 1 2.857% 14.286% # ( 380, 580 ] 1 2.857% 17.143% # ( 580, 870 ] 6 17.143% 34.286% ### ( 870, 1300 ] 8 22.857% 57.143% ##### ( 1300, 1900 ] 1 2.857% 60.000% # ( 1900, 2900 ] 7 20.000% 80.000% #### ( 2900, 4400 ] 3 8.571% 88.571% ## ( 4400, 6600 ] 4 11.429% 100.000% ## ** DB Stats ** Uptime(secs): 3.1 total, 3.0 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 313.17 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.377 H:M:S, 12.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 330.15 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.377 H:M:S, 12.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/10/14-19:58:33 ... thread 0: (1000000,2000000) ops and (357496.5,346265.1) ops/second in (2.797230,5.775922) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/9 1.29 GB 50.4 0.5 0.0 0.5 2.3 1.8 0.0 1.3 71.4 330.3 7.06 6.49 28 0.252 511K 475 0.0 0.0 L1 4/4 314.59 MB 0.0 0.3 0.3 0.0 0.3 0.3 0.2 1.0 244.7 244.4 1.29 1.25 1 1.287 319K 384 0.0 0.0 L2 1/0 49.68 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.0 1.8 256.6 256.4 1.36 1.31 2 0.679 353K 196 0.0 0.0 L3 17/0 880.33 MB 0.9 0.2 0.2 0.0 0.2 0.2 0.0 1.0 224.0 224.0 0.84 0.76 1 0.836 190K 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.88 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1195/13 97.92 GB 0.0 1.3 0.7 0.6 3.1 2.5 0.2 1.7 128.6 301.8 10.54 9.81 32 0.329 1375K 1055 0.0 0.0 Int 0/0 0.00 KB 0.0 1.3 0.7 0.6 3.1 2.5 0.2 1.7 128.6 301.8 10.54 9.81 32 0.329 1375K 1055 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.3 0.7 0.6 1.3 0.7 0.0 0.0 251.2 251.0 5.39 5.20 6 0.899 1375K 1055 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 355.2 5.14 4.61 26 0.198 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 5.9 total, 5.8 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.11 GB write, 535.50 MB/s write, 1.32 GB read, 228.12 MB/s read, 10.5 seconds Interval compaction: 3.11 GB write, 550.52 MB/s write, 1.32 GB read, 234.51 MB/s read, 10.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 6, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 6, interval: 6 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 374472 Average: 2.0561 StdDev: 12.34 Min: 0 Median: 1.4111 Max: 3169 Percentiles: P50: 1.41 P75: 1.80 P99: 5.13 P99.9: 9.65 P99.99: 14.76 ------------------------------------------------------ [ 0, 1 ] 88271 23.572% 23.572% ##### ( 1, 2 ] 240756 64.292% 87.864% ############# ( 2, 3 ] 29245 7.810% 95.674% ## ( 3, 4 ] 9492 2.535% 98.209% # ( 4, 6 ] 5230 1.397% 99.605% ( 6, 10 ] 1210 0.323% 99.928% ( 10, 15 ] 243 0.065% 99.993% ( 15, 22 ] 23 0.006% 99.999% ( 22, 34 ] 4 0.001% 100.001% ( 34, 51 ] 2 0.001% 100.001% ( 51, 76 ] 3 0.001% 100.002% ( 110, 170 ] 17 0.005% 100.006% ( 170, 250 ] 8 0.002% 100.009% ( 250, 380 ] 13 0.003% 100.012% ( 380, 580 ] 3 0.001% 100.013% ( 580, 870 ] 6 0.002% 100.014% ( 870, 1300 ] 1 0.000% 100.015% ( 1300, 1900 ] 3 0.001% 100.015% ( 1900, 2900 ] 2 0.001% 100.016% ( 2900, 4400 ] 3 0.001% 100.017% ** Level 1 read latency histogram (micros): Count: 48706 Average: 2.0062 StdDev: 5.50 Min: 1 Median: 1.4173 Max: 728 Percentiles: P50: 1.42 P75: 1.79 P99: 4.57 P99.9: 9.79 P99.99: 286.71 ------------------------------------------------------ [ 0, 1 ] 10890 22.359% 22.359% #### ( 1, 2 ] 32263 66.240% 88.599% ############# ( 2, 3 ] 3904 8.015% 96.614% ## ( 3, 4 ] 1028 2.111% 98.725% ( 4, 6 ] 471 0.967% 99.692% ( 6, 10 ] 108 0.222% 99.914% ( 10, 15 ] 34 0.070% 99.984% ( 15, 22 ] 2 0.004% 99.988% ( 51, 76 ] 1 0.002% 99.990% ( 250, 380 ] 4 0.008% 99.998% ( 580, 870 ] 2 0.004% 100.002% ** Level 2 read latency histogram (micros): Count: 88116 Average: 2.2018 StdDev: 15.20 Min: 0 Median: 1.4032 Max: 2750 Percentiles: P50: 1.40 P75: 1.80 P99: 5.65 P99.9: 25.54 P99.99: 507.54 ------------------------------------------------------ [ 0, 1 ] 21852 24.799% 24.799% ##### ( 1, 2 ] 55069 62.496% 87.295% ############ ( 2, 3 ] 6450 7.320% 94.615% # ( 3, 4 ] 2478 2.812% 97.427% # ( 4, 6 ] 1678 1.904% 99.332% ( 6, 10 ] 432 0.490% 99.822% ( 10, 15 ] 57 0.065% 99.887% ( 15, 22 ] 11 0.012% 99.899% ( 22, 34 ] 3 0.003% 99.902% ( 34, 51 ] 33 0.037% 99.940% ( 51, 76 ] 18 0.020% 99.960% ( 76, 110 ] 5 0.006% 99.966% ( 110, 170 ] 5 0.006% 99.972% ( 170, 250 ] 9 0.010% 99.982% ( 250, 380 ] 4 0.005% 99.986% ( 380, 580 ] 5 0.006% 99.992% ( 580, 870 ] 1 0.001% 99.993% ( 870, 1300 ] 3 0.003% 99.997% ( 1300, 1900 ] 2 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 85 Average: 788.2824 StdDev: 1192.77 Min: 0 Median: 63.5000 Max: 4945 Percentiles: P50: 63.50 P75: 1144.72 P99: 4945.00 P99.9: 4945.00 P99.99: 4945.00 ------------------------------------------------------ [ 0, 1 ] 24 28.235% 28.235% ###### ( 1, 2 ] 10 11.765% 40.000% ## ( 2, 3 ] 1 1.176% 41.176% ( 3, 4 ] 1 1.176% 42.353% ( 4, 6 ] 3 3.529% 45.882% # ( 6, 10 ] 2 2.353% 48.235% ( 15, 22 ] 1 1.176% 49.412% ( 51, 76 ] 1 1.176% 50.588% ( 110, 170 ] 1 1.176% 51.765% ( 170, 250 ] 1 1.176% 52.941% ( 250, 380 ] 3 3.529% 56.471% # ( 380, 580 ] 4 4.706% 61.176% # ( 580, 870 ] 6 7.059% 68.235% # ( 870, 1300 ] 9 10.588% 78.824% ## ( 1300, 1900 ] 5 5.882% 84.706% # ( 1900, 2900 ] 6 7.059% 91.765% # ( 2900, 4400 ] 5 5.882% 97.647% # ( 4400, 6600 ] 2 2.353% 100.000% ** Level 4 read latency histogram (micros): Count: 35 Average: 1726.2000 StdDev: 1407.97 Min: 135 Median: 1165.6250 Max: 5081 Percentiles: P50: 1165.62 P75: 2650.00 P99: 5081.00 P99.9: 5081.00 P99.99: 5081.00 ------------------------------------------------------ ( 110, 170 ] 2 5.714% 5.714% # ( 170, 250 ] 2 5.714% 11.429% # ( 250, 380 ] 1 2.857% 14.286% # ( 380, 580 ] 1 2.857% 17.143% # ( 580, 870 ] 6 17.143% 34.286% ### ( 870, 1300 ] 8 22.857% 57.143% ##### ( 1300, 1900 ] 1 2.857% 60.000% # ( 1900, 2900 ] 7 20.000% 80.000% #### ( 2900, 4400 ] 3 8.571% 88.571% ## ( 4400, 6600 ] 4 11.429% 100.000% ## ** DB Stats ** Uptime(secs): 5.9 total, 2.8 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 331.38 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.581 H:M:S, 9.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 351.84 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.204 H:M:S, 7.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/14-19:58:36 ... thread 0: (1000000,3000000) ops and (329671.5,340551.3) ops/second in (3.033323,8.809245) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/8 1.72 GB 119.6 1.3 0.0 1.3 4.1 2.8 0.0 1.5 101.8 319.9 12.99 12.05 45 0.289 1343K 1381 0.0 0.0 L1 4/0 363.05 MB 36.3 1.2 0.9 0.3 1.2 0.9 0.2 1.4 237.3 236.6 5.04 4.92 2 2.520 1214K 3414 0.0 0.0 L2 5/4 564.76 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.5 1.8 256.6 256.4 1.36 1.31 2 0.679 353K 196 0.0 0.0 L3 17/1 880.33 MB 0.8 0.2 0.2 0.0 0.2 0.2 0.0 1.0 224.0 224.0 0.84 0.76 1 0.836 190K 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.88 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1199/13 98.90 GB 0.0 3.0 1.2 1.8 5.7 4.0 0.7 2.1 151.0 290.9 20.23 19.04 50 0.405 3102K 4991 0.0 0.0 Int 0/0 0.00 KB 0.0 3.0 1.2 1.8 5.7 4.0 0.7 2.1 151.0 290.9 20.23 19.04 50 0.405 3102K 4991 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.0 1.2 1.8 3.0 1.2 0.0 0.0 249.7 249.3 12.23 11.89 10 1.223 3102K 4991 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 354.7 7.99 7.15 40 0.200 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.0 total, 8.8 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.75 GB write, 655.83 MB/s write, 2.98 GB read, 340.49 MB/s read, 20.2 seconds Interval compaction: 5.75 GB write, 667.89 MB/s write, 2.98 GB read, 346.75 MB/s read, 20.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 10, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 10, interval: 10 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 660884 Average: 2.1116 StdDev: 10.62 Min: 0 Median: 1.4534 Max: 3169 Percentiles: P50: 1.45 P75: 1.83 P99: 5.36 P99.9: 9.93 P99.99: 14.62 ------------------------------------------------------ [ 0, 1 ] 132698 20.079% 20.079% #### ( 1, 2 ] 436165 65.997% 86.076% ############# ( 2, 3 ] 59520 9.006% 95.082% ## ( 3, 4 ] 18638 2.820% 97.902% # ( 4, 6 ] 10656 1.612% 99.515% ( 6, 10 ] 2590 0.392% 99.907% ( 10, 15 ] 597 0.090% 99.997% ( 15, 22 ] 61 0.009% 100.006% ( 22, 34 ] 10 0.002% 100.008% ( 34, 51 ] 5 0.001% 100.008% ( 51, 76 ] 3 0.000% 100.009% ( 76, 110 ] 2 0.000% 100.009% ( 110, 170 ] 20 0.003% 100.012% ( 170, 250 ] 19 0.003% 100.015% ( 250, 380 ] 17 0.003% 100.018% ( 380, 580 ] 5 0.001% 100.018% ( 580, 870 ] 11 0.002% 100.020% ( 870, 1300 ] 7 0.001% 100.021% ( 1300, 1900 ] 6 0.001% 100.022% ( 1900, 2900 ] 2 0.000% 100.022% ( 2900, 4400 ] 3 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 148941 Average: 2.2144 StdDev: 9.72 Min: 1 Median: 1.5252 Max: 1943 Percentiles: P50: 1.53 P75: 1.89 P99: 5.27 P99.9: 10.46 P99.99: 323.75 ------------------------------------------------------ [ 0, 1 ] 20917 14.044% 14.044% ### ( 1, 2 ] 101977 68.468% 82.512% ############## ( 2, 3 ] 18657 12.526% 95.038% ### ( 3, 4 ] 4436 2.978% 98.017% # ( 4, 6 ] 2316 1.555% 99.572% ( 6, 10 ] 478 0.321% 99.893% ( 10, 15 ] 131 0.088% 99.981% ( 15, 22 ] 8 0.005% 99.986% ( 22, 34 ] 1 0.001% 99.987% ( 51, 76 ] 1 0.001% 99.987% ( 250, 380 ] 9 0.006% 99.993% ( 380, 580 ] 3 0.002% 99.995% ( 580, 870 ] 2 0.001% 99.997% ( 870, 1300 ] 4 0.003% 99.999% ( 1300, 1900 ] 1 0.001% 100.000% ( 1900, 2900 ] 1 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 88116 Average: 2.2018 StdDev: 15.20 Min: 0 Median: 1.4032 Max: 2750 Percentiles: P50: 1.40 P75: 1.80 P99: 5.65 P99.9: 25.54 P99.99: 507.54 ------------------------------------------------------ [ 0, 1 ] 21852 24.799% 24.799% ##### ( 1, 2 ] 55069 62.496% 87.295% ############ ( 2, 3 ] 6450 7.320% 94.615% # ( 3, 4 ] 2478 2.812% 97.427% # ( 4, 6 ] 1678 1.904% 99.332% ( 6, 10 ] 432 0.490% 99.822% ( 10, 15 ] 57 0.065% 99.887% ( 15, 22 ] 11 0.012% 99.899% ( 22, 34 ] 3 0.003% 99.902% ( 34, 51 ] 33 0.037% 99.940% ( 51, 76 ] 18 0.020% 99.960% ( 76, 110 ] 5 0.006% 99.966% ( 110, 170 ] 5 0.006% 99.972% ( 170, 250 ] 9 0.010% 99.982% ( 250, 380 ] 4 0.005% 99.986% ( 380, 580 ] 5 0.006% 99.992% ( 580, 870 ] 1 0.001% 99.993% ( 870, 1300 ] 3 0.003% 99.997% ( 1300, 1900 ] 2 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 14931 Average: 6.6627 StdDev: 107.69 Min: 0 Median: 1.5670 Max: 4945 Percentiles: P50: 1.57 P75: 1.93 P99: 5.72 P99.9: 1668.28 P99.99: 4945.00 ------------------------------------------------------ [ 0, 1 ] 1684 11.279% 11.279% ## ( 1, 2 ] 10197 68.294% 79.573% ############## ( 2, 3 ] 2170 14.534% 94.106% ### ( 3, 4 ] 516 3.456% 97.562% # ( 4, 6 ] 250 1.674% 99.236% ( 6, 10 ] 58 0.388% 99.625% ( 10, 15 ] 10 0.067% 99.692% ( 15, 22 ] 2 0.013% 99.705% ( 22, 34 ] 1 0.007% 99.712% ( 51, 76 ] 1 0.007% 99.719% ( 110, 170 ] 1 0.007% 99.725% ( 170, 250 ] 1 0.007% 99.732% ( 250, 380 ] 3 0.020% 99.752% ( 380, 580 ] 4 0.027% 99.779% ( 580, 870 ] 6 0.040% 99.819% ( 870, 1300 ] 9 0.060% 99.879% ( 1300, 1900 ] 5 0.033% 99.913% ( 1900, 2900 ] 6 0.040% 99.953% ( 2900, 4400 ] 5 0.033% 99.987% ( 4400, 6600 ] 2 0.013% 100.000% ** Level 4 read latency histogram (micros): Count: 35 Average: 1726.2000 StdDev: 1407.97 Min: 135 Median: 1165.6250 Max: 5081 Percentiles: P50: 1165.62 P75: 2650.00 P99: 5081.00 P99.9: 5081.00 P99.99: 5081.00 ------------------------------------------------------ ( 110, 170 ] 2 5.714% 5.714% # ( 170, 250 ] 2 5.714% 11.429% # ( 250, 380 ] 1 2.857% 14.286% # ( 380, 580 ] 1 2.857% 17.143% # ( 580, 870 ] 6 17.143% 34.286% ### ( 870, 1300 ] 8 22.857% 57.143% ##### ( 1300, 1900 ] 1 2.857% 60.000% # ( 1900, 2900 ] 7 20.000% 80.000% #### ( 2900, 4400 ] 3 8.571% 88.571% ## ( 4400, 6600 ] 4 11.429% 100.000% ## ** DB Stats ** Uptime(secs): 9.0 total, 3.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 329.04 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.973 H:M:S, 10.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 324.46 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.392 H:M:S, 12.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/10/14-19:58:39 ... thread 0: (1000000,4000000) ops and (328839.7,337545.9) ops/second in (3.040995,11.850240) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/14 2.64 GB 37.8 2.1 0.0 2.1 5.8 3.7 0.0 1.6 114.3 315.8 18.74 17.46 61 0.307 2174K 2280 0.0 0.0 L1 4/4 363.05 MB 0.0 1.2 0.9 0.3 1.2 0.9 0.2 1.4 237.3 236.6 5.04 4.92 2 2.520 1214K 3414 0.0 0.0 L2 1/0 49.68 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.5 1.8 256.6 256.4 1.36 1.31 2 0.679 353K 196 0.0 0.0 L3 36/0 992.85 MB 1.0 0.8 0.7 0.1 0.8 0.7 0.0 1.2 211.0 210.8 3.94 3.55 2 1.970 844K 961 0.0 0.0 L4 136/0 9.62 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1242/18 99.82 GB 0.0 4.4 1.7 2.7 8.1 5.4 1.4 2.2 155.4 285.1 29.08 27.24 67 0.434 4587K 6851 0.0 0.0 Int 0/0 0.00 KB 0.0 4.4 1.7 2.7 8.1 5.4 1.4 2.2 155.4 285.1 29.08 27.24 67 0.434 4587K 6851 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.4 1.7 2.7 4.4 1.7 0.0 0.0 245.3 244.9 18.42 17.70 14 1.316 4587K 6851 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.7 3.7 0.0 0.0 0.0 354.5 10.66 9.54 53 0.201 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.0 total, 11.9 interval Flush(GB): cumulative 3.691, interval 3.691 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.10 GB write, 690.08 MB/s write, 4.41 GB read, 376.06 MB/s read, 29.1 seconds Interval compaction: 8.10 GB write, 699.51 MB/s write, 4.41 GB read, 381.20 MB/s read, 29.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 14, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 14, interval: 14 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 976404 Average: 2.1472 StdDev: 10.04 Min: 0 Median: 1.4746 Max: 3386 Percentiles: P50: 1.47 P75: 1.85 P99: 5.44 P99.9: 10.24 P99.99: 14.89 ------------------------------------------------------ [ 0, 1 ] 179177 18.351% 18.351% #### ( 1, 2 ] 651105 66.684% 85.035% ############# ( 2, 3 ] 95443 9.775% 94.810% ## ( 3, 4 ] 29058 2.976% 97.786% # ( 4, 6 ] 16528 1.693% 99.478% ( 6, 10 ] 4073 0.417% 99.896% ( 10, 15 ] 946 0.097% 99.992% ( 15, 22 ] 74 0.008% 100.000% ( 22, 34 ] 12 0.001% 100.001% ( 34, 51 ] 6 0.001% 100.002% ( 51, 76 ] 4 0.000% 100.002% ( 76, 110 ] 23 0.002% 100.005% ( 110, 170 ] 42 0.004% 100.009% ( 170, 250 ] 31 0.003% 100.012% ( 250, 380 ] 30 0.003% 100.015% ( 380, 580 ] 14 0.001% 100.017% ( 580, 870 ] 20 0.002% 100.019% ( 870, 1300 ] 11 0.001% 100.020% ( 1300, 1900 ] 6 0.001% 100.020% ( 1900, 2900 ] 2 0.000% 100.021% ( 2900, 4400 ] 4 0.000% 100.021% ** Level 1 read latency histogram (micros): Count: 210710 Average: 2.2242 StdDev: 8.19 Min: 1 Median: 1.5475 Max: 1943 Percentiles: P50: 1.55 P75: 1.91 P99: 5.33 P99.9: 10.15 P99.99: 49.79 ------------------------------------------------------ [ 0, 1 ] 26390 12.524% 12.524% ### ( 1, 2 ] 144221 68.445% 80.970% ############## ( 2, 3 ] 28799 13.668% 94.637% ### ( 3, 4 ] 6819 3.236% 97.873% # ( 4, 6 ] 3572 1.695% 99.569% ( 6, 10 ] 693 0.329% 99.897% ( 10, 15 ] 181 0.086% 99.983% ( 15, 22 ] 11 0.005% 99.989% ( 22, 34 ] 2 0.001% 99.990% ( 34, 51 ] 1 0.000% 99.990% ( 51, 76 ] 1 0.000% 99.991% ( 250, 380 ] 9 0.004% 99.995% ( 380, 580 ] 3 0.001% 99.996% ( 580, 870 ] 2 0.001% 99.997% ( 870, 1300 ] 4 0.002% 99.999% ( 1300, 1900 ] 1 0.000% 100.000% ( 1900, 2900 ] 1 0.000% 100.000% ** Level 2 read latency histogram (micros): Count: 88116 Average: 2.2018 StdDev: 15.20 Min: 0 Median: 1.4032 Max: 2750 Percentiles: P50: 1.40 P75: 1.80 P99: 5.65 P99.9: 25.54 P99.99: 507.54 ------------------------------------------------------ [ 0, 1 ] 21852 24.799% 24.799% ##### ( 1, 2 ] 55069 62.496% 87.295% ############ ( 2, 3 ] 6450 7.320% 94.615% # ( 3, 4 ] 2478 2.812% 97.427% # ( 4, 6 ] 1678 1.904% 99.332% ( 6, 10 ] 432 0.490% 99.822% ( 10, 15 ] 57 0.065% 99.887% ( 15, 22 ] 11 0.012% 99.899% ( 22, 34 ] 3 0.003% 99.902% ( 34, 51 ] 33 0.037% 99.940% ( 51, 76 ] 18 0.020% 99.960% ( 76, 110 ] 5 0.006% 99.966% ( 110, 170 ] 5 0.006% 99.972% ( 170, 250 ] 9 0.010% 99.982% ( 250, 380 ] 4 0.005% 99.986% ( 380, 580 ] 5 0.006% 99.992% ( 580, 870 ] 1 0.001% 99.993% ( 870, 1300 ] 3 0.003% 99.997% ( 1300, 1900 ] 2 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 32992 Average: 4.6243 StdDev: 73.08 Min: 0 Median: 1.5852 Max: 4945 Percentiles: P50: 1.59 P75: 1.96 P99: 7.60 P99.9: 621.76 P99.99: 4010.24 ------------------------------------------------------ [ 0, 1 ] 3616 10.960% 10.960% ## ( 1, 2 ] 22008 66.707% 77.667% ############# ( 2, 3 ] 5173 15.680% 93.347% ### ( 3, 4 ] 1141 3.458% 96.805% # ( 4, 6 ] 645 1.955% 98.760% ( 6, 10 ] 198 0.600% 99.360% ( 10, 15 ] 52 0.158% 99.518% ( 15, 22 ] 14 0.042% 99.560% ( 22, 34 ] 9 0.027% 99.588% ( 34, 51 ] 2 0.006% 99.594% ( 51, 76 ] 29 0.088% 99.682% ( 76, 110 ] 42 0.127% 99.809% ( 110, 170 ] 8 0.024% 99.833% ( 170, 250 ] 5 0.015% 99.848% ( 250, 380 ] 7 0.021% 99.870% ( 380, 580 ] 9 0.027% 99.897% ( 580, 870 ] 7 0.021% 99.918% ( 870, 1300 ] 9 0.027% 99.945% ( 1300, 1900 ] 5 0.015% 99.961% ( 1900, 2900 ] 6 0.018% 99.979% ( 2900, 4400 ] 5 0.015% 99.994% ( 4400, 6600 ] 2 0.006% 100.000% ** Level 4 read latency histogram (micros): Count: 35 Average: 1726.2000 StdDev: 1407.97 Min: 135 Median: 1165.6250 Max: 5081 Percentiles: P50: 1165.62 P75: 2650.00 P99: 5081.00 P99.9: 5081.00 P99.99: 5081.00 ------------------------------------------------------ ( 110, 170 ] 2 5.714% 5.714% # ( 170, 250 ] 2 5.714% 11.429% # ( 250, 380 ] 1 2.857% 14.286% # ( 380, 580 ] 1 2.857% 17.143% # ( 580, 870 ] 6 17.143% 34.286% ### ( 870, 1300 ] 8 22.857% 57.143% ##### ( 1300, 1900 ] 1 2.857% 60.000% # ( 1900, 2900 ] 7 20.000% 80.000% #### ( 2900, 4400 ] 3 8.571% 88.571% ## ( 4400, 6600 ] 4 11.429% 100.000% ## ** Level 5 read latency histogram (micros): Count: 20 Average: 736.6500 StdDev: 742.54 Min: 127 Median: 480.0000 Max: 2765 Percentiles: P50: 480.00 P75: 1085.00 P99: 2765.00 P99.9: 2765.00 P99.99: 2765.00 ------------------------------------------------------ ( 110, 170 ] 6 30.000% 30.000% ###### ( 170, 250 ] 2 10.000% 40.000% ## ( 250, 380 ] 1 5.000% 45.000% # ( 380, 580 ] 2 10.000% 55.000% ## ( 580, 870 ] 3 15.000% 70.000% ### ( 870, 1300 ] 2 10.000% 80.000% ## ( 1300, 1900 ] 2 10.000% 90.000% ## ( 1900, 2900 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 12.0 total, 3.0 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 327.67 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.373 H:M:S, 11.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 323.63 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.400 H:M:S, 13.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/14-19:58:43 ... thread 0: (1000000,5000000) ops and (230098.1,308714.1) ops/second in (4.345973,16.196213) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/10 3.63 GB 169.9 3.2 0.0 3.2 7.9 4.7 0.0 1.7 127.1 312.6 25.78 24.12 79 0.326 3326K 3475 0.0 0.0 L1 4/4 363.05 MB 0.0 1.2 0.9 0.3 1.2 0.9 0.2 1.4 237.3 236.6 5.04 4.92 2 2.520 1214K 3414 0.0 0.0 L2 1/0 49.68 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.5 1.8 256.6 256.4 1.36 1.31 2 0.679 353K 196 0.0 0.0 L3 36/0 992.85 MB 1.0 0.8 0.7 0.1 0.8 0.7 0.0 1.2 211.0 210.8 3.94 3.55 2 1.970 844K 961 0.0 0.0 L4 136/0 9.62 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1244/14 100.80 GB 0.0 5.5 1.7 3.8 10.2 6.4 1.4 2.2 156.5 288.8 36.12 33.90 85 0.425 5738K 8046 0.0 0.0 Int 0/0 0.00 KB 0.0 5.5 1.7 3.8 10.2 6.4 1.4 2.2 156.5 288.8 36.12 33.90 85 0.425 5738K 8046 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.5 1.7 3.8 5.5 1.7 0.0 0.0 250.0 249.6 22.61 21.82 18 1.256 5738K 8046 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 354.4 13.51 12.07 67 0.202 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.4 total, 16.2 interval Flush(GB): cumulative 4.676, interval 4.676 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.19 GB write, 637.61 MB/s write, 5.52 GB read, 345.47 MB/s read, 36.1 seconds Interval compaction: 10.19 GB write, 643.98 MB/s write, 5.52 GB read, 348.93 MB/s read, 36.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 4, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 4, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 17, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 4, total-stops: 17, interval: 21 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1453525 Average: 2.1778 StdDev: 10.92 Min: 0 Median: 1.4787 Max: 3386 Percentiles: P50: 1.48 P75: 1.85 P99: 5.50 P99.9: 10.50 P99.99: 21.68 ------------------------------------------------------ [ 0, 1 ] 260577 17.927% 17.927% #### ( 1, 2 ] 973923 67.004% 84.931% ############# ( 2, 3 ] 142832 9.827% 94.758% ## ( 3, 4 ] 43087 2.964% 97.722% # ( 4, 6 ] 24715 1.700% 99.423% ( 6, 10 ] 6804 0.468% 99.891% ( 10, 15 ] 1331 0.092% 99.982% ( 15, 22 ] 116 0.008% 99.990% ( 22, 34 ] 19 0.001% 99.992% ( 34, 51 ] 8 0.001% 99.992% ( 51, 76 ] 17 0.001% 99.993% ( 76, 110 ] 84 0.006% 99.999% ( 110, 170 ] 108 0.007% 100.007% ( 170, 250 ] 73 0.005% 100.012% ( 250, 380 ] 56 0.004% 100.015% ( 380, 580 ] 33 0.002% 100.018% ( 580, 870 ] 35 0.002% 100.020% ( 870, 1300 ] 17 0.001% 100.021% ( 1300, 1900 ] 7 0.000% 100.022% ( 1900, 2900 ] 3 0.000% 100.022% ( 2900, 4400 ] 8 0.001% 100.023% ** Level 1 read latency histogram (micros): Count: 232961 Average: 2.2541 StdDev: 8.64 Min: 1 Median: 1.5371 Max: 1943 Percentiles: P50: 1.54 P75: 1.90 P99: 5.42 P99.9: 11.58 P99.99: 311.15 ------------------------------------------------------ [ 0, 1 ] 31153 13.373% 13.373% ### ( 1, 2 ] 158874 68.198% 81.570% ############## ( 2, 3 ] 30486 13.086% 94.657% ### ( 3, 4 ] 7333 3.148% 97.804% # ( 4, 6 ] 3927 1.686% 99.490% ( 6, 10 ] 893 0.383% 99.873% ( 10, 15 ] 196 0.084% 99.958% ( 15, 22 ] 12 0.005% 99.963% ( 22, 34 ] 2 0.001% 99.964% ( 34, 51 ] 1 0.000% 99.964% ( 51, 76 ] 1 0.000% 99.964% ( 76, 110 ] 18 0.008% 99.972% ( 110, 170 ] 25 0.011% 99.983% ( 170, 250 ] 12 0.005% 99.988% ( 250, 380 ] 10 0.004% 99.992% ( 380, 580 ] 9 0.004% 99.996% ( 580, 870 ] 3 0.001% 99.997% ( 870, 1300 ] 4 0.002% 99.999% ( 1300, 1900 ] 1 0.000% 100.000% ( 1900, 2900 ] 1 0.000% 100.000% ** Level 2 read latency histogram (micros): Count: 88116 Average: 2.2018 StdDev: 15.20 Min: 0 Median: 1.4032 Max: 2750 Percentiles: P50: 1.40 P75: 1.80 P99: 5.65 P99.9: 25.54 P99.99: 507.54 ------------------------------------------------------ [ 0, 1 ] 21852 24.799% 24.799% ##### ( 1, 2 ] 55069 62.496% 87.295% ############ ( 2, 3 ] 6450 7.320% 94.615% # ( 3, 4 ] 2478 2.812% 97.427% # ( 4, 6 ] 1678 1.904% 99.332% ( 6, 10 ] 432 0.490% 99.822% ( 10, 15 ] 57 0.065% 99.887% ( 15, 22 ] 11 0.012% 99.899% ( 22, 34 ] 3 0.003% 99.902% ( 34, 51 ] 33 0.037% 99.940% ( 51, 76 ] 18 0.020% 99.960% ( 76, 110 ] 5 0.006% 99.966% ( 110, 170 ] 5 0.006% 99.972% ( 170, 250 ] 9 0.010% 99.982% ( 250, 380 ] 4 0.005% 99.986% ( 380, 580 ] 5 0.006% 99.992% ( 580, 870 ] 1 0.001% 99.993% ( 870, 1300 ] 3 0.003% 99.997% ( 1300, 1900 ] 2 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 32992 Average: 4.6243 StdDev: 73.08 Min: 0 Median: 1.5852 Max: 4945 Percentiles: P50: 1.59 P75: 1.96 P99: 7.60 P99.9: 621.76 P99.99: 4010.24 ------------------------------------------------------ [ 0, 1 ] 3616 10.960% 10.960% ## ( 1, 2 ] 22008 66.707% 77.667% ############# ( 2, 3 ] 5173 15.680% 93.347% ### ( 3, 4 ] 1141 3.458% 96.805% # ( 4, 6 ] 645 1.955% 98.760% ( 6, 10 ] 198 0.600% 99.360% ( 10, 15 ] 52 0.158% 99.518% ( 15, 22 ] 14 0.042% 99.560% ( 22, 34 ] 9 0.027% 99.588% ( 34, 51 ] 2 0.006% 99.594% ( 51, 76 ] 29 0.088% 99.682% ( 76, 110 ] 42 0.127% 99.809% ( 110, 170 ] 8 0.024% 99.833% ( 170, 250 ] 5 0.015% 99.848% ( 250, 380 ] 7 0.021% 99.870% ( 380, 580 ] 9 0.027% 99.897% ( 580, 870 ] 7 0.021% 99.918% ( 870, 1300 ] 9 0.027% 99.945% ( 1300, 1900 ] 5 0.015% 99.961% ( 1900, 2900 ] 6 0.018% 99.979% ( 2900, 4400 ] 5 0.015% 99.994% ( 4400, 6600 ] 2 0.006% 100.000% ** Level 4 read latency histogram (micros): Count: 35 Average: 1726.2000 StdDev: 1407.97 Min: 135 Median: 1165.6250 Max: 5081 Percentiles: P50: 1165.62 P75: 2650.00 P99: 5081.00 P99.9: 5081.00 P99.99: 5081.00 ------------------------------------------------------ ( 110, 170 ] 2 5.714% 5.714% # ( 170, 250 ] 2 5.714% 11.429% # ( 250, 380 ] 1 2.857% 14.286% # ( 380, 580 ] 1 2.857% 17.143% # ( 580, 870 ] 6 17.143% 34.286% ### ( 870, 1300 ] 8 22.857% 57.143% ##### ( 1300, 1900 ] 1 2.857% 60.000% # ( 1900, 2900 ] 7 20.000% 80.000% #### ( 2900, 4400 ] 3 8.571% 88.571% ## ( 4400, 6600 ] 4 11.429% 100.000% ## ** Level 5 read latency histogram (micros): Count: 20 Average: 736.6500 StdDev: 742.54 Min: 127 Median: 480.0000 Max: 2765 Percentiles: P50: 480.00 P75: 1085.00 P99: 2765.00 P99.9: 2765.00 P99.99: 2765.00 ------------------------------------------------------ ( 110, 170 ] 6 30.000% 30.000% ###### ( 170, 250 ] 2 10.000% 40.000% ## ( 250, 380 ] 1 5.000% 45.000% # ( 380, 580 ] 2 10.000% 55.000% ## ( 580, 870 ] 3 15.000% 70.000% ### ( 870, 1300 ] 2 10.000% 80.000% ## ( 1300, 1900 ] 2 10.000% 90.000% ## ( 1900, 2900 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 16.4 total, 4.3 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 300.79 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.039 H:M:S, 18.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 226.46 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.665 H:M:S, 38.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/10/14-19:58:50 ... thread 0: (1000000,6000000) ops and (139898.2,257022.4) ops/second in (7.148057,23.344270) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/11 2.64 GB 6.3 4.9 0.0 4.9 10.6 5.6 0.0 1.9 138.3 297.0 36.45 33.52 96 0.380 5116K 10K 0.0 0.0 L1 4/4 409.81 MB 0.0 3.5 2.8 0.7 3.5 2.8 0.2 1.2 230.5 228.6 15.50 15.00 3 5.167 3628K 29K 0.0 0.0 L2 2/2 178.46 MB 0.0 0.3 0.2 0.2 0.3 0.2 2.4 1.8 256.6 256.4 1.36 1.31 2 0.679 353K 196 0.0 0.0 L3 26/9 1.69 GB 1.0 2.1 1.6 0.4 2.1 1.6 0.8 1.3 179.2 178.8 11.82 9.53 8 1.478 1974K 1682 0.0 0.0 L4 156/0 9.68 GB 1.0 0.1 0.1 0.0 0.1 0.1 1.3 1.0 126.8 126.8 0.78 0.48 1 0.776 49K 0 0.0 0.0 L5 1060/0 87.17 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1260/26 101.75 GB 0.0 10.9 4.7 6.2 16.5 10.4 6.0 2.9 169.6 256.9 65.91 59.84 110 0.599 11M 41K 0.0 0.0 Int 0/0 0.00 KB 0.0 10.9 4.7 6.2 16.5 10.4 6.0 2.9 169.6 256.9 65.91 59.84 110 0.599 11M 41K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.9 4.7 6.2 10.9 4.7 0.0 0.0 228.7 227.8 48.87 45.14 29 1.685 11M 41K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 340.2 17.04 14.70 81 0.210 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 23.5 total, 23.3 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 16.53 GB write, 720.09 MB/s write, 10.92 GB read, 475.50 MB/s read, 65.9 seconds Interval compaction: 16.53 GB write, 725.09 MB/s write, 10.92 GB read, 478.80 MB/s read, 65.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 8, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 8, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 31, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 8, total-stops: 31, interval: 39 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2137323 Average: 2.4130 StdDev: 16.35 Min: 0 Median: 1.5163 Max: 5100 Percentiles: P50: 1.52 P75: 1.90 P99: 5.94 P99.9: 13.47 P99.99: 204.68 ------------------------------------------------------ [ 0, 1 ] 341711 15.988% 15.988% ### ( 1, 2 ] 1407887 65.872% 81.859% ############# ( 2, 3 ] 237082 11.092% 92.952% ## ( 3, 4 ] 78105 3.654% 96.606% # ( 4, 6 ] 52866 2.473% 99.080% ( 6, 10 ] 15782 0.738% 99.818% ( 10, 15 ] 2525 0.118% 99.936% ( 15, 22 ] 247 0.012% 99.948% ( 22, 34 ] 27 0.001% 99.949% ( 34, 51 ] 11 0.001% 99.949% ( 51, 76 ] 25 0.001% 99.951% ( 76, 110 ] 320 0.015% 99.966% ( 110, 170 ] 424 0.020% 99.985% ( 170, 250 ] 229 0.011% 99.996% ( 250, 380 ] 177 0.008% 100.004% ( 380, 580 ] 159 0.007% 100.012% ( 580, 870 ] 119 0.006% 100.017% ( 870, 1300 ] 55 0.003% 100.020% ( 1300, 1900 ] 24 0.001% 100.021% ( 1900, 2900 ] 12 0.001% 100.022% ( 2900, 4400 ] 18 0.001% 100.023% ( 4400, 6600 ] 2 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 581883 Average: 3.2733 StdDev: 35.06 Min: 1 Median: 1.7140 Max: 5020 Percentiles: P50: 1.71 P75: 2.42 P99: 7.97 P99.9: 16.29 P99.99: 342.27 ------------------------------------------------------ [ 0, 1 ] 43754 7.519% 7.519% ## ( 1, 2 ] 346178 59.493% 67.012% ############ ( 2, 3 ] 111726 19.201% 86.213% #### ( 3, 4 ] 37429 6.432% 92.645% # ( 4, 6 ] 33239 5.712% 98.358% # ( 6, 10 ] 7574 1.302% 99.659% ( 10, 15 ] 1373 0.236% 99.895% ( 15, 22 ] 153 0.026% 99.921% ( 22, 34 ] 14 0.002% 99.924% ( 34, 51 ] 7 0.001% 99.925% ( 51, 76 ] 3 0.001% 99.926% ( 76, 110 ] 60 0.010% 99.936% ( 110, 170 ] 178 0.031% 99.966% ( 170, 250 ] 85 0.015% 99.981% ( 250, 380 ] 73 0.013% 99.994% ( 380, 580 ] 117 0.020% 100.014% ( 580, 870 ] 100 0.017% 100.031% ( 870, 1300 ] 48 0.008% 100.039% ( 1300, 1900 ] 22 0.004% 100.043% ( 1900, 2900 ] 14 0.002% 100.045% ( 2900, 4400 ] 26 0.004% 100.050% ( 4400, 6600 ] 5 0.001% 100.051% ** Level 2 read latency histogram (micros): Count: 88406 Average: 2.9023 StdDev: 39.10 Min: 0 Median: 1.4056 Max: 3606 Percentiles: P50: 1.41 P75: 1.81 P99: 5.74 P99.9: 147.29 P99.99: 2315.94 ------------------------------------------------------ [ 0, 1 ] 21852 24.718% 24.718% ##### ( 1, 2 ] 55102 62.328% 87.046% ############ ( 2, 3 ] 6568 7.429% 94.475% # ( 3, 4 ] 2520 2.850% 97.326% # ( 4, 6 ] 1703 1.926% 99.252% ( 6, 10 ] 438 0.495% 99.748% ( 10, 15 ] 59 0.067% 99.814% ( 15, 22 ] 11 0.012% 99.827% ( 22, 34 ] 3 0.003% 99.830% ( 34, 51 ] 33 0.037% 99.868% ( 51, 76 ] 18 0.020% 99.888% ( 76, 110 ] 5 0.006% 99.894% ( 110, 170 ] 9 0.010% 99.904% ( 170, 250 ] 13 0.015% 99.919% ( 250, 380 ] 13 0.015% 99.933% ( 380, 580 ] 21 0.024% 99.957% ( 580, 870 ] 13 0.015% 99.972% ( 870, 1300 ] 8 0.009% 99.981% ( 1300, 1900 ] 4 0.005% 99.985% ( 1900, 2900 ] 10 0.011% 99.997% ( 2900, 4400 ] 3 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 293913 Average: 8.5482 StdDev: 99.05 Min: 0 Median: 1.8698 Max: 6541 Percentiles: P50: 1.87 P75: 2.80 P99: 10.84 P99.9: 840.82 P99.99: 1268.24 ------------------------------------------------------ [ 0, 1 ] 9906 3.370% 3.370% # ( 1, 2 ] 157566 53.610% 56.980% ########### ( 2, 3 ] 65937 22.434% 79.414% #### ( 3, 4 ] 26476 9.008% 88.422% ## ( 4, 6 ] 24438 8.315% 96.737% ## ( 6, 10 ] 6481 2.205% 98.942% ( 10, 15 ] 1022 0.348% 99.290% ( 15, 22 ] 168 0.057% 99.347% ( 22, 34 ] 20 0.007% 99.354% ( 34, 51 ] 5 0.002% 99.356% ( 51, 76 ] 33 0.011% 99.367% ( 76, 110 ] 99 0.034% 99.401% ( 110, 170 ] 217 0.074% 99.474% ( 170, 250 ] 257 0.087% 99.562% ( 250, 380 ] 340 0.116% 99.677% ( 380, 580 ] 352 0.120% 99.797% ( 580, 870 ] 337 0.115% 99.912% ( 870, 1300 ] 249 0.085% 99.997% ( 1300, 1900 ] 114 0.039% 100.035% ( 1900, 2900 ] 128 0.044% 100.079% ( 2900, 4400 ] 77 0.026% 100.105% ( 4400, 6600 ] 15 0.005% 100.110% ** Level 4 read latency histogram (micros): Count: 60 Average: 1070.0667 StdDev: 1360.64 Min: 1 Median: 628.3333 Max: 5081 Percentiles: P50: 628.33 P75: 1700.00 P99: 5081.00 P99.9: 5081.00 P99.99: 5081.00 ------------------------------------------------------ [ 0, 1 ] 6 10.000% 10.000% ## ( 1, 2 ] 6 10.000% 20.000% ## ( 2, 3 ] 3 5.000% 25.000% # ( 3, 4 ] 1 1.667% 26.667% ( 4, 6 ] 1 1.667% 28.333% ( 6, 10 ] 1 1.667% 30.000% ( 10, 15 ] 1 1.667% 31.667% ( 15, 22 ] 1 1.667% 33.333% ( 22, 34 ] 1 1.667% 35.000% ( 34, 51 ] 1 1.667% 36.667% ( 110, 170 ] 3 5.000% 41.667% # ( 170, 250 ] 2 3.333% 45.000% # ( 250, 380 ] 1 1.667% 46.667% ( 380, 580 ] 1 1.667% 48.333% ( 580, 870 ] 6 10.000% 58.333% ## ( 870, 1300 ] 8 13.333% 71.667% ### ( 1300, 1900 ] 3 5.000% 76.667% # ( 1900, 2900 ] 7 11.667% 88.333% ## ( 2900, 4400 ] 3 5.000% 93.333% # ( 4400, 6600 ] 4 6.667% 100.000% # ** Level 5 read latency histogram (micros): Count: 80 Average: 854.2500 StdDev: 996.35 Min: 95 Median: 424.4444 Max: 4073 Percentiles: P50: 424.44 P75: 1115.71 P99: 4073.00 P99.9: 4073.00 P99.99: 4073.00 ------------------------------------------------------ ( 76, 110 ] 3 3.750% 3.750% # ( 110, 170 ] 19 23.750% 27.500% ##### ( 170, 250 ] 6 7.500% 35.000% ## ( 250, 380 ] 10 12.500% 47.500% ### ( 380, 580 ] 9 11.250% 58.750% ## ( 580, 870 ] 9 11.250% 70.000% ## ( 870, 1300 ] 7 8.750% 78.750% ## ( 1300, 1900 ] 4 5.000% 83.750% # ( 1900, 2900 ] 9 11.250% 95.000% ## ( 2900, 4400 ] 4 5.000% 100.000% # ** DB Stats ** Uptime(secs): 23.5 total, 7.1 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 251.19 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.181 H:M:S, 30.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 137.69 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.143 H:M:S, 58.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 4 num-running-flushes: 1 2024/10/14-19:58:54 ... thread 0: (1000000,7000000) ops and (308386.2,263287.0) ops/second in (3.242687,26.586957) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/11 3.62 GB 113.3 5.8 0.0 5.8 12.4 6.6 0.0 1.9 138.4 297.3 42.76 39.45 112 0.382 6011K 11K 0.0 0.0 L1 4/4 409.81 MB 0.0 3.5 2.8 0.7 3.5 2.8 0.2 1.2 230.5 228.6 15.50 15.00 3 5.167 3628K 29K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 2.4 1.8 256.6 256.4 1.36 1.31 2 0.679 353K 196 0.0 0.0 L3 16/0 987.68 MB 1.0 3.0 1.8 1.2 3.0 1.8 0.8 1.6 185.2 184.7 16.40 13.26 10 1.640 2906K 3086 0.0 0.0 L4 161/0 9.59 GB 1.0 0.3 0.2 0.0 0.3 0.2 2.1 1.1 166.4 166.1 1.56 1.17 2 0.782 214K 448 0.0 0.0 L5 1072/0 88.15 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1269/15 102.73 GB 0.0 12.8 5.0 7.8 19.4 11.6 7.8 2.9 169.3 256.4 77.59 70.20 129 0.601 13M 44K 0.0 0.0 Int 0/0 0.00 KB 0.0 12.8 5.0 7.8 19.4 11.6 7.8 2.9 169.3 256.4 77.59 70.20 129 0.601 13M 44K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.8 5.0 7.8 12.8 5.0 0.0 0.0 227.9 227.1 57.65 52.91 35 1.647 13M 44K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 341.2 19.94 17.28 94 0.212 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 26.8 total, 26.6 interval Flush(GB): cumulative 6.644, interval 6.644 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 19.43 GB write, 743.70 MB/s write, 12.83 GB read, 491.19 MB/s read, 77.6 seconds Interval compaction: 19.43 GB write, 748.23 MB/s write, 12.83 GB read, 494.18 MB/s read, 77.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 8, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 8, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 35, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 8, total-stops: 35, interval: 43 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2562454 Average: 2.4193 StdDev: 15.73 Min: 0 Median: 1.5314 Max: 5100 Percentiles: P50: 1.53 P75: 1.91 P99: 5.95 P99.9: 12.96 P99.99: 167.38 ------------------------------------------------------ [ 0, 1 ] 384302 14.997% 14.997% ### ( 1, 2 ] 1687959 65.873% 80.870% ############# ( 2, 3 ] 301328 11.759% 92.630% ## ( 3, 4 ] 97221 3.794% 96.424% # ( 4, 6 ] 67682 2.641% 99.065% # ( 6, 10 ] 19680 0.768% 99.833% ( 10, 15 ] 2906 0.113% 99.946% ( 15, 22 ] 275 0.011% 99.957% ( 22, 34 ] 30 0.001% 99.958% ( 34, 51 ] 13 0.001% 99.959% ( 51, 76 ] 26 0.001% 99.960% ( 76, 110 ] 334 0.013% 99.973% ( 110, 170 ] 463 0.018% 99.991% ( 170, 250 ] 258 0.010% 100.001% ( 250, 380 ] 196 0.008% 100.009% ( 380, 580 ] 174 0.007% 100.015% ( 580, 870 ] 131 0.005% 100.020% ( 870, 1300 ] 62 0.002% 100.023% ( 1300, 1900 ] 29 0.001% 100.024% ( 1900, 2900 ] 14 0.001% 100.025% ( 2900, 4400 ] 20 0.001% 100.025% ( 4400, 6600 ] 2 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 581883 Average: 3.2733 StdDev: 35.06 Min: 1 Median: 1.7140 Max: 5020 Percentiles: P50: 1.71 P75: 2.42 P99: 7.97 P99.9: 16.29 P99.99: 342.27 ------------------------------------------------------ [ 0, 1 ] 43754 7.519% 7.519% ## ( 1, 2 ] 346178 59.493% 67.012% ############ ( 2, 3 ] 111726 19.201% 86.213% #### ( 3, 4 ] 37429 6.432% 92.645% # ( 4, 6 ] 33239 5.712% 98.358% # ( 6, 10 ] 7574 1.302% 99.659% ( 10, 15 ] 1373 0.236% 99.895% ( 15, 22 ] 153 0.026% 99.921% ( 22, 34 ] 14 0.002% 99.924% ( 34, 51 ] 7 0.001% 99.925% ( 51, 76 ] 3 0.001% 99.926% ( 76, 110 ] 60 0.010% 99.936% ( 110, 170 ] 178 0.031% 99.966% ( 170, 250 ] 85 0.015% 99.981% ( 250, 380 ] 73 0.013% 99.994% ( 380, 580 ] 117 0.020% 100.014% ( 580, 870 ] 100 0.017% 100.031% ( 870, 1300 ] 48 0.008% 100.039% ( 1300, 1900 ] 22 0.004% 100.043% ( 1900, 2900 ] 14 0.002% 100.045% ( 2900, 4400 ] 26 0.004% 100.050% ( 4400, 6600 ] 5 0.001% 100.051% ** Level 2 read latency histogram (micros): Count: 100732 Average: 2.9127 StdDev: 36.91 Min: 0 Median: 1.4481 Max: 3606 Percentiles: P50: 1.45 P75: 1.85 P99: 5.95 P99.9: 131.79 P99.99: 2192.68 ------------------------------------------------------ [ 0, 1 ] 22360 22.198% 22.198% #### ( 1, 2 ] 62502 62.048% 84.245% ############ ( 2, 3 ] 9031 8.965% 93.211% ## ( 3, 4 ] 3474 3.449% 96.659% # ( 4, 6 ] 2419 2.401% 99.061% ( 6, 10 ] 691 0.686% 99.747% ( 10, 15 ] 78 0.077% 99.824% ( 15, 22 ] 13 0.013% 99.837% ( 22, 34 ] 3 0.003% 99.840% ( 34, 51 ] 33 0.033% 99.873% ( 51, 76 ] 18 0.018% 99.891% ( 76, 110 ] 6 0.006% 99.897% ( 110, 170 ] 9 0.009% 99.906% ( 170, 250 ] 15 0.015% 99.921% ( 250, 380 ] 16 0.016% 99.936% ( 380, 580 ] 23 0.023% 99.959% ( 580, 870 ] 16 0.016% 99.975% ( 870, 1300 ] 8 0.008% 99.983% ( 1300, 1900 ] 4 0.004% 99.987% ( 1900, 2900 ] 10 0.010% 99.997% ( 2900, 4400 ] 3 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 366564 Average: 7.6142 StdDev: 89.46 Min: 0 Median: 1.8483 Max: 6541 Percentiles: P50: 1.85 P75: 2.76 P99: 9.90 P99.9: 783.83 P99.99: 1239.14 ------------------------------------------------------ [ 0, 1 ] 11858 3.235% 3.235% # ( 1, 2 ] 202085 55.130% 58.364% ########### ( 2, 3 ] 80682 22.010% 80.375% #### ( 3, 4 ] 32056 8.745% 89.120% ## ( 4, 6 ] 28600 7.802% 96.922% ## ( 6, 10 ] 7819 2.133% 99.055% ( 10, 15 ] 1118 0.305% 99.360% ( 15, 22 ] 182 0.050% 99.410% ( 22, 34 ] 21 0.006% 99.415% ( 34, 51 ] 5 0.001% 99.417% ( 51, 76 ] 34 0.009% 99.426% ( 76, 110 ] 117 0.032% 99.458% ( 110, 170 ] 282 0.077% 99.535% ( 170, 250 ] 304 0.083% 99.618% ( 250, 380 ] 387 0.106% 99.723% ( 380, 580 ] 393 0.107% 99.831% ( 580, 870 ] 362 0.099% 99.929% ( 870, 1300 ] 259 0.071% 100.000% ( 1300, 1900 ] 117 0.032% 100.032% ( 1900, 2900 ] 129 0.035% 100.067% ( 2900, 4400 ] 77 0.021% 100.088% ( 4400, 6600 ] 15 0.004% 100.092% ** Level 4 read latency histogram (micros): Count: 80 Average: 829.8125 StdDev: 1252.69 Min: 1 Median: 160.0000 Max: 5081 Percentiles: P50: 160.00 P75: 1138.75 P99: 5081.00 P99.9: 5081.00 P99.99: 5081.00 ------------------------------------------------------ [ 0, 1 ] 13 16.250% 16.250% ### ( 1, 2 ] 9 11.250% 27.500% ## ( 2, 3 ] 5 6.250% 33.750% # ( 3, 4 ] 1 1.250% 35.000% ( 4, 6 ] 1 1.250% 36.250% ( 6, 10 ] 1 1.250% 37.500% ( 10, 15 ] 1 1.250% 38.750% ( 15, 22 ] 1 1.250% 40.000% ( 22, 34 ] 2 2.500% 42.500% # ( 34, 51 ] 1 1.250% 43.750% ( 110, 170 ] 6 7.500% 51.250% ## ( 170, 250 ] 2 2.500% 53.750% # ( 250, 380 ] 2 2.500% 56.250% # ( 380, 580 ] 4 5.000% 61.250% # ( 580, 870 ] 6 7.500% 68.750% ## ( 870, 1300 ] 8 10.000% 78.750% ## ( 1300, 1900 ] 3 3.750% 82.500% # ( 1900, 2900 ] 7 8.750% 91.250% ## ( 2900, 4400 ] 3 3.750% 95.000% # ( 4400, 6600 ] 4 5.000% 100.000% # ** Level 5 read latency histogram (micros): Count: 140 Average: 938.0357 StdDev: 1119.78 Min: 80 Median: 468.8889 Max: 5541 Percentiles: P50: 468.89 P75: 1200.77 P99: 5060.00 P99.9: 5541.00 P99.99: 5541.00 ------------------------------------------------------ ( 76, 110 ] 10 7.143% 7.143% # ( 110, 170 ] 25 17.857% 25.000% #### ( 170, 250 ] 10 7.143% 32.143% # ( 250, 380 ] 17 12.143% 44.286% ## ( 380, 580 ] 18 12.857% 57.143% ### ( 580, 870 ] 15 10.714% 67.857% ## ( 870, 1300 ] 13 9.286% 77.143% ## ( 1300, 1900 ] 8 5.714% 82.857% # ( 1900, 2900 ] 14 10.000% 92.857% ## ( 2900, 4400 ] 8 5.714% 98.571% # ( 4400, 6600 ] 2 1.429% 100.000% ** DB Stats ** Uptime(secs): 26.8 total, 3.2 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 257.53 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.799 H:M:S, 29.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 303.50 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.618 H:M:S, 19.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/14-19:58:58 ... thread 0: (1000000,8000000) ops and (217286.9,256499.3) ops/second in (4.602210,31.189167) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/11 4.54 GB 163.6 6.8 0.0 6.8 14.3 7.6 0.0 1.9 140.9 298.3 49.16 45.50 130 0.378 7035K 12K 0.0 0.0 L1 4/4 409.81 MB 0.0 3.5 2.8 0.7 3.5 2.8 0.2 1.2 230.5 228.6 15.50 15.00 3 5.167 3628K 29K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 2.4 1.8 256.6 256.4 1.36 1.31 2 0.679 353K 196 0.0 0.0 L3 16/0 987.68 MB 1.0 3.0 1.8 1.2 3.0 1.8 0.8 1.6 185.2 184.7 16.40 13.26 10 1.640 2906K 3086 0.0 0.0 L4 161/0 9.59 GB 1.0 0.3 0.2 0.0 0.3 0.2 2.1 1.1 166.4 166.1 1.56 1.17 2 0.782 214K 448 0.0 0.0 L5 1072/0 88.15 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1271/15 103.65 GB 0.0 13.8 5.0 8.8 21.3 12.6 7.8 2.8 168.5 260.1 83.98 76.24 147 0.571 14M 45K 0.0 0.0 Int 0/0 0.00 KB 0.0 13.8 5.0 8.8 21.3 12.6 7.8 2.8 168.5 260.1 83.98 76.24 147 0.571 14M 45K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.8 5.0 8.8 13.8 5.0 0.0 0.0 230.6 229.8 61.36 56.56 39 1.573 14M 45K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 342.4 22.63 19.68 108 0.210 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 31.4 total, 31.2 interval Flush(GB): cumulative 7.567, interval 7.567 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 21.33 GB write, 696.79 MB/s write, 13.82 GB read, 451.24 MB/s read, 84.0 seconds Interval compaction: 21.33 GB write, 700.41 MB/s write, 13.82 GB read, 453.58 MB/s read, 84.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 37, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 37, interval: 50 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3067926 Average: 2.3807 StdDev: 14.72 Min: 0 Median: 1.5248 Max: 5100 Percentiles: P50: 1.52 P75: 1.90 P99: 5.94 P99.9: 12.62 P99.99: 158.07 ------------------------------------------------------ [ 0, 1 ] 466020 15.190% 15.190% ### ( 1, 2 ] 2034996 66.331% 81.521% ############# ( 2, 3 ] 351791 11.467% 92.988% ## ( 3, 4 ] 111660 3.640% 96.628% # ( 4, 6 ] 75182 2.451% 99.078% ( 6, 10 ] 23486 0.766% 99.844% ( 10, 15 ] 3287 0.107% 99.951% ( 15, 22 ] 329 0.011% 99.962% ( 22, 34 ] 35 0.001% 99.963% ( 34, 51 ] 13 0.000% 99.963% ( 51, 76 ] 31 0.001% 99.964% ( 76, 110 ] 380 0.012% 99.977% ( 110, 170 ] 512 0.017% 99.993% ( 170, 250 ] 289 0.009% 100.003% ( 250, 380 ] 211 0.007% 100.010% ( 380, 580 ] 187 0.006% 100.016% ( 580, 870 ] 147 0.005% 100.021% ( 870, 1300 ] 64 0.002% 100.023% ( 1300, 1900 ] 29 0.001% 100.024% ( 1900, 2900 ] 15 0.000% 100.024% ( 2900, 4400 ] 21 0.001% 100.025% ( 4400, 6600 ] 2 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 587494 Average: 3.2798 StdDev: 34.95 Min: 1 Median: 1.7129 Max: 5020 Percentiles: P50: 1.71 P75: 2.41 P99: 8.01 P99.9: 18.98 P99.99: 348.39 ------------------------------------------------------ [ 0, 1 ] 44289 7.539% 7.539% ## ( 1, 2 ] 349943 59.565% 67.104% ############ ( 2, 3 ] 112592 19.165% 86.269% #### ( 3, 4 ] 37602 6.400% 92.669% # ( 4, 6 ] 33346 5.676% 98.345% # ( 6, 10 ] 7671 1.306% 99.651% ( 10, 15 ] 1376 0.234% 99.885% ( 15, 22 ] 154 0.026% 99.911% ( 22, 34 ] 15 0.003% 99.914% ( 34, 51 ] 7 0.001% 99.915% ( 51, 76 ] 3 0.001% 99.916% ( 76, 110 ] 89 0.015% 99.931% ( 110, 170 ] 202 0.034% 99.965% ( 170, 250 ] 91 0.015% 99.981% ( 250, 380 ] 73 0.012% 99.993% ( 380, 580 ] 119 0.020% 100.013% ( 580, 870 ] 102 0.017% 100.031% ( 870, 1300 ] 48 0.008% 100.039% ( 1300, 1900 ] 22 0.004% 100.043% ( 1900, 2900 ] 14 0.002% 100.045% ( 2900, 4400 ] 26 0.004% 100.049% ( 4400, 6600 ] 5 0.001% 100.050% ** Level 2 read latency histogram (micros): Count: 100732 Average: 2.9127 StdDev: 36.91 Min: 0 Median: 1.4481 Max: 3606 Percentiles: P50: 1.45 P75: 1.85 P99: 5.95 P99.9: 131.79 P99.99: 2192.68 ------------------------------------------------------ [ 0, 1 ] 22360 22.198% 22.198% #### ( 1, 2 ] 62502 62.048% 84.245% ############ ( 2, 3 ] 9031 8.965% 93.211% ## ( 3, 4 ] 3474 3.449% 96.659% # ( 4, 6 ] 2419 2.401% 99.061% ( 6, 10 ] 691 0.686% 99.747% ( 10, 15 ] 78 0.077% 99.824% ( 15, 22 ] 13 0.013% 99.837% ( 22, 34 ] 3 0.003% 99.840% ( 34, 51 ] 33 0.033% 99.873% ( 51, 76 ] 18 0.018% 99.891% ( 76, 110 ] 6 0.006% 99.897% ( 110, 170 ] 9 0.009% 99.906% ( 170, 250 ] 15 0.015% 99.921% ( 250, 380 ] 16 0.016% 99.936% ( 380, 580 ] 23 0.023% 99.959% ( 580, 870 ] 16 0.016% 99.975% ( 870, 1300 ] 8 0.008% 99.983% ( 1300, 1900 ] 4 0.004% 99.987% ( 1900, 2900 ] 10 0.010% 99.997% ( 2900, 4400 ] 3 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 366564 Average: 7.6142 StdDev: 89.46 Min: 0 Median: 1.8483 Max: 6541 Percentiles: P50: 1.85 P75: 2.76 P99: 9.90 P99.9: 783.83 P99.99: 1239.14 ------------------------------------------------------ [ 0, 1 ] 11858 3.235% 3.235% # ( 1, 2 ] 202085 55.130% 58.364% ########### ( 2, 3 ] 80682 22.010% 80.375% #### ( 3, 4 ] 32056 8.745% 89.120% ## ( 4, 6 ] 28600 7.802% 96.922% ## ( 6, 10 ] 7819 2.133% 99.055% ( 10, 15 ] 1118 0.305% 99.360% ( 15, 22 ] 182 0.050% 99.410% ( 22, 34 ] 21 0.006% 99.415% ( 34, 51 ] 5 0.001% 99.417% ( 51, 76 ] 34 0.009% 99.426% ( 76, 110 ] 117 0.032% 99.458% ( 110, 170 ] 282 0.077% 99.535% ( 170, 250 ] 304 0.083% 99.618% ( 250, 380 ] 387 0.106% 99.723% ( 380, 580 ] 393 0.107% 99.831% ( 580, 870 ] 362 0.099% 99.929% ( 870, 1300 ] 259 0.071% 100.000% ( 1300, 1900 ] 117 0.032% 100.032% ( 1900, 2900 ] 129 0.035% 100.067% ( 2900, 4400 ] 77 0.021% 100.088% ( 4400, 6600 ] 15 0.004% 100.092% ** Level 4 read latency histogram (micros): Count: 80 Average: 829.8125 StdDev: 1252.69 Min: 1 Median: 160.0000 Max: 5081 Percentiles: P50: 160.00 P75: 1138.75 P99: 5081.00 P99.9: 5081.00 P99.99: 5081.00 ------------------------------------------------------ [ 0, 1 ] 13 16.250% 16.250% ### ( 1, 2 ] 9 11.250% 27.500% ## ( 2, 3 ] 5 6.250% 33.750% # ( 3, 4 ] 1 1.250% 35.000% ( 4, 6 ] 1 1.250% 36.250% ( 6, 10 ] 1 1.250% 37.500% ( 10, 15 ] 1 1.250% 38.750% ( 15, 22 ] 1 1.250% 40.000% ( 22, 34 ] 2 2.500% 42.500% # ( 34, 51 ] 1 1.250% 43.750% ( 110, 170 ] 6 7.500% 51.250% ## ( 170, 250 ] 2 2.500% 53.750% # ( 250, 380 ] 2 2.500% 56.250% # ( 380, 580 ] 4 5.000% 61.250% # ( 580, 870 ] 6 7.500% 68.750% ## ( 870, 1300 ] 8 10.000% 78.750% ## ( 1300, 1900 ] 3 3.750% 82.500% # ( 1900, 2900 ] 7 8.750% 91.250% ## ( 2900, 4400 ] 3 3.750% 95.000% # ( 4400, 6600 ] 4 5.000% 100.000% # ** Level 5 read latency histogram (micros): Count: 140 Average: 938.0357 StdDev: 1119.78 Min: 80 Median: 468.8889 Max: 5541 Percentiles: P50: 468.89 P75: 1200.77 P99: 5060.00 P99.9: 5541.00 P99.99: 5541.00 ------------------------------------------------------ ( 76, 110 ] 10 7.143% 7.143% # ( 110, 170 ] 25 17.857% 25.000% #### ( 170, 250 ] 10 7.143% 32.143% # ( 250, 380 ] 17 12.143% 44.286% ## ( 380, 580 ] 18 12.857% 57.143% ### ( 580, 870 ] 15 10.714% 67.857% ## ( 870, 1300 ] 13 9.286% 77.143% ## ( 1300, 1900 ] 8 5.714% 82.857% # ( 1900, 2900 ] 14 10.000% 92.857% ## ( 2900, 4400 ] 8 5.714% 98.571% # ( 4400, 6600 ] 2 1.429% 100.000% ** DB Stats ** Uptime(secs): 31.4 total, 4.6 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 251.12 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.760 H:M:S, 31.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 213.86 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.960 H:M:S, 42.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/14-19:59:07 ... thread 0: (1000000,9000000) ops and (119921.8,227687.1) ops/second in (8.338765,39.527932) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/19 3.20 GB 0.0 7.7 0.0 7.7 16.2 8.5 0.0 1.9 141.6 297.7 55.60 51.39 146 0.381 7993K 14K 0.0 0.0 L1 21/21 2.14 GB 0.0 6.2 5.1 1.1 6.1 5.0 0.2 1.2 228.3 226.2 27.62 26.62 4 6.905 6402K 56K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 2.9 1.8 256.6 256.4 1.36 1.31 2 0.679 353K 196 0.0 0.0 L3 19/0 960.37 MB 1.0 4.3 2.3 2.0 4.3 2.3 0.8 1.8 196.1 195.3 22.29 18.84 14 1.592 4261K 13K 0.0 0.0 L4 166/0 9.74 GB 1.0 0.9 0.7 0.2 0.9 0.7 2.1 1.2 156.2 155.1 5.90 4.25 10 0.590 885K 6219 0.0 0.0 L5 1076/0 88.52 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1301/40 104.53 GB 0.0 19.4 8.3 11.0 27.8 16.7 8.7 3.3 175.7 252.0 112.77 102.42 176 0.641 19M 91K 0.0 0.0 Int 0/0 0.00 KB 0.0 19.4 8.3 11.0 27.8 16.7 8.7 3.3 175.7 252.0 112.77 102.42 176 0.641 19M 91K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 19.4 8.3 11.0 19.3 8.2 0.0 0.0 227.4 226.3 87.16 80.25 54 1.614 19M 91K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.5 8.5 0.0 0.0 0.0 339.4 25.61 22.16 122 0.210 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 39.7 total, 39.5 interval Flush(GB): cumulative 8.490, interval 8.490 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 27.75 GB write, 715.92 MB/s write, 19.35 GB read, 499.31 MB/s read, 112.8 seconds Interval compaction: 27.75 GB write, 718.86 MB/s write, 19.35 GB read, 501.36 MB/s read, 112.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 17, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 17, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 38, pending-compaction-bytes-delays: 7, pending-compaction-bytes-stops: 0, total-delays: 24, total-stops: 38, interval: 62 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3658458 Average: 2.3920 StdDev: 14.49 Min: 0 Median: 1.5264 Max: 5100 Percentiles: P50: 1.53 P75: 1.90 P99: 5.94 P99.9: 13.10 P99.99: 173.04 ------------------------------------------------------ [ 0, 1 ] 547315 14.960% 14.960% ### ( 1, 2 ] 2435055 66.560% 81.520% ############# ( 2, 3 ] 421312 11.516% 93.036% ## ( 3, 4 ] 133513 3.649% 96.685% # ( 4, 6 ] 87436 2.390% 99.075% ( 6, 10 ] 27735 0.758% 99.833% ( 10, 15 ] 3922 0.107% 99.941% ( 15, 22 ] 426 0.012% 99.952% ( 22, 34 ] 40 0.001% 99.953% ( 34, 51 ] 15 0.000% 99.954% ( 51, 76 ] 38 0.001% 99.955% ( 76, 110 ] 537 0.015% 99.970% ( 110, 170 ] 735 0.020% 99.990% ( 170, 250 ] 372 0.010% 100.000% ( 250, 380 ] 281 0.008% 100.007% ( 380, 580 ] 232 0.006% 100.014% ( 580, 870 ] 183 0.005% 100.019% ( 870, 1300 ] 78 0.002% 100.021% ( 1300, 1900 ] 35 0.001% 100.022% ( 1900, 2900 ] 18 0.000% 100.022% ( 2900, 4400 ] 22 0.001% 100.023% ( 4400, 6600 ] 2 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 999735 Average: 3.1302 StdDev: 30.29 Min: 1 Median: 1.6663 Max: 5020 Percentiles: P50: 1.67 P75: 2.23 P99: 8.18 P99.9: 111.09 P99.99: 399.35 ------------------------------------------------------ [ 0, 1 ] 80761 8.078% 8.078% ## ( 1, 2 ] 629025 62.919% 70.997% ############# ( 2, 3 ] 171220 17.127% 88.124% ### ( 3, 4 ] 55583 5.560% 93.684% # ( 4, 6 ] 45357 4.537% 98.221% # ( 6, 10 ] 14284 1.429% 99.649% ( 10, 15 ] 1898 0.190% 99.839% ( 15, 22 ] 247 0.025% 99.864% ( 22, 34 ] 19 0.002% 99.866% ( 34, 51 ] 9 0.001% 99.867% ( 51, 76 ] 16 0.002% 99.868% ( 76, 110 ] 308 0.031% 99.899% ( 110, 170 ] 509 0.051% 99.950% ( 170, 250 ] 211 0.021% 99.971% ( 250, 380 ] 169 0.017% 99.988% ( 380, 580 ] 207 0.021% 100.009% ( 580, 870 ] 146 0.015% 100.023% ( 870, 1300 ] 60 0.006% 100.029% ( 1300, 1900 ] 25 0.003% 100.032% ( 1900, 2900 ] 17 0.002% 100.034% ( 2900, 4400 ] 31 0.003% 100.037% ( 4400, 6600 ] 6 0.001% 100.037% ** Level 2 read latency histogram (micros): Count: 100732 Average: 2.9127 StdDev: 36.91 Min: 0 Median: 1.4481 Max: 3606 Percentiles: P50: 1.45 P75: 1.85 P99: 5.95 P99.9: 131.79 P99.99: 2192.68 ------------------------------------------------------ [ 0, 1 ] 22360 22.198% 22.198% #### ( 1, 2 ] 62502 62.048% 84.245% ############ ( 2, 3 ] 9031 8.965% 93.211% ## ( 3, 4 ] 3474 3.449% 96.659% # ( 4, 6 ] 2419 2.401% 99.061% ( 6, 10 ] 691 0.686% 99.747% ( 10, 15 ] 78 0.077% 99.824% ( 15, 22 ] 13 0.013% 99.837% ( 22, 34 ] 3 0.003% 99.840% ( 34, 51 ] 33 0.033% 99.873% ( 51, 76 ] 18 0.018% 99.891% ( 76, 110 ] 6 0.006% 99.897% ( 110, 170 ] 9 0.009% 99.906% ( 170, 250 ] 15 0.015% 99.921% ( 250, 380 ] 16 0.016% 99.936% ( 380, 580 ] 23 0.023% 99.959% ( 580, 870 ] 16 0.016% 99.975% ( 870, 1300 ] 8 0.008% 99.983% ( 1300, 1900 ] 4 0.004% 99.987% ( 1900, 2900 ] 10 0.010% 99.997% ( 2900, 4400 ] 3 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 609688 Average: 6.2294 StdDev: 76.88 Min: 0 Median: 1.8560 Max: 7298 Percentiles: P50: 1.86 P75: 2.77 P99: 9.38 P99.9: 473.98 P99.99: 779.26 ------------------------------------------------------ [ 0, 1 ] 17842 2.926% 2.926% # ( 1, 2 ] 335264 54.989% 57.916% ########### ( 2, 3 ] 135698 22.257% 80.173% #### ( 3, 4 ] 54549 8.947% 89.120% ## ( 4, 6 ] 50609 8.301% 97.421% ## ( 6, 10 ] 11408 1.871% 99.292% ( 10, 15 ] 1704 0.279% 99.571% ( 15, 22 ] 238 0.039% 99.610% ( 22, 34 ] 25 0.004% 99.614% ( 34, 51 ] 5 0.001% 99.615% ( 51, 76 ] 34 0.006% 99.621% ( 76, 110 ] 156 0.026% 99.646% ( 110, 170 ] 410 0.067% 99.714% ( 170, 250 ] 405 0.066% 99.780% ( 250, 380 ] 494 0.081% 99.861% ( 380, 580 ] 505 0.083% 99.944% ( 580, 870 ] 409 0.067% 100.011% ( 870, 1300 ] 279 0.046% 100.057% ( 1300, 1900 ] 121 0.020% 100.077% ( 1900, 2900 ] 134 0.022% 100.099% ( 2900, 4400 ] 107 0.018% 100.116% ( 4400, 6600 ] 17 0.003% 100.119% ( 6600, 9900 ] 1 0.000% 100.119% ** Level 4 read latency histogram (micros): Count: 160 Average: 518.6125 StdDev: 1095.10 Min: 1 Median: 10.0000 Max: 6213 Percentiles: P50: 10.00 P75: 446.67 P99: 5896.00 P99.9: 6213.00 P99.99: 6213.00 ------------------------------------------------------ [ 0, 1 ] 27 16.875% 16.875% ### ( 1, 2 ] 30 18.750% 35.625% #### ( 2, 3 ] 13 8.125% 43.750% ## ( 3, 4 ] 3 1.875% 45.625% ( 4, 6 ] 4 2.500% 48.125% # ( 6, 10 ] 3 1.875% 50.000% ( 10, 15 ] 1 0.625% 50.625% ( 15, 22 ] 2 1.250% 51.875% ( 22, 34 ] 3 1.875% 53.750% ( 34, 51 ] 2 1.250% 55.000% ( 51, 76 ] 6 3.750% 58.750% # ( 110, 170 ] 7 4.375% 63.125% # ( 170, 250 ] 12 7.500% 70.625% ## ( 250, 380 ] 5 3.125% 73.750% # ( 380, 580 ] 6 3.750% 77.500% # ( 580, 870 ] 9 5.625% 83.125% # ( 870, 1300 ] 8 5.000% 88.125% # ( 1300, 1900 ] 3 1.875% 90.000% ( 1900, 2900 ] 7 4.375% 94.375% # ( 2900, 4400 ] 4 2.500% 96.875% # ( 4400, 6600 ] 5 3.125% 100.000% # ** Level 5 read latency histogram (micros): Count: 160 Average: 936.7188 StdDev: 1115.49 Min: 80 Median: 474.7368 Max: 5541 Percentiles: P50: 474.74 P75: 1177.14 P99: 4840.00 P99.9: 5541.00 P99.99: 5541.00 ------------------------------------------------------ ( 76, 110 ] 11 6.875% 6.875% # ( 110, 170 ] 27 16.875% 23.750% ### ( 170, 250 ] 14 8.750% 32.500% ## ( 250, 380 ] 19 11.875% 44.375% ## ( 380, 580 ] 19 11.875% 56.250% ## ( 580, 870 ] 20 12.500% 68.750% ### ( 870, 1300 ] 14 8.750% 77.500% ## ( 1300, 1900 ] 8 5.000% 82.500% # ( 1900, 2900 ] 16 10.000% 92.500% ## ( 2900, 4400 ] 10 6.250% 98.750% # ( 4400, 6600 ] 2 1.250% 100.000% ** DB Stats ** Uptime(secs): 39.7 total, 8.3 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 223.16 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:15.242 H:M:S, 38.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 118.03 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.482 H:M:S, 65.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 6.326 micros/op 158073 ops/sec 60.117 seconds 9502999 operations; 153.2 MB/s Microseconds per write: Count: 9502999 Average: 6.3260 StdDev: 289.85 Min: 0 Median: 1.7363 Max: 443815 Percentiles: P50: 1.74 P75: 2.77 P99: 9.58 P99.9: 1203.38 P99.99: 2758.52 ------------------------------------------------------ [ 0, 1 ] 1320684 13.898% 13.898% ### ( 1, 2 ] 4659650 49.033% 62.931% ########## ( 2, 3 ] 1497575 15.759% 78.690% ### ( 3, 4 ] 1304432 13.727% 92.417% ### ( 4, 6 ] 518397 5.455% 97.872% # ( 6, 10 ] 119684 1.259% 99.131% ( 10, 15 ] 37486 0.394% 99.526% ( 15, 22 ] 9760 0.103% 99.628% ( 22, 34 ] 2334 0.025% 99.653% ( 34, 51 ] 317 0.003% 99.656% ( 51, 76 ] 53 0.001% 99.657% ( 76, 110 ] 6363 0.067% 99.724% ( 110, 170 ] 2798 0.029% 99.753% ( 170, 250 ] 312 0.003% 99.756% ( 250, 380 ] 168 0.002% 99.758% ( 380, 580 ] 30 0.000% 99.758% ( 580, 870 ] 17 0.000% 99.759% ( 870, 1300 ] 17330 0.182% 99.941% ( 1300, 1900 ] 51 0.001% 99.942% ( 1900, 2900 ] 5367 0.056% 99.998% ( 2900, 4400 ] 123 0.001% 99.999% ( 4400, 6600 ] 31 0.000% 100.000% ( 6600, 9900 ] 13 0.000% 100.000% ( 9900, 14000 ] 2 0.000% 100.000% ( 14000, 22000 ] 3 0.000% 100.000% ( 22000, 33000 ] 3 0.000% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% ( 110000, 170000 ] 2 0.000% 100.000% ( 170000, 250000 ] 12 0.000% 100.000% ( 380000, 570000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 220513 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 9502999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 9807094968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 477 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 33779614 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 31338314300 rocksdb.compact.write.bytes COUNT : 31107988723 rocksdb.flush.write.bytes COUNT : 9776361800 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 43 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 278 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 9697597440 rocksdb.memtable.garbage.bytes.at.flush COUNT : 3834880 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 31427102145 rocksdb.non.last.level.read.count COUNT : 7594073 rocksdb.block.checksum.compute.count COUNT : 7593162 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.388548 P95 : 3.937831 P99 : 8.378239 P100 : 443812.000000 COUNT : 9502999 SUM : 55694516 rocksdb.compaction.times.micros P50 : 1090714.285714 P95 : 5840000.000000 P99 : 22957537.000000 P100 : 22957537.000000 COUNT : 66 SUM : 128019043 rocksdb.compaction.times.cpu_micros P50 : 1035151.515152 P95 : 5840000.000000 P99 : 16720000.000000 P100 : 20862845.000000 COUNT : 66 SUM : 115236184 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 692.133333 P95 : 1281.571429 P99 : 8844.000000 P100 : 14854.000000 COUNT : 132 SUM : 114021 rocksdb.compaction.outfile.sync.micros P50 : 846.194030 P95 : 53333.333333 P99 : 168560.000000 P100 : 223128.000000 COUNT : 312 SUM : 2712467 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 102.714286 P95 : 400.000000 P99 : 2580.000000 P100 : 4021.000000 COUNT : 232 SUM : 43746 rocksdb.table.open.io.micros P50 : 1366.233766 P95 : 6522.083333 P99 : 11555.375000 P100 : 12104.000000 COUNT : 477 SUM : 930576 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.644899 P95 : 7.494340 P99 : 10.546994 P100 : 12295.000000 COUNT : 7592103 SUM : 42998548 rocksdb.read.block.get.micros P50 : 460.733945 P95 : 3240.243902 P99 : 5288.250000 P100 : 6229.000000 COUNT : 954 SUM : 772768 rocksdb.write.raw.block.micros P50 : 0.512054 P95 : 0.972902 P99 : 2.801030 P100 : 13323.000000 COUNT : 9905518 SUM : 18157485 rocksdb.numfiles.in.singlecompaction P50 : 3.313333 P95 : 5.871429 P99 : 14.275000 P100 : 32.000000 COUNT : 129 SUM : 507 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.559452 P95 : 1169.397893 P99 : 2525.681393 P100 : 443560.000000 COUNT : 216161 SUM : 33799274 rocksdb.sst.read.micros P50 : 1.590853 P95 : 4.405604 P99 : 8.244346 P100 : 10059.000000 COUNT : 7594744 SUM : 27333754 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 9502999 SUM : 9807094968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 211702.127660 P95 : 354000.000000 P99 : 539600.000000 P100 : 631232.000000 COUNT : 132 SUM : 27428954 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 300 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0