[2024-07-15 21:20:13.849957] Starting SPDK v24.09-pre git sha1 a95bbf233 / DPDK 23.11.0 initialization... [2024-07-15 21:20:13.850218] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1987549 ] Set seed to 1721071213843524 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-15 21:20:13.958659] app.c: 909:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-15 21:20:14.000464] app.c: 603:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-15 21:20:14.000531] app.c: 604:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1987549' to capture a snapshot of events at runtime. [2024-07-15 21:20:14.000542] app.c: 609:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-15 21:20:14.000552] app.c: 610:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-15 21:20:14.000563] app.c: 611:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1987549 for offline analysis/debug. [2024-07-15 21:20:14.000651] reactor.c: 941:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Jul 15 21:20:18 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/07/15-21:20:26 ... thread 0: (1000000,1000000) ops and (123054.1,123054.1) ops/second in (8.126508,8.126508) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 7/4 881.33 MB 25.2 0.4 0.0 0.4 1.2 0.9 0.0 1.4 40.0 133.1 9.46 4.15 11 0.860 383K 523 0.0 0.0 L1 2/2 121.61 MB 0.0 0.4 0.4 0.1 0.4 0.4 0.0 1.2 103.4 103.2 4.42 2.16 1 4.420 463K 673 0.0 0.0 L2 15/14 1.64 GB 1.1 0.1 0.1 0.1 0.1 0.1 0.3 2.3 106.0 105.8 1.15 0.56 1 1.151 123K 264 0.0 0.0 L3 44/31 3.37 GB 0.9 3.3 2.1 1.2 3.3 2.1 0.0 1.6 101.3 100.2 33.56 15.79 18 1.865 1173K 0 0.0 0.0 L4 181/2 9.60 GB 1.0 1.7 1.3 0.4 1.7 1.3 0.3 1.3 119.6 118.3 14.43 8.17 13 1.110 1066K 66 0.0 0.0 L5 1100/0 90.55 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1349/53 106.15 GB 0.0 5.9 3.8 2.1 6.7 4.6 2.2 7.8 96.5 109.6 63.02 30.82 44 1.432 3211K 1526 0.0 0.0 Int 0/0 0.00 KB 0.0 5.9 3.8 2.1 6.7 4.6 2.2 7.8 96.5 109.6 63.02 30.82 44 1.432 3211K 1526 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.9 3.8 2.1 5.9 3.8 0.0 0.0 108.5 107.5 56.05 28.20 34 1.648 3211K 1526 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 126.5 6.97 2.62 10 0.697 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.3 total, 8.1 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.75 GB write, 829.55 MB/s write, 5.94 GB read, 730.45 MB/s read, 63.0 seconds Interval compaction: 6.75 GB write, 849.72 MB/s write, 5.94 GB read, 748.21 MB/s read, 63.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 10, pending-compaction-bytes-stops: 0, total-delays: 10, total-stops: 26, interval: 36 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 246224 Average: 9.6929 StdDev: 128.20 Min: 1 Median: 1.8864 Max: 15372 Percentiles: P50: 1.89 P75: 2.90 P99: 10.66 P99.9: 1865.00 P99.99: 4838.17 ------------------------------------------------------ [ 0, 1 ] 9530 3.870% 3.870% # ( 1, 2 ] 128146 52.044% 55.915% ########## ( 2, 3 ] 52330 21.253% 77.168% #### ( 3, 4 ] 24171 9.817% 86.985% ## ( 4, 6 ] 23749 9.645% 96.630% ## ( 6, 10 ] 5699 2.315% 98.944% ( 10, 15 ] 1046 0.425% 99.369% ( 15, 22 ] 121 0.049% 99.418% ( 22, 34 ] 25 0.010% 99.429% ( 34, 51 ] 12 0.005% 99.433% ( 51, 76 ] 14 0.006% 99.439% ( 76, 110 ] 11 0.004% 99.444% ( 110, 170 ] 21 0.009% 99.452% ( 170, 250 ] 68 0.028% 99.480% ( 250, 380 ] 128 0.052% 99.532% ( 380, 580 ] 214 0.087% 99.619% ( 580, 870 ] 302 0.123% 99.741% ( 870, 1300 ] 259 0.105% 99.846% ( 1300, 1900 ] 141 0.057% 99.904% ( 1900, 2900 ] 121 0.049% 99.953% ( 2900, 4400 ] 87 0.035% 99.988% ( 4400, 6600 ] 27 0.011% 99.999% ( 6600, 9900 ] 6 0.002% 100.002% ( 9900, 14000 ] 1 0.000% 100.002% ( 14000, 22000 ] 1 0.000% 100.002% ** Level 1 read latency histogram (micros): Count: 74438 Average: 16.9358 StdDev: 171.51 Min: 1 Median: 2.0946 Max: 8400 Percentiles: P50: 2.09 P75: 3.19 P99: 290.01 P99.9: 2740.54 P99.99: 4601.39 ------------------------------------------------------ [ 0, 1 ] 1087 1.460% 1.460% ( 1, 2 ] 34356 46.154% 47.614% ######### ( 2, 3 ] 18767 25.212% 72.826% ##### ( 3, 4 ] 8479 11.391% 84.216% ## ( 4, 6 ] 8340 11.204% 95.420% ## ( 6, 10 ] 2158 2.899% 98.319% # ( 10, 15 ] 375 0.504% 98.823% ( 15, 22 ] 43 0.058% 98.881% ( 22, 34 ] 8 0.011% 98.892% ( 34, 51 ] 10 0.013% 98.905% ( 51, 76 ] 7 0.009% 98.915% ( 76, 110 ] 2 0.003% 98.917% ( 110, 170 ] 14 0.019% 98.936% ( 170, 250 ] 31 0.042% 98.978% ( 250, 380 ] 54 0.073% 99.050% ( 380, 580 ] 112 0.150% 99.201% ( 580, 870 ] 153 0.206% 99.406% ( 870, 1300 ] 181 0.243% 99.649% ( 1300, 1900 ] 121 0.163% 99.812% ( 1900, 2900 ] 78 0.105% 99.917% ( 2900, 4400 ] 53 0.071% 99.988% ( 4400, 6600 ] 17 0.023% 100.011% ( 6600, 9900 ] 1 0.001% 100.012% ** Level 2 read latency histogram (micros): Count: 627047 Average: 15.0922 StdDev: 194.27 Min: 1 Median: 1.9792 Max: 26923 Percentiles: P50: 1.98 P75: 3.03 P99: 12.12 P99.9: 1962.41 P99.99: 3097.57 ------------------------------------------------------ [ 0, 1 ] 15750 2.512% 2.512% # ( 1, 2 ] 304109 48.499% 51.010% ########## ( 2, 3 ] 148540 23.689% 74.699% ##### ( 3, 4 ] 65585 10.459% 85.159% ## ( 4, 6 ] 67504 10.765% 95.924% ## ( 6, 10 ] 17818 2.842% 98.765% # ( 10, 15 ] 3471 0.554% 99.319% ( 15, 22 ] 466 0.074% 99.393% ( 22, 34 ] 76 0.012% 99.405% ( 34, 51 ] 42 0.007% 99.412% ( 51, 76 ] 38 0.006% 99.418% ( 76, 110 ] 19 0.003% 99.421% ( 110, 170 ] 49 0.008% 99.429% ( 170, 250 ] 97 0.015% 99.445% ( 250, 380 ] 219 0.035% 99.479% ( 380, 580 ] 493 0.079% 99.558% ( 580, 870 ] 719 0.115% 99.673% ( 870, 1300 ] 812 0.129% 99.802% ( 1300, 1900 ] 581 0.093% 99.895% ( 1900, 2900 ] 512 0.082% 99.977% ( 2900, 4400 ] 640 0.102% 100.079% ( 4400, 6600 ] 220 0.035% 100.114% ( 6600, 9900 ] 34 0.005% 100.119% ( 9900, 14000 ] 1 0.000% 100.119% ( 14000, 22000 ] 3 0.000% 100.120% ( 22000, 33000 ] 2 0.000% 100.120% ** Level 3 read latency histogram (micros): Count: 883318 Average: 10.8063 StdDev: 144.08 Min: 1 Median: 1.9723 Max: 29017 Percentiles: P50: 1.97 P75: 2.99 P99: 9.77 P99.9: 845.53 P99.99: 1213.17 ------------------------------------------------------ [ 0, 1 ] 15997 1.811% 1.811% ( 1, 2 ] 437802 49.563% 51.374% ########## ( 2, 3 ] 210944 23.881% 75.255% ##### ( 3, 4 ] 93929 10.634% 85.889% ## ( 4, 6 ] 95228 10.781% 96.670% ## ( 6, 10 ] 21812 2.469% 99.139% ( 10, 15 ] 3910 0.443% 99.582% ( 15, 22 ] 478 0.054% 99.636% ( 22, 34 ] 48 0.005% 99.641% ( 34, 51 ] 65 0.007% 99.648% ( 51, 76 ] 52 0.006% 99.654% ( 76, 110 ] 42 0.005% 99.659% ( 110, 170 ] 76 0.009% 99.668% ( 170, 250 ] 146 0.017% 99.684% ( 250, 380 ] 373 0.042% 99.726% ( 380, 580 ] 683 0.077% 99.804% ( 580, 870 ] 928 0.105% 99.909% ( 870, 1300 ] 898 0.102% 100.011% ( 1300, 1900 ] 640 0.072% 100.083% ( 1900, 2900 ] 523 0.059% 100.142% ( 2900, 4400 ] 510 0.058% 100.200% ( 4400, 6600 ] 120 0.014% 100.214% ( 6600, 9900 ] 16 0.002% 100.215% ( 9900, 14000 ] 1 0.000% 100.215% ( 22000, 33000 ] 2 0.000% 100.216% ** Level 4 read latency histogram (micros): Count: 97655 Average: 56.0706 StdDev: 411.29 Min: 1 Median: 1.9709 Max: 31184 Percentiles: P50: 1.97 P75: 3.18 P99: 1755.27 P99.9: 4515.36 P99.99: 6113.35 ------------------------------------------------------ [ 0, 1 ] 2228 2.282% 2.282% ( 1, 2 ] 47994 49.146% 51.428% ########## ( 2, 3 ] 21066 21.572% 73.000% #### ( 3, 4 ] 10653 10.909% 83.909% ## ( 4, 6 ] 9582 9.812% 93.721% ## ( 6, 10 ] 2422 2.480% 96.201% ( 10, 15 ] 384 0.393% 96.594% ( 15, 22 ] 43 0.044% 96.638% ( 22, 34 ] 7 0.007% 96.645% ( 34, 51 ] 3 0.003% 96.648% ( 51, 76 ] 3 0.003% 96.651% ( 76, 110 ] 5 0.005% 96.657% ( 110, 170 ] 46 0.047% 96.704% ( 170, 250 ] 108 0.111% 96.814% ( 250, 380 ] 158 0.162% 96.976% ( 380, 580 ] 406 0.416% 97.392% ( 580, 870 ] 579 0.593% 97.985% ( 870, 1300 ] 650 0.666% 98.650% ( 1300, 1900 ] 450 0.461% 99.111% ( 1900, 2900 ] 427 0.437% 99.548% ( 2900, 4400 ] 337 0.345% 99.894% ( 4400, 6600 ] 121 0.124% 100.017% ( 6600, 9900 ] 13 0.013% 100.031% ( 9900, 14000 ] 4 0.004% 100.035% ( 14000, 22000 ] 4 0.004% 100.039% ( 22000, 33000 ] 3 0.003% 100.042% ** Level 5 read latency histogram (micros): Count: 14529 Average: 1604.5555 StdDev: 1704.68 Min: 84 Median: 1051.2745 Max: 75403 Percentiles: P50: 1051.27 P75: 2150.00 P99: 6563.56 P99.9: 12143.11 P99.99: 37650.35 ------------------------------------------------------ ( 76, 110 ] 25 0.172% 0.172% ( 110, 170 ] 158 1.087% 1.260% ( 170, 250 ] 395 2.719% 3.978% # ( 250, 380 ] 956 6.580% 10.558% # ( 380, 580 ] 1927 13.263% 23.821% ### ( 580, 870 ] 2707 18.632% 42.453% #### ( 870, 1300 ] 2601 17.902% 60.355% #### ( 1300, 1900 ] 1746 12.017% 72.372% ## ( 1900, 2900 ] 1527 10.510% 82.883% ## ( 2900, 4400 ] 1612 11.095% 93.978% ## ( 4400, 6600 ] 742 5.107% 99.085% # ( 6600, 9900 ] 113 0.778% 99.862% ( 9900, 14000 ] 10 0.069% 99.931% ( 14000, 22000 ] 2 0.014% 99.945% ( 22000, 33000 ] 6 0.041% 99.986% ( 33000, 50000 ] 2 0.014% 100.000% ( 75000, 110000 ] 1 0.007% 100.007% ** DB Stats ** Uptime(secs): 8.3 total, 8.1 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 118.19 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.180 H:M:S, 50.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 121.06 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.180 H:M:S, 51.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 8 num-running-flushes: 1 2024/07/15-21:20:33 ... thread 0: (1000000,2000000) ops and (136668.9,129504.6) ops/second in (7.316956,15.443464) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/4 1.23 GB 94.4 1.0 0.0 1.0 2.9 1.8 0.0 1.6 51.9 143.5 20.61 9.87 25 0.825 1087K 1416 0.0 0.0 L1 2/0 214.21 MB 21.4 1.2 1.0 0.2 1.2 1.0 0.0 1.2 122.9 122.6 9.83 5.69 2 4.915 1226K 2631 0.0 0.0 L2 5/3 399.34 MB 1.9 0.1 0.1 0.1 0.1 0.1 0.8 2.3 106.0 105.8 1.15 0.56 1 1.151 123K 264 0.0 0.0 L3 39/23 2.90 GB 1.1 7.5 3.7 3.8 7.4 3.7 0.1 2.0 114.0 113.1 67.30 35.81 24 2.804 3193K 1574 0.0 0.0 L4 189/10 10.11 GB 1.0 4.9 3.5 1.4 4.8 3.4 0.4 1.4 116.4 114.7 42.89 23.18 36 1.191 3364K 1384 0.0 0.0 L5 1120/0 92.21 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1364/40 107.05 GB 0.0 14.7 8.2 6.5 16.4 9.9 4.6 8.9 106.3 118.6 141.78 75.11 88 1.611 8995K 7269 0.0 0.0 Int 0/0 0.00 KB 0.0 14.7 8.2 6.5 16.4 9.9 4.6 8.9 106.3 118.6 141.78 75.11 88 1.611 8995K 7269 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.7 8.2 6.5 14.6 8.1 0.0 0.0 118.1 117.0 127.54 69.61 66 1.932 8995K 7269 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 132.6 14.25 5.50 22 0.648 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 15.6 total, 15.4 interval Flush(GB): cumulative 1.845, interval 1.845 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 16.42 GB write, 1074.98 MB/s write, 14.71 GB read, 963.19 MB/s read, 141.8 seconds Interval compaction: 16.42 GB write, 1088.74 MB/s write, 14.71 GB read, 975.52 MB/s read, 141.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 49, pending-compaction-bytes-delays: 10, pending-compaction-bytes-stops: 0, total-delays: 10, total-stops: 49, interval: 59 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 606857 Average: 9.4574 StdDev: 134.20 Min: 1 Median: 1.8592 Max: 32119 Percentiles: P50: 1.86 P75: 2.84 P99: 9.89 P99.9: 1747.50 P99.99: 4143.41 ------------------------------------------------------ [ 0, 1 ] 24855 4.096% 4.096% # ( 1, 2 ] 324226 53.427% 57.523% ########### ( 2, 3 ] 125975 20.759% 78.281% #### ( 3, 4 ] 57589 9.490% 87.771% ## ( 4, 6 ] 56166 9.255% 97.026% ## ( 6, 10 ] 12323 2.031% 99.057% ( 10, 15 ] 2273 0.375% 99.431% ( 15, 22 ] 221 0.036% 99.468% ( 22, 34 ] 34 0.006% 99.474% ( 34, 51 ] 28 0.005% 99.478% ( 51, 76 ] 45 0.007% 99.486% ( 76, 110 ] 18 0.003% 99.489% ( 110, 170 ] 38 0.006% 99.495% ( 170, 250 ] 113 0.019% 99.513% ( 250, 380 ] 245 0.040% 99.554% ( 380, 580 ] 506 0.083% 99.637% ( 580, 870 ] 719 0.118% 99.756% ( 870, 1300 ] 633 0.104% 99.860% ( 1300, 1900 ] 326 0.054% 99.914% ( 1900, 2900 ] 271 0.045% 99.958% ( 2900, 4400 ] 232 0.038% 99.997% ( 4400, 6600 ] 68 0.011% 100.008% ( 6600, 9900 ] 8 0.001% 100.009% ( 9900, 14000 ] 3 0.000% 100.010% ( 14000, 22000 ] 3 0.000% 100.010% ( 22000, 33000 ] 1 0.000% 100.010% ** Level 1 read latency histogram (micros): Count: 204798 Average: 14.2623 StdDev: 164.18 Min: 1 Median: 2.0214 Max: 13175 Percentiles: P50: 2.02 P75: 3.06 P99: 14.02 P99.9: 2593.60 P99.99: 4274.36 ------------------------------------------------------ [ 0, 1 ] 3182 1.554% 1.554% ( 1, 2 ] 98126 47.914% 49.467% ########## ( 2, 3 ] 50953 24.880% 74.347% ##### ( 3, 4 ] 21685 10.588% 84.935% ## ( 4, 6 ] 22360 10.918% 95.853% ## ( 6, 10 ] 5645 2.756% 98.610% # ( 10, 15 ] 995 0.486% 99.096% ( 15, 22 ] 109 0.053% 99.149% ( 22, 34 ] 13 0.006% 99.155% ( 34, 51 ] 25 0.012% 99.167% ( 51, 76 ] 17 0.008% 99.176% ( 76, 110 ] 8 0.004% 99.180% ( 110, 170 ] 23 0.011% 99.191% ( 170, 250 ] 59 0.029% 99.220% ( 250, 380 ] 102 0.050% 99.270% ( 380, 580 ] 243 0.119% 99.388% ( 580, 870 ] 357 0.174% 99.562% ( 870, 1300 ] 365 0.178% 99.741% ( 1300, 1900 ] 218 0.106% 99.847% ( 1900, 2900 ] 156 0.076% 99.923% ( 2900, 4400 ] 149 0.073% 99.996% ( 4400, 6600 ] 35 0.017% 100.013% ( 6600, 9900 ] 9 0.004% 100.018% ( 9900, 14000 ] 1 0.000% 100.018% ** Level 2 read latency histogram (micros): Count: 718127 Average: 15.1307 StdDev: 197.17 Min: 1 Median: 1.9838 Max: 26923 Percentiles: P50: 1.98 P75: 3.03 P99: 12.13 P99.9: 2180.46 P99.99: 3342.87 ------------------------------------------------------ [ 0, 1 ] 16642 2.317% 2.317% ( 1, 2 ] 348046 48.466% 50.783% ########## ( 2, 3 ] 171357 23.862% 74.645% ##### ( 3, 4 ] 75110 10.459% 85.104% ## ( 4, 6 ] 77756 10.828% 95.932% ## ( 6, 10 ] 20357 2.835% 98.766% # ( 10, 15 ] 3946 0.549% 99.316% ( 15, 22 ] 519 0.072% 99.388% ( 22, 34 ] 83 0.012% 99.400% ( 34, 51 ] 45 0.006% 99.406% ( 51, 76 ] 49 0.007% 99.413% ( 76, 110 ] 21 0.003% 99.416% ( 110, 170 ] 49 0.007% 99.423% ( 170, 250 ] 111 0.015% 99.438% ( 250, 380 ] 249 0.035% 99.473% ( 380, 580 ] 555 0.077% 99.550% ( 580, 870 ] 807 0.112% 99.662% ( 870, 1300 ] 891 0.124% 99.786% ( 1300, 1900 ] 649 0.090% 99.877% ( 1900, 2900 ] 595 0.083% 99.960% ( 2900, 4400 ] 739 0.103% 100.063% ( 4400, 6600 ] 252 0.035% 100.098% ( 6600, 9900 ] 39 0.005% 100.103% ( 9900, 14000 ] 3 0.000% 100.103% ( 14000, 22000 ] 6 0.001% 100.104% ( 22000, 33000 ] 2 0.000% 100.105% ** Level 3 read latency histogram (micros): Count: 2097087 Average: 11.0126 StdDev: 153.24 Min: 1 Median: 1.9735 Max: 36012 Percentiles: P50: 1.97 P75: 2.98 P99: 9.75 P99.9: 778.36 P99.99: 1121.76 ------------------------------------------------------ [ 0, 1 ] 32423 1.546% 1.546% ( 1, 2 ] 1043743 49.771% 51.317% ########## ( 2, 3 ] 508192 24.233% 75.550% ##### ( 3, 4 ] 218666 10.427% 85.978% ## ( 4, 6 ] 223267 10.647% 96.624% ## ( 6, 10 ] 53107 2.532% 99.156% # ( 10, 15 ] 9595 0.458% 99.614% ( 15, 22 ] 1174 0.056% 99.670% ( 22, 34 ] 112 0.005% 99.675% ( 34, 51 ] 188 0.009% 99.684% ( 51, 76 ] 191 0.009% 99.693% ( 76, 110 ] 81 0.004% 99.697% ( 110, 170 ] 142 0.007% 99.704% ( 170, 250 ] 295 0.014% 99.718% ( 250, 380 ] 802 0.038% 99.756% ( 380, 580 ] 1557 0.074% 99.831% ( 580, 870 ] 2130 0.102% 99.932% ( 870, 1300 ] 2074 0.099% 100.031% ( 1300, 1900 ] 1317 0.063% 100.094% ( 1900, 2900 ] 1252 0.060% 100.154% ( 2900, 4400 ] 1338 0.064% 100.217% ( 4400, 6600 ] 333 0.016% 100.233% ( 6600, 9900 ] 46 0.002% 100.235% ( 9900, 14000 ] 7 0.000% 100.236% ( 14000, 22000 ] 7 0.000% 100.236% ( 22000, 33000 ] 4 0.000% 100.236% ( 33000, 50000 ] 1 0.000% 100.236% ** Level 4 read latency histogram (micros): Count: 395463 Average: 34.3243 StdDev: 297.03 Min: 1 Median: 1.9387 Max: 34471 Percentiles: P50: 1.94 P75: 2.99 P99: 1006.65 P99.9: 3477.07 P99.99: 4153.71 ------------------------------------------------------ [ 0, 1 ] 9225 2.333% 2.333% ( 1, 2 ] 200819 50.781% 53.113% ########## ( 2, 3 ] 86998 21.999% 75.112% #### ( 3, 4 ] 40602 10.267% 85.379% ## ( 4, 6 ] 37226 9.413% 94.793% ## ( 6, 10 ] 10163 2.570% 97.363% # ( 10, 15 ] 1521 0.385% 97.747% ( 15, 22 ] 177 0.045% 97.792% ( 22, 34 ] 26 0.007% 97.799% ( 34, 51 ] 25 0.006% 97.805% ( 51, 76 ] 40 0.010% 97.815% ( 76, 110 ] 16 0.004% 97.819% ( 110, 170 ] 101 0.026% 97.845% ( 170, 250 ] 265 0.067% 97.912% ( 250, 380 ] 561 0.142% 98.053% ( 380, 580 ] 1258 0.318% 98.372% ( 580, 870 ] 1916 0.484% 98.856% ( 870, 1300 ] 1798 0.455% 99.311% ( 1300, 1900 ] 1114 0.282% 99.592% ( 1900, 2900 ] 915 0.231% 99.824% ( 2900, 4400 ] 789 0.200% 100.023% ( 4400, 6600 ] 222 0.056% 100.079% ( 6600, 9900 ] 24 0.006% 100.085% ( 9900, 14000 ] 4 0.001% 100.086% ( 14000, 22000 ] 5 0.001% 100.088% ( 22000, 33000 ] 5 0.001% 100.089% ( 33000, 50000 ] 1 0.000% 100.089% ** Level 5 read latency histogram (micros): Count: 31808 Average: 1429.1628 StdDev: 1545.90 Min: 84 Median: 943.5334 Max: 75403 Percentiles: P50: 943.53 P75: 1768.08 P99: 6331.06 P99.9: 10388.72 P99.99: 42102.93 ------------------------------------------------------ ( 76, 110 ] 47 0.148% 0.148% ( 110, 170 ] 323 1.015% 1.163% ( 170, 250 ] 821 2.581% 3.744% # ( 250, 380 ] 2183 6.863% 10.607% # ( 380, 580 ] 4689 14.742% 25.349% ### ( 580, 870 ] 6784 21.328% 46.677% #### ( 870, 1300 ] 6181 19.432% 66.109% #### ( 1300, 1900 ] 3625 11.397% 77.506% ## ( 1900, 2900 ] 2845 8.944% 86.450% ## ( 2900, 4400 ] 3022 9.501% 95.951% ## ( 4400, 6600 ] 1105 3.474% 99.425% # ( 6600, 9900 ] 150 0.472% 99.896% ( 9900, 14000 ] 10 0.031% 99.928% ( 14000, 22000 ] 8 0.025% 99.953% ( 22000, 33000 ] 7 0.022% 99.975% ( 33000, 50000 ] 9 0.028% 100.003% ( 75000, 110000 ] 1 0.003% 100.006% ** DB Stats ** Uptime(secs): 15.6 total, 7.3 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 125.82 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.641 H:M:S, 48.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 134.51 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.461 H:M:S, 47.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/07/15-21:20:39 ... thread 0: (1000000,3000000) ops and (166542.3,139873.5) ops/second in (6.004480,21.447944) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/10 2.15 GB 62.9 1.7 0.0 1.7 4.5 2.8 0.0 1.6 58.6 152.7 30.10 15.33 38 0.792 1790K 2335 0.0 0.0 L1 2/2 214.21 MB 0.0 1.2 1.0 0.2 1.2 1.0 0.0 1.2 122.9 122.6 9.83 5.69 2 4.915 1226K 2631 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.1 0.1 0.1 0.8 2.3 106.0 105.8 1.15 0.56 1 1.151 123K 264 0.0 0.0 L3 16/0 957.82 MB 1.0 9.3 4.0 5.3 9.2 3.9 0.3 2.3 117.0 116.1 81.33 44.33 28 2.905 5065K 12K 0.0 0.0 L4 186/0 9.74 GB 1.0 9.4 5.7 3.7 9.2 5.6 0.5 1.6 121.5 119.6 79.07 44.38 62 1.275 6811K 4826 0.0 0.0 L5 1152/0 94.86 GB 1.0 0.0 0.0 0.0 0.0 0.0 6.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1370/12 107.89 GB 0.0 21.7 10.7 11.0 24.2 13.3 7.5 8.8 110.3 123.2 201.48 110.29 131 1.538 15M 22K 0.0 0.0 Int 0/0 0.00 KB 0.0 21.7 10.7 11.0 24.2 13.3 7.5 8.8 110.3 123.2 201.48 110.29 131 1.538 15M 22K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 21.7 10.7 11.0 21.5 10.5 0.0 0.0 122.3 121.1 181.59 102.13 98 1.853 15M 22K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 142.6 19.88 8.16 33 0.602 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 21.6 total, 21.5 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 24.24 GB write, 1146.64 MB/s write, 21.69 GB read, 1026.20 MB/s read, 201.5 seconds Interval compaction: 24.24 GB write, 1157.21 MB/s write, 21.69 GB read, 1035.65 MB/s read, 201.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 65, pending-compaction-bytes-delays: 10, pending-compaction-bytes-stops: 0, total-delays: 10, total-stops: 65, interval: 75 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 904874 Average: 11.0374 StdDev: 138.91 Min: 1 Median: 1.8364 Max: 32119 Percentiles: P50: 1.84 P75: 2.79 P99: 12.71 P99.9: 1816.71 P99.99: 3945.56 ------------------------------------------------------ [ 0, 1 ] 43888 4.850% 4.850% # ( 1, 2 ] 488477 53.983% 58.833% ########### ( 2, 3 ] 185847 20.538% 79.371% #### ( 3, 4 ] 80432 8.889% 88.260% ## ( 4, 6 ] 77718 8.589% 96.849% ## ( 6, 10 ] 17662 1.952% 98.801% ( 10, 15 ] 3331 0.368% 99.169% ( 15, 22 ] 327 0.036% 99.205% ( 22, 34 ] 39 0.004% 99.210% ( 34, 51 ] 32 0.004% 99.213% ( 51, 76 ] 65 0.007% 99.220% ( 76, 110 ] 210 0.023% 99.243% ( 110, 170 ] 557 0.062% 99.305% ( 170, 250 ] 463 0.051% 99.356% ( 250, 380 ] 621 0.069% 99.425% ( 380, 580 ] 1074 0.119% 99.543% ( 580, 870 ] 1425 0.157% 99.701% ( 870, 1300 ] 1257 0.139% 99.840% ( 1300, 1900 ] 633 0.070% 99.910% ( 1900, 2900 ] 447 0.049% 99.959% ( 2900, 4400 ] 401 0.044% 100.004% ( 4400, 6600 ] 105 0.012% 100.015% ( 6600, 9900 ] 15 0.002% 100.017% ( 9900, 14000 ] 3 0.000% 100.017% ( 14000, 22000 ] 3 0.000% 100.017% ( 22000, 33000 ] 1 0.000% 100.018% ** Level 1 read latency histogram (micros): Count: 285516 Average: 13.3734 StdDev: 157.50 Min: 1 Median: 1.9926 Max: 13175 Percentiles: P50: 1.99 P75: 3.01 P99: 13.49 P99.9: 2427.98 P99.99: 4285.80 ------------------------------------------------------ [ 0, 1 ] 4691 1.643% 1.643% ( 1, 2 ] 139097 48.718% 50.361% ########## ( 2, 3 ] 70087 24.547% 74.908% ##### ( 3, 4 ] 29518 10.338% 85.247% ## ( 4, 6 ] 30532 10.694% 95.940% ## ( 6, 10 ] 7794 2.730% 98.670% # ( 10, 15 ] 1350 0.473% 99.143% ( 15, 22 ] 150 0.053% 99.195% ( 22, 34 ] 14 0.005% 99.200% ( 34, 51 ] 32 0.011% 99.212% ( 51, 76 ] 25 0.009% 99.220% ( 76, 110 ] 12 0.004% 99.225% ( 110, 170 ] 46 0.016% 99.241% ( 170, 250 ] 81 0.028% 99.269% ( 250, 380 ] 143 0.050% 99.319% ( 380, 580 ] 310 0.109% 99.428% ( 580, 870 ] 484 0.170% 99.597% ( 870, 1300 ] 469 0.164% 99.761% ( 1300, 1900 ] 292 0.102% 99.864% ( 1900, 2900 ] 196 0.069% 99.932% ( 2900, 4400 ] 178 0.062% 99.995% ( 4400, 6600 ] 46 0.016% 100.011% ( 6600, 9900 ] 13 0.005% 100.015% ( 9900, 14000 ] 1 0.000% 100.016% ** Level 2 read latency histogram (micros): Count: 745036 Average: 14.9747 StdDev: 195.26 Min: 1 Median: 1.9842 Max: 26923 Percentiles: P50: 1.98 P75: 3.03 P99: 12.10 P99.9: 2193.49 P99.99: 3363.14 ------------------------------------------------------ [ 0, 1 ] 16951 2.275% 2.275% ( 1, 2 ] 361268 48.490% 50.765% ########## ( 2, 3 ] 177993 23.891% 74.656% ##### ( 3, 4 ] 78011 10.471% 85.126% ## ( 4, 6 ] 80463 10.800% 95.926% ## ( 6, 10 ] 21177 2.842% 98.769% # ( 10, 15 ] 4092 0.549% 99.318% ( 15, 22 ] 540 0.072% 99.390% ( 22, 34 ] 84 0.011% 99.402% ( 34, 51 ] 50 0.007% 99.408% ( 51, 76 ] 53 0.007% 99.416% ( 76, 110 ] 22 0.003% 99.419% ( 110, 170 ] 49 0.007% 99.425% ( 170, 250 ] 114 0.015% 99.440% ( 250, 380 ] 254 0.034% 99.475% ( 380, 580 ] 568 0.076% 99.551% ( 580, 870 ] 835 0.112% 99.663% ( 870, 1300 ] 920 0.123% 99.786% ( 1300, 1900 ] 665 0.089% 99.876% ( 1900, 2900 ] 620 0.083% 99.959% ( 2900, 4400 ] 753 0.101% 100.060% ( 4400, 6600 ] 255 0.034% 100.094% ( 6600, 9900 ] 39 0.005% 100.099% ( 9900, 14000 ] 3 0.000% 100.100% ( 14000, 22000 ] 6 0.001% 100.101% ( 22000, 33000 ] 2 0.000% 100.101% ** Level 3 read latency histogram (micros): Count: 2763638 Average: 10.6093 StdDev: 149.33 Min: 1 Median: 1.9687 Max: 36012 Percentiles: P50: 1.97 P75: 2.97 P99: 9.73 P99.9: 766.13 P99.99: 1122.17 ------------------------------------------------------ [ 0, 1 ] 42093 1.523% 1.523% ( 1, 2 ] 1383046 50.044% 51.567% ########## ( 2, 3 ] 670326 24.255% 75.823% ##### ( 3, 4 ] 284913 10.309% 86.132% ## ( 4, 6 ] 290514 10.512% 96.644% ## ( 6, 10 ] 69786 2.525% 99.169% # ( 10, 15 ] 12693 0.459% 99.628% ( 15, 22 ] 1498 0.054% 99.683% ( 22, 34 ] 133 0.005% 99.688% ( 34, 51 ] 236 0.009% 99.696% ( 51, 76 ] 252 0.009% 99.705% ( 76, 110 ] 110 0.004% 99.709% ( 110, 170 ] 217 0.008% 99.717% ( 170, 250 ] 408 0.015% 99.732% ( 250, 380 ] 1003 0.036% 99.768% ( 380, 580 ] 1932 0.070% 99.838% ( 580, 870 ] 2671 0.097% 99.935% ( 870, 1300 ] 2610 0.094% 100.029% ( 1300, 1900 ] 1606 0.058% 100.087% ( 1900, 2900 ] 1519 0.055% 100.142% ( 2900, 4400 ] 1650 0.060% 100.202% ( 4400, 6600 ] 446 0.016% 100.218% ( 6600, 9900 ] 73 0.003% 100.221% ( 9900, 14000 ] 10 0.000% 100.221% ( 14000, 22000 ] 7 0.000% 100.221% ( 22000, 33000 ] 4 0.000% 100.221% ( 33000, 50000 ] 1 0.000% 100.221% ** Level 4 read latency histogram (micros): Count: 947396 Average: 24.2277 StdDev: 233.26 Min: 1 Median: 1.9569 Max: 34471 Percentiles: P50: 1.96 P75: 2.98 P99: 635.20 P99.9: 2612.66 P99.99: 3438.56 ------------------------------------------------------ [ 0, 1 ] 17948 1.894% 1.894% ( 1, 2 ] 476258 50.270% 52.165% ########## ( 2, 3 ] 219673 23.187% 75.352% ##### ( 3, 4 ] 96170 10.151% 85.503% ## ( 4, 6 ] 92469 9.760% 95.263% ## ( 6, 10 ] 24933 2.632% 97.895% # ( 10, 15 ] 4165 0.440% 98.334% ( 15, 22 ] 483 0.051% 98.385% ( 22, 34 ] 53 0.006% 98.391% ( 34, 51 ] 83 0.009% 98.400% ( 51, 76 ] 107 0.011% 98.411% ( 76, 110 ] 153 0.016% 98.427% ( 110, 170 ] 518 0.055% 98.482% ( 170, 250 ] 742 0.078% 98.560% ( 250, 380 ] 1207 0.127% 98.688% ( 380, 580 ] 2325 0.245% 98.933% ( 580, 870 ] 3336 0.352% 99.285% ( 870, 1300 ] 3041 0.321% 99.606% ( 1300, 1900 ] 1799 0.190% 99.796% ( 1900, 2900 ] 1383 0.146% 99.942% ( 2900, 4400 ] 1268 0.134% 100.076% ( 4400, 6600 ] 334 0.035% 100.111% ( 6600, 9900 ] 42 0.004% 100.115% ( 9900, 14000 ] 4 0.000% 100.116% ( 14000, 22000 ] 5 0.001% 100.116% ( 22000, 33000 ] 5 0.001% 100.117% ( 33000, 50000 ] 1 0.000% 100.117% ** Level 5 read latency histogram (micros): Count: 51842 Average: 1217.3825 StdDev: 1385.66 Min: 78 Median: 800.4347 Max: 75403 Percentiles: P50: 800.43 P75: 1461.50 P99: 6127.89 P99.9: 9549.09 P99.99: 32710.54 ------------------------------------------------------ ( 76, 110 ] 757 1.460% 1.460% ( 110, 170 ] 2826 5.451% 6.911% # ( 170, 250 ] 2634 5.081% 11.992% # ( 250, 380 ] 4110 7.928% 19.920% ## ( 380, 580 ] 7707 14.866% 34.786% ### ( 580, 870 ] 10376 20.015% 54.801% #### ( 870, 1300 ] 9085 17.524% 72.326% #### ( 1300, 1900 ] 5151 9.936% 82.261% ## ( 1900, 2900 ] 3716 7.168% 89.429% # ( 2900, 4400 ] 3895 7.513% 96.943% ## ( 4400, 6600 ] 1358 2.619% 99.562% # ( 6600, 9900 ] 196 0.378% 99.940% ( 9900, 14000 ] 11 0.021% 99.961% ( 14000, 22000 ] 8 0.015% 99.977% ( 22000, 33000 ] 7 0.014% 99.990% ( 33000, 50000 ] 9 0.017% 100.008% ( 75000, 110000 ] 1 0.002% 100.010% ** DB Stats ** Uptime(secs): 21.6 total, 6.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 136.39 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.876 H:M:S, 45.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 163.91 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.235 H:M:S, 37.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/15-21:20:43 ... thread 0: (1000000,4000000) ops and (297676.5,161242.9) ops/second in (3.359352,24.807296) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/10 3.13 GB 169.9 2.8 0.0 2.8 6.5 3.7 0.0 1.7 75.6 178.1 37.47 22.15 58 0.646 2878K 3456 0.0 0.0 L1 2/2 214.21 MB 0.0 1.2 1.0 0.2 1.2 1.0 0.0 1.2 122.9 122.6 9.83 5.69 2 4.915 1226K 2631 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.1 0.1 0.1 0.8 2.3 106.0 105.8 1.15 0.56 1 1.151 123K 264 0.0 0.0 L3 16/0 957.82 MB 1.0 9.3 4.0 5.3 9.2 3.9 0.3 2.3 117.0 116.1 81.33 44.33 28 2.905 5065K 12K 0.0 0.0 L4 186/0 9.74 GB 1.0 9.4 5.7 3.7 9.2 5.6 0.5 1.6 121.5 119.6 79.07 44.38 62 1.275 6811K 4826 0.0 0.0 L5 1152/0 94.86 GB 1.0 0.0 0.0 0.0 0.0 0.0 6.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1374/12 108.87 GB 0.0 22.7 10.7 12.0 26.3 14.3 7.5 7.0 111.5 128.8 208.85 117.11 151 1.383 16M 23K 0.0 0.0 Int 0/0 0.00 KB 0.0 22.7 10.7 12.0 26.3 14.3 7.5 7.0 111.5 128.8 208.85 117.11 151 1.383 16M 23K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 22.7 10.7 12.0 22.5 10.5 0.0 0.0 125.4 124.1 185.75 106.19 102 1.821 16M 23K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 166.3 23.10 10.92 49 0.472 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 25.0 total, 24.8 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 26.27 GB write, 1075.70 MB/s write, 22.74 GB read, 931.17 MB/s read, 208.9 seconds Interval compaction: 26.27 GB write, 1084.27 MB/s write, 22.74 GB read, 938.58 MB/s read, 208.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 65, pending-compaction-bytes-delays: 10, pending-compaction-bytes-stops: 0, total-delays: 10, total-stops: 65, interval: 75 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1180937 Average: 11.9260 StdDev: 125.64 Min: 0 Median: 1.7457 Max: 32119 Percentiles: P50: 1.75 P75: 2.64 P99: 194.16 P99.9: 1408.90 P99.99: 3127.60 ------------------------------------------------------ [ 0, 1 ] 113805 9.637% 9.637% ## ( 1, 2 ] 639187 54.125% 63.762% ########### ( 2, 3 ] 208718 17.674% 81.436% #### ( 3, 4 ] 89665 7.593% 89.029% ## ( 4, 6 ] 82311 6.970% 95.999% # ( 6, 10 ] 18763 1.589% 97.588% ( 10, 15 ] 3571 0.302% 97.890% ( 15, 22 ] 395 0.033% 97.924% ( 22, 34 ] 45 0.004% 97.927% ( 34, 51 ] 33 0.003% 97.930% ( 51, 76 ] 72 0.006% 97.936% ( 76, 110 ] 2629 0.223% 98.159% ( 110, 170 ] 8665 0.734% 98.893% ( 170, 250 ] 4201 0.356% 99.248% ( 250, 380 ] 2160 0.183% 99.431% ( 380, 580 ] 2306 0.195% 99.626% ( 580, 870 ] 1784 0.151% 99.778% ( 870, 1300 ] 1327 0.112% 99.890% ( 1300, 1900 ] 656 0.056% 99.945% ( 1900, 2900 ] 464 0.039% 99.985% ( 2900, 4400 ] 408 0.035% 100.019% ( 4400, 6600 ] 105 0.009% 100.028% ( 6600, 9900 ] 15 0.001% 100.029% ( 9900, 14000 ] 3 0.000% 100.030% ( 14000, 22000 ] 3 0.000% 100.030% ( 22000, 33000 ] 1 0.000% 100.030% ** Level 1 read latency histogram (micros): Count: 285989 Average: 13.6779 StdDev: 157.63 Min: 1 Median: 1.9943 Max: 13175 Percentiles: P50: 1.99 P75: 3.02 P99: 17.81 P99.9: 2425.57 P99.99: 4285.40 ------------------------------------------------------ [ 0, 1 ] 4691 1.640% 1.640% ( 1, 2 ] 139097 48.637% 50.277% ########## ( 2, 3 ] 70087 24.507% 74.784% ##### ( 3, 4 ] 29518 10.321% 85.106% ## ( 4, 6 ] 30532 10.676% 95.782% ## ( 6, 10 ] 7794 2.725% 98.507% # ( 10, 15 ] 1350 0.472% 98.979% ( 15, 22 ] 150 0.052% 99.031% ( 22, 34 ] 14 0.005% 99.036% ( 34, 51 ] 32 0.011% 99.048% ( 51, 76 ] 25 0.009% 99.056% ( 76, 110 ] 83 0.029% 99.085% ( 110, 170 ] 266 0.093% 99.178% ( 170, 250 ] 175 0.061% 99.239% ( 250, 380 ] 185 0.065% 99.304% ( 380, 580 ] 340 0.119% 99.423% ( 580, 870 ] 500 0.175% 99.598% ( 870, 1300 ] 469 0.164% 99.762% ( 1300, 1900 ] 292 0.102% 99.864% ( 1900, 2900 ] 196 0.069% 99.933% ( 2900, 4400 ] 178 0.062% 99.995% ( 4400, 6600 ] 46 0.016% 100.011% ( 6600, 9900 ] 13 0.005% 100.015% ( 9900, 14000 ] 1 0.000% 100.016% ** Level 2 read latency histogram (micros): Count: 745036 Average: 14.9747 StdDev: 195.26 Min: 1 Median: 1.9842 Max: 26923 Percentiles: P50: 1.98 P75: 3.03 P99: 12.10 P99.9: 2193.49 P99.99: 3363.14 ------------------------------------------------------ [ 0, 1 ] 16951 2.275% 2.275% ( 1, 2 ] 361268 48.490% 50.765% ########## ( 2, 3 ] 177993 23.891% 74.656% ##### ( 3, 4 ] 78011 10.471% 85.126% ## ( 4, 6 ] 80463 10.800% 95.926% ## ( 6, 10 ] 21177 2.842% 98.769% # ( 10, 15 ] 4092 0.549% 99.318% ( 15, 22 ] 540 0.072% 99.390% ( 22, 34 ] 84 0.011% 99.402% ( 34, 51 ] 50 0.007% 99.408% ( 51, 76 ] 53 0.007% 99.416% ( 76, 110 ] 22 0.003% 99.419% ( 110, 170 ] 49 0.007% 99.425% ( 170, 250 ] 114 0.015% 99.440% ( 250, 380 ] 254 0.034% 99.475% ( 380, 580 ] 568 0.076% 99.551% ( 580, 870 ] 835 0.112% 99.663% ( 870, 1300 ] 920 0.123% 99.786% ( 1300, 1900 ] 665 0.089% 99.876% ( 1900, 2900 ] 620 0.083% 99.959% ( 2900, 4400 ] 753 0.101% 100.060% ( 4400, 6600 ] 255 0.034% 100.094% ( 6600, 9900 ] 39 0.005% 100.099% ( 9900, 14000 ] 3 0.000% 100.100% ( 14000, 22000 ] 6 0.001% 100.101% ( 22000, 33000 ] 2 0.000% 100.101% ** Level 3 read latency histogram (micros): Count: 2764463 Average: 10.6643 StdDev: 149.36 Min: 1 Median: 1.9690 Max: 36012 Percentiles: P50: 1.97 P75: 2.97 P99: 9.78 P99.9: 767.13 P99.99: 1122.60 ------------------------------------------------------ [ 0, 1 ] 42093 1.523% 1.523% ( 1, 2 ] 1383046 50.029% 51.552% ########## ( 2, 3 ] 670326 24.248% 75.800% ##### ( 3, 4 ] 284913 10.306% 86.106% ## ( 4, 6 ] 290514 10.509% 96.615% ## ( 6, 10 ] 69786 2.524% 99.140% # ( 10, 15 ] 12693 0.459% 99.599% ( 15, 22 ] 1498 0.054% 99.653% ( 22, 34 ] 133 0.005% 99.658% ( 34, 51 ] 236 0.009% 99.666% ( 51, 76 ] 252 0.009% 99.675% ( 76, 110 ] 220 0.008% 99.683% ( 110, 170 ] 632 0.023% 99.706% ( 170, 250 ] 568 0.021% 99.727% ( 250, 380 ] 1081 0.039% 99.766% ( 380, 580 ] 1975 0.071% 99.837% ( 580, 870 ] 2685 0.097% 99.934% ( 870, 1300 ] 2614 0.095% 100.029% ( 1300, 1900 ] 1607 0.058% 100.087% ( 1900, 2900 ] 1519 0.055% 100.142% ( 2900, 4400 ] 1650 0.060% 100.202% ( 4400, 6600 ] 446 0.016% 100.218% ( 6600, 9900 ] 73 0.003% 100.221% ( 9900, 14000 ] 10 0.000% 100.221% ( 14000, 22000 ] 7 0.000% 100.221% ( 22000, 33000 ] 4 0.000% 100.221% ( 33000, 50000 ] 1 0.000% 100.221% ** Level 4 read latency histogram (micros): Count: 955277 Average: 25.5427 StdDev: 233.26 Min: 1 Median: 1.9651 Max: 34471 Percentiles: P50: 1.97 P75: 3.02 P99: 643.13 P99.9: 2609.87 P99.99: 3439.14 ------------------------------------------------------ [ 0, 1 ] 17948 1.879% 1.879% ( 1, 2 ] 476334 49.863% 51.742% ########## ( 2, 3 ] 219981 23.028% 74.770% ##### ( 3, 4 ] 96350 10.086% 84.856% ## ( 4, 6 ] 92712 9.705% 94.562% ## ( 6, 10 ] 24969 2.614% 97.175% # ( 10, 15 ] 4172 0.437% 97.612% ( 15, 22 ] 484 0.051% 97.663% ( 22, 34 ] 54 0.006% 97.668% ( 34, 51 ] 83 0.009% 97.677% ( 51, 76 ] 107 0.011% 97.688% ( 76, 110 ] 1104 0.116% 97.804% ( 110, 170 ] 3782 0.396% 98.200% ( 170, 250 ] 2234 0.234% 98.434% ( 250, 380 ] 1840 0.193% 98.626% ( 380, 580 ] 2810 0.294% 98.920% ( 580, 870 ] 3492 0.366% 99.286% ( 870, 1300 ] 3071 0.321% 99.607% ( 1300, 1900 ] 1808 0.189% 99.797% ( 1900, 2900 ] 1390 0.146% 99.942% ( 2900, 4400 ] 1270 0.133% 100.075% ( 4400, 6600 ] 334 0.035% 100.110% ( 6600, 9900 ] 42 0.004% 100.115% ( 9900, 14000 ] 4 0.000% 100.115% ( 14000, 22000 ] 5 0.001% 100.115% ( 22000, 33000 ] 5 0.001% 100.116% ( 33000, 50000 ] 1 0.000% 100.116% ** Level 5 read latency histogram (micros): Count: 93725 Average: 762.7403 StdDev: 1152.75 Min: 75 Median: 354.2016 Max: 75403 Percentiles: P50: 354.20 P75: 879.59 P99: 5439.65 P99.9: 8742.90 P99.99: 18627.50 ------------------------------------------------------ ( 51, 76 ] 5 0.005% 0.005% ( 76, 110 ] 6592 7.033% 7.039% # ( 110, 170 ] 22792 24.318% 31.357% ##### ( 170, 250 ] 11425 12.190% 43.547% ## ( 250, 380 ] 7546 8.051% 51.598% ## ( 380, 580 ] 10493 11.196% 62.793% ## ( 580, 870 ] 11235 11.987% 74.780% ## ( 870, 1300 ] 9226 9.844% 84.624% ## ( 1300, 1900 ] 5184 5.531% 90.155% # ( 1900, 2900 ] 3739 3.989% 94.145% # ( 2900, 4400 ] 3909 4.171% 98.315% # ( 4400, 6600 ] 1358 1.449% 99.764% ( 6600, 9900 ] 196 0.209% 99.973% ( 9900, 14000 ] 11 0.012% 99.985% ( 14000, 22000 ] 8 0.009% 99.994% ( 22000, 33000 ] 7 0.007% 100.001% ( 33000, 50000 ] 9 0.010% 100.011% ( 75000, 110000 ] 1 0.001% 100.012% ** DB Stats ** Uptime(secs): 25.0 total, 3.4 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 157.42 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.876 H:M:S, 39.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 292.97 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/15-21:20:48 ... thread 0: (1000000,5000000) ops and (174303.3,163696.0) ops/second in (5.737125,30.544421) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/13 4.12 GB 138.4 4.0 0.0 4.0 8.7 4.7 0.0 1.8 90.2 197.0 45.38 29.51 79 0.574 4157K 4741 0.0 0.0 L1 2/2 214.21 MB 0.0 1.2 1.0 0.2 1.2 1.0 0.0 1.2 122.9 122.6 9.83 5.69 2 4.915 1226K 2631 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.1 0.1 0.1 0.8 2.3 106.0 105.8 1.15 0.56 1 1.151 123K 264 0.0 0.0 L3 16/0 957.82 MB 1.0 9.3 4.0 5.3 9.2 3.9 0.3 2.3 117.0 116.1 81.33 44.33 28 2.905 5065K 12K 0.0 0.0 L4 186/0 9.74 GB 1.0 9.4 5.7 3.7 9.2 5.6 0.5 1.6 121.5 119.6 79.07 44.38 62 1.275 6811K 4826 0.0 0.0 L5 1152/0 94.86 GB 1.0 0.0 0.0 0.0 0.0 0.0 6.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1375/15 109.86 GB 0.0 24.0 10.7 13.2 28.5 15.2 7.5 6.0 113.2 134.6 216.76 124.47 172 1.260 17M 25K 0.0 0.0 Int 0/0 0.00 KB 0.0 24.0 10.7 13.2 28.5 15.2 7.5 6.0 113.2 134.6 216.76 124.47 172 1.260 17M 25K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 24.0 10.7 13.2 23.7 10.5 0.0 0.0 128.9 127.7 190.48 110.79 107 1.780 17M 25K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 184.6 26.28 13.67 65 0.404 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 30.7 total, 30.5 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 28.48 GB write, 948.69 MB/s write, 23.97 GB read, 798.39 MB/s read, 216.8 seconds Interval compaction: 28.48 GB write, 954.83 MB/s write, 23.97 GB read, 803.56 MB/s read, 216.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 9, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 9, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 65, pending-compaction-bytes-delays: 10, pending-compaction-bytes-stops: 0, total-delays: 19, total-stops: 65, interval: 84 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1554146 Average: 13.7787 StdDev: 114.43 Min: 0 Median: 1.6763 Max: 32119 Percentiles: P50: 1.68 P75: 2.51 P99: 231.15 P99.9: 1164.70 P99.99: 2510.71 ------------------------------------------------------ [ 0, 1 ] 215078 13.839% 13.839% ### ( 1, 2 ] 831033 53.472% 67.311% ########### ( 2, 3 ] 233731 15.039% 82.350% ### ( 3, 4 ] 99902 6.428% 88.778% # ( 4, 6 ] 87865 5.654% 94.432% # ( 6, 10 ] 19906 1.281% 95.713% ( 10, 15 ] 3943 0.254% 95.966% ( 15, 22 ] 436 0.028% 95.994% ( 22, 34 ] 51 0.003% 95.998% ( 34, 51 ] 36 0.002% 96.000% ( 51, 76 ] 95 0.006% 96.006% ( 76, 110 ] 9080 0.584% 96.590% ( 110, 170 ] 28993 1.866% 98.456% ( 170, 250 ] 11065 0.712% 99.168% ( 250, 380 ] 4333 0.279% 99.447% ( 380, 580 ] 3869 0.249% 99.696% ( 580, 870 ] 2230 0.143% 99.839% ( 870, 1300 ] 1383 0.089% 99.928% ( 1300, 1900 ] 668 0.043% 99.971% ( 1900, 2900 ] 484 0.031% 100.002% ( 2900, 4400 ] 422 0.027% 100.029% ( 4400, 6600 ] 105 0.007% 100.036% ( 6600, 9900 ] 15 0.001% 100.037% ( 9900, 14000 ] 3 0.000% 100.037% ( 14000, 22000 ] 3 0.000% 100.038% ( 22000, 33000 ] 1 0.000% 100.038% ** Level 1 read latency histogram (micros): Count: 286849 Average: 14.1428 StdDev: 157.71 Min: 1 Median: 1.9974 Max: 13175 Percentiles: P50: 2.00 P75: 3.04 P99: 146.99 P99.9: 2421.18 P99.99: 4284.68 ------------------------------------------------------ [ 0, 1 ] 4691 1.635% 1.635% ( 1, 2 ] 139097 48.491% 50.127% ########## ( 2, 3 ] 70087 24.433% 74.560% ##### ( 3, 4 ] 29518 10.290% 84.851% ## ( 4, 6 ] 30532 10.644% 95.494% ## ( 6, 10 ] 7794 2.717% 98.212% # ( 10, 15 ] 1350 0.471% 98.682% ( 15, 22 ] 150 0.052% 98.735% ( 22, 34 ] 14 0.005% 98.739% ( 34, 51 ] 32 0.011% 98.751% ( 51, 76 ] 25 0.009% 98.759% ( 76, 110 ] 230 0.080% 98.839% ( 110, 170 ] 747 0.260% 99.100% ( 170, 250 ] 320 0.112% 99.211% ( 250, 380 ] 234 0.082% 99.293% ( 380, 580 ] 371 0.129% 99.422% ( 580, 870 ] 506 0.176% 99.599% ( 870, 1300 ] 470 0.164% 99.763% ( 1300, 1900 ] 292 0.102% 99.864% ( 1900, 2900 ] 196 0.068% 99.933% ( 2900, 4400 ] 178 0.062% 99.995% ( 4400, 6600 ] 46 0.016% 100.011% ( 6600, 9900 ] 13 0.005% 100.015% ( 9900, 14000 ] 1 0.000% 100.016% ** Level 2 read latency histogram (micros): Count: 745036 Average: 14.9747 StdDev: 195.26 Min: 1 Median: 1.9842 Max: 26923 Percentiles: P50: 1.98 P75: 3.03 P99: 12.10 P99.9: 2193.49 P99.99: 3363.14 ------------------------------------------------------ [ 0, 1 ] 16951 2.275% 2.275% ( 1, 2 ] 361268 48.490% 50.765% ########## ( 2, 3 ] 177993 23.891% 74.656% ##### ( 3, 4 ] 78011 10.471% 85.126% ## ( 4, 6 ] 80463 10.800% 95.926% ## ( 6, 10 ] 21177 2.842% 98.769% # ( 10, 15 ] 4092 0.549% 99.318% ( 15, 22 ] 540 0.072% 99.390% ( 22, 34 ] 84 0.011% 99.402% ( 34, 51 ] 50 0.007% 99.408% ( 51, 76 ] 53 0.007% 99.416% ( 76, 110 ] 22 0.003% 99.419% ( 110, 170 ] 49 0.007% 99.425% ( 170, 250 ] 114 0.015% 99.440% ( 250, 380 ] 254 0.034% 99.475% ( 380, 580 ] 568 0.076% 99.551% ( 580, 870 ] 835 0.112% 99.663% ( 870, 1300 ] 920 0.123% 99.786% ( 1300, 1900 ] 665 0.089% 99.876% ( 1900, 2900 ] 620 0.083% 99.959% ( 2900, 4400 ] 753 0.101% 100.060% ( 4400, 6600 ] 255 0.034% 100.094% ( 6600, 9900 ] 39 0.005% 100.099% ( 9900, 14000 ] 3 0.000% 100.100% ( 14000, 22000 ] 6 0.001% 100.101% ( 22000, 33000 ] 2 0.000% 100.101% ** Level 3 read latency histogram (micros): Count: 2765959 Average: 10.7541 StdDev: 149.40 Min: 1 Median: 1.9695 Max: 36012 Percentiles: P50: 1.97 P75: 2.97 P99: 9.86 P99.9: 768.12 P99.99: 1122.94 ------------------------------------------------------ [ 0, 1 ] 42093 1.522% 1.522% ( 1, 2 ] 1383046 50.002% 51.524% ########## ( 2, 3 ] 670326 24.235% 75.759% ##### ( 3, 4 ] 284913 10.301% 86.060% ## ( 4, 6 ] 290514 10.503% 96.563% ## ( 6, 10 ] 69786 2.523% 99.086% # ( 10, 15 ] 12693 0.459% 99.545% ( 15, 22 ] 1498 0.054% 99.599% ( 22, 34 ] 133 0.005% 99.604% ( 34, 51 ] 236 0.009% 99.612% ( 51, 76 ] 252 0.009% 99.622% ( 76, 110 ] 466 0.017% 99.638% ( 110, 170 ] 1454 0.053% 99.691% ( 170, 250 ] 826 0.030% 99.721% ( 250, 380 ] 1164 0.042% 99.763% ( 380, 580 ] 2039 0.074% 99.837% ( 580, 870 ] 2704 0.098% 99.934% ( 870, 1300 ] 2617 0.095% 100.029% ( 1300, 1900 ] 1607 0.058% 100.087% ( 1900, 2900 ] 1520 0.055% 100.142% ( 2900, 4400 ] 1650 0.060% 100.202% ( 4400, 6600 ] 446 0.016% 100.218% ( 6600, 9900 ] 73 0.003% 100.220% ( 9900, 14000 ] 10 0.000% 100.221% ( 14000, 22000 ] 7 0.000% 100.221% ( 22000, 33000 ] 4 0.000% 100.221% ( 33000, 50000 ] 1 0.000% 100.221% ** Level 4 read latency histogram (micros): Count: 970480 Average: 27.7498 StdDev: 232.55 Min: 1 Median: 1.9809 Max: 34471 Percentiles: P50: 1.98 P75: 3.14 P99: 642.79 P99.9: 2601.45 P99.99: 3438.86 ------------------------------------------------------ [ 0, 1 ] 17949 1.849% 1.849% ( 1, 2 ] 476391 49.088% 50.938% ########## ( 2, 3 ] 220164 22.686% 73.624% ##### ( 3, 4 ] 96494 9.943% 83.567% ## ( 4, 6 ] 92876 9.570% 93.137% ## ( 6, 10 ] 24976 2.574% 95.710% # ( 10, 15 ] 4174 0.430% 96.140% ( 15, 22 ] 484 0.050% 96.190% ( 22, 34 ] 54 0.006% 96.196% ( 34, 51 ] 83 0.009% 96.204% ( 51, 76 ] 108 0.011% 96.216% ( 76, 110 ] 3613 0.372% 96.588% ( 110, 170 ] 11853 1.221% 97.809% ( 170, 250 ] 4741 0.489% 98.298% ( 250, 380 ] 2598 0.268% 98.565% ( 380, 580 ] 3428 0.353% 98.919% ( 580, 870 ] 3645 0.376% 99.294% ( 870, 1300 ] 3088 0.318% 99.612% ( 1300, 1900 ] 1812 0.187% 99.799% ( 1900, 2900 ] 1395 0.144% 99.943% ( 2900, 4400 ] 1272 0.131% 100.074% ( 4400, 6600 ] 334 0.034% 100.108% ( 6600, 9900 ] 42 0.004% 100.113% ( 9900, 14000 ] 4 0.000% 100.113% ( 14000, 22000 ] 5 0.001% 100.114% ( 22000, 33000 ] 5 0.001% 100.114% ( 33000, 50000 ] 1 0.000% 100.114% ** Level 5 read latency histogram (micros): Count: 174282 Average: 490.1471 StdDev: 898.45 Min: 73 Median: 169.4520 Max: 75403 Percentiles: P50: 169.45 P75: 468.59 P99: 4334.53 P99.9: 7268.70 P99.99: 10113.09 ------------------------------------------------------ ( 51, 76 ] 16 0.009% 0.009% ( 76, 110 ] 20295 11.645% 11.654% ## ( 110, 170 ] 67447 38.700% 50.354% ######## ( 170, 250 ] 24981 14.334% 64.688% ### ( 250, 380 ] 11867 6.809% 71.497% # ( 380, 580 ] 13785 7.910% 79.406% ## ( 580, 870 ] 12066 6.923% 86.330% # ( 870, 1300 ] 9371 5.377% 91.707% # ( 1300, 1900 ] 5201 2.984% 94.691% # ( 1900, 2900 ] 3768 2.162% 96.853% ( 2900, 4400 ] 3914 2.246% 99.099% ( 4400, 6600 ] 1358 0.779% 99.878% ( 6600, 9900 ] 196 0.112% 99.990% ( 9900, 14000 ] 11 0.006% 99.997% ( 14000, 22000 ] 8 0.005% 100.001% ( 22000, 33000 ] 7 0.004% 100.005% ( 33000, 50000 ] 9 0.005% 100.010% ( 75000, 110000 ] 1 0.001% 100.011% ** DB Stats ** Uptime(secs): 30.7 total, 5.7 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 160.06 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:12.140 H:M:S, 39.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 171.55 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.264 H:M:S, 39.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/15-21:20:58 ... thread 0: (1000000,6000000) ops and (107487.1,150572.7) ops/second in (9.303445,39.847866) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/10 5.03 GB 363.9 6.3 0.0 6.3 11.9 5.6 0.0 2.1 112.5 213.8 57.09 40.67 98 0.583 6523K 16K 0.0 0.0 L1 2/2 214.21 MB 0.0 1.2 1.0 0.2 1.2 1.0 0.0 1.2 122.9 122.6 9.83 5.69 2 4.915 1226K 2631 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.1 0.1 0.1 0.8 2.3 106.0 105.8 1.15 0.56 1 1.151 123K 264 0.0 0.0 L3 16/0 957.82 MB 1.0 9.3 4.0 5.3 9.2 3.9 0.3 2.3 117.0 116.1 81.33 44.33 28 2.905 5065K 12K 0.0 0.0 L4 186/0 9.74 GB 1.0 9.4 5.7 3.7 9.2 5.6 0.5 1.6 121.5 119.6 79.07 44.38 62 1.275 6811K 4826 0.0 0.0 L5 1152/0 94.86 GB 1.0 0.0 0.0 0.0 0.0 0.0 6.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1375/12 110.77 GB 0.0 26.2 10.7 15.5 31.7 16.1 7.5 5.6 117.6 141.9 228.47 135.63 191 1.196 19M 36K 0.0 0.0 Int 0/0 0.00 KB 0.0 26.2 10.7 15.5 31.7 16.1 7.5 5.6 117.6 141.9 228.47 135.63 191 1.196 19M 36K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 26.2 10.7 15.5 26.0 10.5 0.0 0.0 134.9 133.7 199.17 119.33 111 1.794 19M 36K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 197.8 29.30 16.30 80 0.366 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 40.0 total, 39.9 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 31.67 GB write, 809.78 MB/s write, 26.25 GB read, 671.09 MB/s read, 228.5 seconds Interval compaction: 31.67 GB write, 813.80 MB/s write, 26.25 GB read, 674.41 MB/s read, 228.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 16, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 16, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 65, pending-compaction-bytes-delays: 10, pending-compaction-bytes-stops: 0, total-delays: 26, total-stops: 65, interval: 91 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2253717 Average: 15.0905 StdDev: 101.20 Min: 0 Median: 1.6024 Max: 32119 Percentiles: P50: 1.60 P75: 2.32 P99: 238.45 P99.9: 861.38 P99.99: 1722.22 ------------------------------------------------------ [ 0, 1 ] 411056 18.239% 18.239% #### ( 1, 2 ] 1188246 52.724% 70.963% ########### ( 2, 3 ] 283478 12.578% 83.541% ### ( 3, 4 ] 117493 5.213% 88.754% # ( 4, 6 ] 97137 4.310% 93.064% # ( 6, 10 ] 22106 0.981% 94.045% ( 10, 15 ] 4657 0.207% 94.252% ( 15, 22 ] 550 0.024% 94.276% ( 22, 34 ] 79 0.004% 94.280% ( 34, 51 ] 41 0.002% 94.282% ( 51, 76 ] 119 0.005% 94.287% ( 76, 110 ] 22411 0.994% 95.281% ( 110, 170 ] 65260 2.896% 98.177% # ( 170, 250 ] 21675 0.962% 99.139% ( 250, 380 ] 7864 0.349% 99.488% ( 380, 580 ] 6462 0.287% 99.774% ( 580, 870 ] 2916 0.129% 99.904% ( 870, 1300 ] 1461 0.065% 99.969% ( 1300, 1900 ] 683 0.030% 99.999% ( 1900, 2900 ] 507 0.022% 100.021% ( 2900, 4400 ] 443 0.020% 100.041% ( 4400, 6600 ] 107 0.005% 100.046% ( 6600, 9900 ] 15 0.001% 100.047% ( 9900, 14000 ] 3 0.000% 100.047% ( 14000, 22000 ] 3 0.000% 100.047% ( 22000, 33000 ] 1 0.000% 100.047% ** Level 1 read latency histogram (micros): Count: 288293 Average: 14.9111 StdDev: 157.87 Min: 1 Median: 2.0051 Max: 13175 Percentiles: P50: 2.01 P75: 3.08 P99: 179.52 P99.9: 2413.81 P99.99: 4283.46 ------------------------------------------------------ [ 0, 1 ] 4691 1.627% 1.627% ( 1, 2 ] 139097 48.248% 49.876% ########## ( 2, 3 ] 70087 24.311% 74.187% ##### ( 3, 4 ] 29518 10.239% 84.426% ## ( 4, 6 ] 30532 10.591% 95.016% ## ( 6, 10 ] 7794 2.703% 97.720% # ( 10, 15 ] 1350 0.468% 98.188% ( 15, 22 ] 150 0.052% 98.240% ( 22, 34 ] 14 0.005% 98.245% ( 34, 51 ] 32 0.011% 98.256% ( 51, 76 ] 26 0.009% 98.265% ( 76, 110 ] 501 0.174% 98.439% ( 110, 170 ] 1553 0.539% 98.977% ( 170, 250 ] 547 0.190% 99.167% ( 250, 380 ] 306 0.106% 99.273% ( 380, 580 ] 421 0.146% 99.419% ( 580, 870 ] 520 0.180% 99.600% ( 870, 1300 ] 471 0.163% 99.763% ( 1300, 1900 ] 294 0.102% 99.865% ( 1900, 2900 ] 196 0.068% 99.933% ( 2900, 4400 ] 178 0.062% 99.995% ( 4400, 6600 ] 46 0.016% 100.011% ( 6600, 9900 ] 13 0.005% 100.015% ( 9900, 14000 ] 1 0.000% 100.016% ** Level 2 read latency histogram (micros): Count: 745036 Average: 14.9747 StdDev: 195.26 Min: 1 Median: 1.9842 Max: 26923 Percentiles: P50: 1.98 P75: 3.03 P99: 12.10 P99.9: 2193.49 P99.99: 3363.14 ------------------------------------------------------ [ 0, 1 ] 16951 2.275% 2.275% ( 1, 2 ] 361268 48.490% 50.765% ########## ( 2, 3 ] 177993 23.891% 74.656% ##### ( 3, 4 ] 78011 10.471% 85.126% ## ( 4, 6 ] 80463 10.800% 95.926% ## ( 6, 10 ] 21177 2.842% 98.769% # ( 10, 15 ] 4092 0.549% 99.318% ( 15, 22 ] 540 0.072% 99.390% ( 22, 34 ] 84 0.011% 99.402% ( 34, 51 ] 50 0.007% 99.408% ( 51, 76 ] 53 0.007% 99.416% ( 76, 110 ] 22 0.003% 99.419% ( 110, 170 ] 49 0.007% 99.425% ( 170, 250 ] 114 0.015% 99.440% ( 250, 380 ] 254 0.034% 99.475% ( 380, 580 ] 568 0.076% 99.551% ( 580, 870 ] 835 0.112% 99.663% ( 870, 1300 ] 920 0.123% 99.786% ( 1300, 1900 ] 665 0.089% 99.876% ( 1900, 2900 ] 620 0.083% 99.959% ( 2900, 4400 ] 753 0.101% 100.060% ( 4400, 6600 ] 255 0.034% 100.094% ( 6600, 9900 ] 39 0.005% 100.099% ( 9900, 14000 ] 3 0.000% 100.100% ( 14000, 22000 ] 6 0.001% 100.101% ( 22000, 33000 ] 2 0.000% 100.101% ** Level 3 read latency histogram (micros): Count: 2768466 Average: 10.8966 StdDev: 149.43 Min: 1 Median: 1.9704 Max: 36012 Percentiles: P50: 1.97 P75: 2.97 P99: 10.04 P99.9: 769.28 P99.99: 1123.17 ------------------------------------------------------ [ 0, 1 ] 42093 1.520% 1.520% ( 1, 2 ] 1383046 49.957% 51.478% ########## ( 2, 3 ] 670326 24.213% 75.690% ##### ( 3, 4 ] 284913 10.291% 85.982% ## ( 4, 6 ] 290514 10.494% 96.476% ## ( 6, 10 ] 69786 2.521% 98.996% # ( 10, 15 ] 12693 0.458% 99.455% ( 15, 22 ] 1498 0.054% 99.509% ( 22, 34 ] 133 0.005% 99.514% ( 34, 51 ] 236 0.009% 99.522% ( 51, 76 ] 254 0.009% 99.531% ( 76, 110 ] 917 0.033% 99.564% ( 110, 170 ] 2871 0.104% 99.668% ( 170, 250 ] 1210 0.044% 99.712% ( 250, 380 ] 1298 0.047% 99.759% ( 380, 580 ] 2127 0.077% 99.836% ( 580, 870 ] 2731 0.099% 99.934% ( 870, 1300 ] 2621 0.095% 100.029% ( 1300, 1900 ] 1607 0.058% 100.087% ( 1900, 2900 ] 1520 0.055% 100.142% ( 2900, 4400 ] 1650 0.060% 100.201% ( 4400, 6600 ] 446 0.016% 100.218% ( 6600, 9900 ] 73 0.003% 100.220% ( 9900, 14000 ] 10 0.000% 100.221% ( 14000, 22000 ] 7 0.000% 100.221% ( 22000, 33000 ] 4 0.000% 100.221% ( 33000, 50000 ] 1 0.000% 100.221% ** Level 4 read latency histogram (micros): Count: 994956 Average: 31.2346 StdDev: 231.37 Min: 1 Median: 2.0141 Max: 34471 Percentiles: P50: 2.01 P75: 3.33 P99: 643.34 P99.9: 2585.97 P99.99: 3436.73 ------------------------------------------------------ [ 0, 1 ] 17949 1.804% 1.804% ( 1, 2 ] 476421 47.884% 49.688% ########## ( 2, 3 ] 220236 22.135% 71.823% #### ( 3, 4 ] 96537 9.703% 81.526% ## ( 4, 6 ] 92925 9.340% 90.865% ## ( 6, 10 ] 24979 2.511% 93.376% # ( 10, 15 ] 4176 0.420% 93.795% ( 15, 22 ] 485 0.049% 93.844% ( 22, 34 ] 54 0.005% 93.850% ( 34, 51 ] 83 0.008% 93.858% ( 51, 76 ] 109 0.011% 93.869% ( 76, 110 ] 7807 0.785% 94.654% ( 110, 170 ] 25490 2.562% 97.215% # ( 170, 250 ] 8686 0.873% 98.088% ( 250, 380 ] 3850 0.387% 98.475% ( 380, 580 ] 4365 0.439% 98.914% ( 580, 870 ] 3912 0.393% 99.307% ( 870, 1300 ] 3116 0.313% 99.620% ( 1300, 1900 ] 1820 0.183% 99.803% ( 1900, 2900 ] 1401 0.141% 99.944% ( 2900, 4400 ] 1273 0.128% 100.072% ( 4400, 6600 ] 334 0.034% 100.106% ( 6600, 9900 ] 42 0.004% 100.110% ( 9900, 14000 ] 4 0.000% 100.110% ( 14000, 22000 ] 5 0.001% 100.111% ( 22000, 33000 ] 5 0.001% 100.111% ( 33000, 50000 ] 1 0.000% 100.111% ** Level 5 read latency histogram (micros): Count: 303833 Average: 353.5296 StdDev: 702.22 Min: 73 Median: 156.7198 Max: 75403 Percentiles: P50: 156.72 P75: 247.80 P99: 3834.26 P99.9: 6432.16 P99.99: 9455.79 ------------------------------------------------------ ( 51, 76 ] 43 0.014% 0.014% ( 76, 110 ] 43264 14.239% 14.254% ### ( 110, 170 ] 139482 45.907% 60.161% ######### ( 170, 250 ] 46359 15.258% 75.419% ### ( 250, 380 ] 18603 6.123% 81.542% # ( 380, 580 ] 18752 6.172% 87.714% # ( 580, 870 ] 13311 4.381% 92.095% # ( 870, 1300 ] 9526 3.135% 95.230% # ( 1300, 1900 ] 5222 1.719% 96.949% ( 1900, 2900 ] 3793 1.248% 98.197% ( 2900, 4400 ] 3917 1.289% 99.486% ( 4400, 6600 ] 1361 0.448% 99.934% ( 6600, 9900 ] 196 0.065% 99.999% ( 9900, 14000 ] 11 0.004% 100.002% ( 14000, 22000 ] 8 0.003% 100.005% ( 22000, 33000 ] 7 0.002% 100.007% ( 33000, 50000 ] 9 0.003% 100.010% ( 75000, 110000 ] 1 0.000% 100.011% ** DB Stats ** Uptime(secs): 40.0 total, 9.3 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 147.45 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:17.714 H:M:S, 44.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 105.79 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.574 H:M:S, 59.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 298.661 micros/op 13386 ops/sec 60.135 seconds 804996 operations; 13.0 MB/s (197999 of 197999 found) Microseconds per read: Count: 804996 Average: 298.6649 StdDev: 892.10 Min: 1 Median: 180.7484 Max: 114658 Percentiles: P50: 180.75 P75: 243.72 P99: 2510.56 P99.9: 13703.53 P99.99: 32102.65 ------------------------------------------------------ [ 0, 1 ] 1 0.000% 0.000% ( 1, 2 ] 7 0.001% 0.001% ( 2, 3 ] 35 0.004% 0.005% ( 3, 4 ] 30 0.004% 0.009% ( 4, 6 ] 89 0.011% 0.020% ( 6, 10 ] 374 0.046% 0.067% ( 10, 15 ] 2920 0.363% 0.429% ( 15, 22 ] 2901 0.360% 0.790% ( 22, 34 ] 7373 0.916% 1.706% ( 34, 51 ] 1675 0.208% 1.914% ( 51, 76 ] 220 0.027% 1.941% ( 76, 110 ] 17368 2.158% 4.099% ( 110, 170 ] 335157 41.635% 45.733% ######## ( 170, 250 ] 255650 31.758% 77.491% ###### ( 250, 380 ] 85594 10.633% 88.124% ## ( 380, 580 ] 43666 5.424% 93.548% # ( 580, 870 ] 22628 2.811% 96.359% # ( 870, 1300 ] 12593 1.564% 97.924% ( 1300, 1900 ] 6141 0.763% 98.686% ( 1900, 2900 ] 4134 0.514% 99.200% ( 2900, 4400 ] 3734 0.464% 99.664% ( 4400, 6600 ] 1152 0.143% 99.807% ( 6600, 9900 ] 377 0.047% 99.854% ( 9900, 14000 ] 401 0.050% 99.904% ( 14000, 22000 ] 521 0.065% 99.968% ( 22000, 33000 ] 190 0.024% 99.992% ( 33000, 50000 ] 47 0.006% 99.998% ( 50000, 75000 ] 15 0.002% 100.000% ( 75000, 110000 ] 2 0.000% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 12170859 rocksdb.bloom.filter.full.positive COUNT : 923323 rocksdb.bloom.filter.full.true.positive COUNT : 804613 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 383 rocksdb.memtable.miss COUNT : 804613 rocksdb.l0.hit COUNT : 32802 rocksdb.l1.hit COUNT : 5835 rocksdb.l2andup.hit COUNT : 765976 rocksdb.compaction.key.drop.new COUNT : 273987 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 6319688 rocksdb.number.keys.read COUNT : 804996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 6521918016 rocksdb.bytes.read COUNT : 804996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1794 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 36555769 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 32213652618 rocksdb.compact.write.bytes COUNT : 31925996133 rocksdb.flush.write.bytes COUNT : 6473689554 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 40 rocksdb.number.superversion_acquires COUNT : 1314 rocksdb.number.superversion_releases COUNT : 1275 rocksdb.number.superversion_cleanups COUNT : 331 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 419 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 6421493760 rocksdb.memtable.garbage.bytes.at.flush COUNT : 2490368 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 35712082191 rocksdb.non.last.level.read.count COUNT : 8487984 rocksdb.block.checksum.compute.count COUNT : 8734164 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 179.084318 P95 : 728.754453 P99 : 2509.116009 P100 : 114654.000000 COUNT : 804996 SUM : 239683383 rocksdb.db.write.micros P50 : 1.520760 P95 : 3.965083 P99 : 7.940944 P100 : 767353.000000 COUNT : 6319688 SUM : 51972875 rocksdb.compaction.times.micros P50 : 1451086.956522 P95 : 5743750.000000 P99 : 13370000.000000 P100 : 29805257.000000 COUNT : 115 SUM : 235670238 rocksdb.compaction.times.cpu_micros P50 : 802460.317460 P95 : 3775000.000000 P99 : 6417500.000000 P100 : 9664248.000000 COUNT : 115 SUM : 135597053 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 840.256410 P95 : 73125.000000 P99 : 181200.000000 P100 : 212266.000000 COUNT : 86 SUM : 1181235 rocksdb.compaction.outfile.sync.micros P50 : 34565.789474 P95 : 229466.666667 P99 : 371658.333333 P100 : 751569.000000 COUNT : 377 SUM : 23737778 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 530.000000 P95 : 3511.538462 P99 : 5617.333333 P100 : 6766.000000 COUNT : 234 SUM : 237701 rocksdb.table.open.io.micros P50 : 8687.305122 P95 : 20397.122302 P99 : 27698.709677 P100 : 90558.000000 COUNT : 1794 SUM : 16008593 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.934986 P95 : 9.731748 P99 : 167.222438 P100 : 36017.000000 COUNT : 7800078 SUM : 113679320 rocksdb.read.block.get.micros P50 : 144.651899 P95 : 596.077554 P99 : 2301.564551 P100 : 75476.000000 COUNT : 932285 SUM : 209174301 rocksdb.write.raw.block.micros P50 : 0.524369 P95 : 0.996302 P99 : 3.468655 P100 : 46065.000000 COUNT : 9300587 SUM : 29151111 rocksdb.numfiles.in.singlecompaction P50 : 2.784091 P95 : 5.829412 P99 : 13.775000 P100 : 20.000000 COUNT : 149 SUM : 532 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.561201 P95 : 1196.006195 P99 : 2589.289322 P100 : 767172.000000 COUNT : 186779 SUM : 36568717 rocksdb.sst.read.micros P50 : 1.934818 P95 : 156.760925 P99 : 527.786125 P100 : 75403.000000 COUNT : 8488045 SUM : 301027871 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 804996 SUM : 804996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 6319688 SUM : 6521918016 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 232545.454545 P95 : 991142.857143 P99 : 1010831.000000 P100 : 1010831.000000 COUNT : 86 SUM : 30514221 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 420 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0