Invalid opts->opts_size 0 too small, please set opts_size correctly [2024-08-11 14:52:28.543194] Starting SPDK v24.09-pre git sha1 227b8322c / DPDK 22.11.4 initialization... [2024-08-11 14:52:28.543449] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2370830 ] Set seed to 1723380748536473 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-08-11 14:52:28.641500] app.c: 910:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-08-11 14:52:28.683177] app.c: 604:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-08-11 14:52:28.683226] app.c: 605:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2370830' to capture a snapshot of events at runtime. [2024-08-11 14:52:28.683237] app.c: 610:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-08-11 14:52:28.683250] app.c: 611:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-08-11 14:52:28.683258] app.c: 612:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2370830 for offline analysis/debug. [2024-08-11 14:52:28.683341] reactor.c: 941:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Sun Aug 11 14:52:32 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/08/11-14:52:35 ... thread 0: (1000000,1000000) ops and (358518.0,358518.0) ops/second in (2.789260,2.789260) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 8/6 504.01 MB 12.6 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 359.3 2.28 2.06 12 0.190 0 0 0.0 0.0 L1 1/0 1.90 MB 0.2 0.2 0.2 0.1 0.2 0.2 0.1 1.3 253.8 253.6 0.99 0.96 1 0.993 255K 177 0.0 0.0 L2 3/2 298.79 MB 1.2 0.1 0.1 0.0 0.1 0.1 0.3 1.8 235.5 235.5 0.47 0.45 1 0.473 113K 22 0.0 0.0 L3 16/9 755.46 MB 0.0 0.1 0.1 0.0 0.1 0.1 0.0 1.0 200.7 200.7 0.31 0.27 1 0.312 63K 0 0.0 0.0 L4 116/0 9.66 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1036/0 85.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1180/17 96.94 GB 0.0 0.4 0.3 0.1 1.2 1.1 0.4 1.5 105.0 306.8 4.06 3.74 15 0.270 432K 199 0.0 0.0 Int 0/0 0.00 KB 0.0 0.4 0.3 0.1 1.2 1.1 0.4 1.5 105.0 306.8 4.06 3.74 15 0.270 432K 199 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.4 0.3 0.1 0.4 0.3 0.0 0.0 239.6 239.5 1.78 1.67 3 0.593 432K 199 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 359.3 2.28 2.06 12 0.190 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.0 total, 2.8 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.22 GB write, 421.45 MB/s write, 0.42 GB read, 144.22 MB/s read, 4.1 seconds Interval compaction: 1.22 GB write, 446.04 MB/s write, 0.42 GB read, 152.63 MB/s read, 4.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 4, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 4, interval: 4 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 115764 Average: 1.8443 StdDev: 17.46 Min: 0 Median: 1.1855 Max: 3773 Percentiles: P50: 1.19 P75: 1.64 P99: 4.11 P99.9: 9.53 P99.99: 305.63 ------------------------------------------------------ [ 0, 1 ] 46062 39.790% 39.790% ######## ( 1, 2 ] 63734 55.055% 94.845% ########### ( 2, 3 ] 3489 3.014% 97.859% # ( 3, 4 ] 1277 1.103% 98.962% ( 4, 6 ] 800 0.691% 99.653% ( 6, 10 ] 324 0.280% 99.933% ( 10, 15 ] 42 0.036% 99.969% ( 15, 22 ] 7 0.006% 99.975% ( 22, 34 ] 1 0.001% 99.976% ( 34, 51 ] 3 0.003% 99.978% ( 51, 76 ] 1 0.001% 99.979% ( 110, 170 ] 8 0.007% 99.986% ( 170, 250 ] 1 0.001% 99.987% ( 250, 380 ] 8 0.007% 99.994% ( 380, 580 ] 1 0.001% 99.995% ( 580, 870 ] 1 0.001% 99.996% ( 870, 1300 ] 2 0.002% 99.997% ( 1900, 2900 ] 1 0.001% 99.998% ( 2900, 4400 ] 2 0.002% 100.000% ** Level 1 read latency histogram (micros): Count: 17336 Average: 2.9620 StdDev: 27.28 Min: 1 Median: 1.3935 Max: 2435 Percentiles: P50: 1.39 P75: 1.85 P99: 9.11 P99.9: 225.74 P99.99: 1459.84 ------------------------------------------------------ [ 0, 1 ] 4914 28.346% 28.346% ###### ( 1, 2 ] 9539 55.024% 83.370% ########### ( 2, 3 ] 1718 9.910% 93.280% ## ( 3, 4 ] 442 2.550% 95.829% # ( 4, 6 ] 409 2.359% 98.189% ( 6, 10 ] 181 1.044% 99.233% ( 10, 15 ] 34 0.196% 99.429% ( 15, 22 ] 4 0.023% 99.452% ( 34, 51 ] 41 0.237% 99.689% ( 51, 76 ] 23 0.133% 99.821% ( 76, 110 ] 5 0.029% 99.850% ( 110, 170 ] 1 0.006% 99.856% ( 170, 250 ] 11 0.063% 99.919% ( 250, 380 ] 3 0.017% 99.937% ( 380, 580 ] 5 0.029% 99.965% ( 580, 870 ] 4 0.023% 99.988% ( 1300, 1900 ] 1 0.006% 99.994% ( 1900, 2900 ] 1 0.006% 100.000% ** Level 2 read latency histogram (micros): Count: 15886 Average: 1.8297 StdDev: 9.44 Min: 1 Median: 1.1690 Max: 900 Percentiles: P50: 1.17 P75: 1.63 P99: 4.94 P99.9: 9.73 P99.99: 699.31 ------------------------------------------------------ [ 0, 1 ] 6501 40.923% 40.923% ######## ( 1, 2 ] 8534 53.720% 94.643% ########### ( 2, 3 ] 410 2.581% 97.224% # ( 3, 4 ] 203 1.278% 98.502% ( 4, 6 ] 168 1.058% 99.559% ( 6, 10 ] 58 0.365% 99.924% ( 10, 15 ] 4 0.025% 99.950% ( 15, 22 ] 3 0.019% 99.969% ( 34, 51 ] 1 0.006% 99.975% ( 110, 170 ] 1 0.006% 99.981% ( 170, 250 ] 1 0.006% 99.987% ( 580, 870 ] 1 0.006% 99.994% ( 870, 1300 ] 1 0.006% 100.000% ** Level 3 read latency histogram (micros): Count: 9994 Average: 9.4083 StdDev: 123.97 Min: 1 Median: 1.7063 Max: 4778 Percentiles: P50: 1.71 P75: 2.52 P99: 86.96 P99.9: 2301.20 P99.99: 4401.32 ------------------------------------------------------ [ 0, 1 ] 1099 10.997% 10.997% ## ( 1, 2 ] 5520 55.233% 66.230% ########### ( 2, 3 ] 1683 16.840% 83.070% ### ( 3, 4 ] 636 6.364% 89.434% # ( 4, 6 ] 643 6.434% 95.868% # ( 6, 10 ] 256 2.562% 98.429% # ( 10, 15 ] 34 0.340% 98.769% ( 15, 22 ] 16 0.160% 98.929% ( 76, 110 ] 25 0.250% 99.180% ( 110, 170 ] 32 0.320% 99.500% ( 170, 250 ] 9 0.090% 99.590% ( 250, 380 ] 9 0.090% 99.680% ( 380, 580 ] 8 0.080% 99.760% ( 580, 870 ] 7 0.070% 99.830% ( 870, 1300 ] 2 0.020% 99.850% ( 1300, 1900 ] 4 0.040% 99.890% ( 1900, 2900 ] 5 0.050% 99.940% ( 2900, 4400 ] 6 0.060% 100.000% ( 4400, 6600 ] 1 0.010% 100.010% ** Level 4 read latency histogram (micros): Count: 35 Average: 1254.7143 StdDev: 953.45 Min: 56 Median: 1146.4286 Max: 4112 Percentiles: P50: 1146.43 P75: 2056.25 P99: 4112.00 P99.9: 4112.00 P99.99: 4112.00 ------------------------------------------------------ ( 51, 76 ] 1 2.857% 2.857% # ( 76, 110 ] 1 2.857% 5.714% # ( 110, 170 ] 3 8.571% 14.286% ## ( 170, 250 ] 3 8.571% 22.857% ## ( 250, 380 ] 2 5.714% 28.571% # ( 380, 580 ] 2 5.714% 34.286% # ( 580, 870 ] 1 2.857% 37.143% # ( 870, 1300 ] 7 20.000% 57.143% #### ( 1300, 1900 ] 5 14.286% 71.429% ### ( 1900, 2900 ] 8 22.857% 94.286% ##### ( 2900, 4400 ] 2 5.714% 100.000% # ** DB Stats ** Uptime(secs): 3.0 total, 2.8 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 333.21 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.182 H:M:S, 6.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 352.55 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.182 H:M:S, 6.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/08/11-14:52:38 ... thread 0: (1000000,2000000) ops and (325173.1,341032.4) ops/second in (3.075285,5.864545) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/6 1.48 GB 56.7 0.9 0.0 0.9 2.6 1.8 0.0 1.5 103.2 316.7 8.55 7.91 29 0.295 895K 1001 0.0 0.0 L1 1/1 1.90 MB 0.0 0.2 0.2 0.1 0.2 0.2 0.1 1.3 253.8 253.6 0.99 0.96 1 0.993 255K 177 0.0 0.0 L2 1/1 121.12 MB 0.0 0.1 0.1 0.0 0.1 0.1 0.3 1.8 235.5 235.5 0.47 0.45 1 0.473 113K 22 0.0 0.0 L3 76/33 932.65 MB 0.8 1.0 0.2 0.7 1.0 0.2 0.0 4.1 214.0 213.9 4.62 3.99 3 1.541 1004K 524 0.0 0.0 L4 116/0 9.66 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1036/0 85.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1241/41 97.92 GB 0.0 2.2 0.5 1.7 4.0 2.3 0.4 2.2 152.7 277.3 14.64 13.31 34 0.431 2269K 1724 0.0 0.0 Int 0/0 0.00 KB 0.0 2.2 0.5 1.7 4.0 2.3 0.4 2.2 152.7 277.3 14.64 13.31 34 0.431 2269K 1724 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.2 0.5 1.7 2.2 0.5 0.0 0.0 235.1 234.9 9.51 8.74 8 1.188 2269K 1724 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 355.9 5.13 4.57 26 0.197 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.0 total, 5.9 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.96 GB write, 673.45 MB/s write, 2.18 GB read, 370.71 MB/s read, 14.6 seconds Interval compaction: 3.96 GB write, 692.15 MB/s write, 2.18 GB read, 381.00 MB/s read, 14.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 6, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 6, interval: 6 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 413253 Average: 2.2407 StdDev: 12.88 Min: 0 Median: 1.5094 Max: 3790 Percentiles: P50: 1.51 P75: 1.91 P99: 5.69 P99.9: 9.60 P99.99: 14.17 ------------------------------------------------------ [ 0, 1 ] 73928 17.889% 17.889% #### ( 1, 2 ] 260486 63.033% 80.922% ############# ( 2, 3 ] 46969 11.366% 92.288% ## ( 3, 4 ] 17409 4.213% 96.501% # ( 4, 6 ] 12247 2.964% 99.464% # ( 6, 10 ] 2000 0.484% 99.948% ( 10, 15 ] 207 0.050% 99.998% ( 15, 22 ] 23 0.006% 100.004% ( 22, 34 ] 8 0.002% 100.006% ( 34, 51 ] 5 0.001% 100.007% ( 51, 76 ] 1 0.000% 100.007% ( 110, 170 ] 13 0.003% 100.010% ( 170, 250 ] 13 0.003% 100.014% ( 250, 380 ] 14 0.003% 100.017% ( 380, 580 ] 2 0.000% 100.017% ( 580, 870 ] 5 0.001% 100.019% ( 870, 1300 ] 5 0.001% 100.020% ( 1300, 1900 ] 2 0.000% 100.020% ( 1900, 2900 ] 2 0.000% 100.021% ( 2900, 4400 ] 3 0.001% 100.022% ** Level 1 read latency histogram (micros): Count: 54377 Average: 2.6561 StdDev: 15.43 Min: 1 Median: 1.6209 Max: 2435 Percentiles: P50: 1.62 P75: 2.16 P99: 7.08 P99.9: 50.84 P99.99: 620.77 ------------------------------------------------------ [ 0, 1 ] 7233 13.302% 13.302% ### ( 1, 2 ] 32140 59.106% 72.407% ############ ( 2, 3 ] 8990 16.533% 88.940% ### ( 3, 4 ] 2794 5.138% 94.078% # ( 4, 6 ] 2550 4.689% 98.768% # ( 6, 10 ] 466 0.857% 99.625% ( 10, 15 ] 99 0.182% 99.807% ( 15, 22 ] 9 0.017% 99.823% ( 22, 34 ] 1 0.002% 99.825% ( 34, 51 ] 41 0.075% 99.901% ( 51, 76 ] 23 0.042% 99.943% ( 76, 110 ] 5 0.009% 99.952% ( 110, 170 ] 1 0.002% 99.954% ( 170, 250 ] 11 0.020% 99.974% ( 250, 380 ] 3 0.006% 99.980% ( 380, 580 ] 5 0.009% 99.989% ( 580, 870 ] 4 0.007% 99.996% ( 1300, 1900 ] 1 0.002% 99.998% ( 1900, 2900 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 28300 Average: 2.0130 StdDev: 7.10 Min: 1 Median: 1.3743 Max: 900 Percentiles: P50: 1.37 P75: 1.79 P99: 5.37 P99.9: 9.59 P99.99: 183.60 ------------------------------------------------------ [ 0, 1 ] 7788 27.519% 27.519% ###### ( 1, 2 ] 16997 60.060% 87.580% ############ ( 2, 3 ] 1989 7.028% 94.608% # ( 3, 4 ] 865 3.057% 97.664% # ( 4, 6 ] 551 1.947% 99.611% ( 6, 10 ] 91 0.322% 99.933% ( 10, 15 ] 11 0.039% 99.972% ( 15, 22 ] 3 0.011% 99.982% ( 34, 51 ] 1 0.004% 99.986% ( 110, 170 ] 1 0.004% 99.989% ( 170, 250 ] 1 0.004% 99.993% ( 580, 870 ] 1 0.004% 99.996% ( 870, 1300 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 194051 Average: 3.4123 StdDev: 34.05 Min: 1 Median: 1.6428 Max: 4778 Percentiles: P50: 1.64 P75: 2.20 P99: 9.07 P99.9: 217.02 P99.99: 742.79 ------------------------------------------------------ [ 0, 1 ] 21645 11.154% 11.154% ## ( 1, 2 ] 117266 60.431% 71.585% ############ ( 2, 3 ] 32657 16.829% 88.414% ### ( 3, 4 ] 10199 5.256% 93.670% # ( 4, 6 ] 7799 4.019% 97.689% # ( 6, 10 ] 3320 1.711% 99.400% ( 10, 15 ] 409 0.211% 99.610% ( 15, 22 ] 120 0.062% 99.672% ( 22, 34 ] 28 0.014% 99.687% ( 34, 51 ] 8 0.004% 99.691% ( 51, 76 ] 5 0.003% 99.693% ( 76, 110 ] 131 0.068% 99.761% ( 110, 170 ] 210 0.108% 99.869% ( 170, 250 ] 102 0.053% 99.922% ( 250, 380 ] 65 0.033% 99.955% ( 380, 580 ] 53 0.027% 99.982% ( 580, 870 ] 26 0.013% 99.996% ( 870, 1300 ] 11 0.006% 100.002% ( 1300, 1900 ] 4 0.002% 100.004% ( 1900, 2900 ] 6 0.003% 100.007% ( 2900, 4400 ] 8 0.004% 100.011% ( 4400, 6600 ] 1 0.001% 100.011% ** Level 4 read latency histogram (micros): Count: 35 Average: 1254.7143 StdDev: 953.45 Min: 56 Median: 1146.4286 Max: 4112 Percentiles: P50: 1146.43 P75: 2056.25 P99: 4112.00 P99.9: 4112.00 P99.99: 4112.00 ------------------------------------------------------ ( 51, 76 ] 1 2.857% 2.857% # ( 76, 110 ] 1 2.857% 5.714% # ( 110, 170 ] 3 8.571% 14.286% ## ( 170, 250 ] 3 8.571% 22.857% ## ( 250, 380 ] 2 5.714% 28.571% # ( 380, 580 ] 2 5.714% 34.286% # ( 580, 870 ] 1 2.857% 37.143% # ( 870, 1300 ] 7 20.000% 57.143% #### ( 1300, 1900 ] 5 14.286% 71.429% ### ( 1900, 2900 ] 8 22.857% 94.286% ##### ( 2900, 4400 ] 2 5.714% 100.000% # ** DB Stats ** Uptime(secs): 6.0 total, 3.1 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 326.49 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.569 H:M:S, 9.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 320.04 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.386 H:M:S, 12.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/08/11-14:52:41 ... thread 0: (1000000,3000000) ops and (330169.9,337333.0) ops/second in (3.028744,8.893289) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/4 1.54 GB 125.9 1.6 0.0 1.6 4.4 2.8 0.0 1.6 115.1 314.3 14.23 13.23 46 0.309 1663K 1773 0.0 0.0 L1 9/0 942.41 MB 94.2 1.2 1.1 0.1 1.2 1.1 0.1 1.1 237.7 237.0 5.04 4.89 2 2.520 1216K 3660 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 0.3 1.8 235.5 235.5 0.47 0.45 1 0.473 113K 22 0.0 0.0 L3 39/0 995.36 MB 1.0 1.2 0.4 0.8 1.2 0.4 0.0 3.3 216.1 216.0 5.60 4.91 4 1.399 1199K 524 0.0 0.0 L4 128/0 9.72 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1036/0 85.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1222/4 98.90 GB 0.0 4.1 1.5 2.5 6.8 4.3 0.5 2.5 164.1 275.7 25.34 23.48 53 0.478 4192K 5979 0.0 0.0 Int 0/0 0.00 KB 0.0 4.1 1.5 2.5 6.8 4.3 0.4 2.5 164.1 275.7 25.34 23.48 53 0.478 4192K 5979 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.1 1.5 2.5 4.1 1.5 0.0 0.0 238.8 238.4 17.41 16.42 13 1.339 4192K 5979 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 357.7 7.93 7.06 40 0.198 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.1 total, 8.9 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.82 GB write, 771.24 MB/s write, 4.06 GB read, 458.95 MB/s read, 25.3 seconds Interval compaction: 6.82 GB write, 785.36 MB/s write, 4.06 GB read, 467.36 MB/s read, 25.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 9, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 9, interval: 9 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 745408 Average: 2.2130 StdDev: 10.37 Min: 0 Median: 1.5046 Max: 3790 Percentiles: P50: 1.50 P75: 1.90 P99: 5.64 P99.9: 9.70 P99.99: 14.37 ------------------------------------------------------ [ 0, 1 ] 132142 17.727% 17.727% #### ( 1, 2 ] 476748 63.958% 81.685% ############# ( 2, 3 ] 83489 11.200% 92.886% ## ( 3, 4 ] 29308 3.932% 96.818% # ( 4, 6 ] 19781 2.654% 99.471% # ( 6, 10 ] 3458 0.464% 99.935% ( 10, 15 ] 467 0.063% 99.998% ( 15, 22 ] 71 0.010% 100.008% ( 22, 34 ] 11 0.001% 100.009% ( 34, 51 ] 8 0.001% 100.010% ( 51, 76 ] 7 0.001% 100.011% ( 76, 110 ] 32 0.004% 100.015% ( 110, 170 ] 34 0.005% 100.020% ( 170, 250 ] 18 0.002% 100.022% ( 250, 380 ] 28 0.004% 100.026% ( 380, 580 ] 7 0.001% 100.027% ( 580, 870 ] 10 0.001% 100.028% ( 870, 1300 ] 7 0.001% 100.029% ( 1300, 1900 ] 3 0.000% 100.030% ( 1900, 2900 ] 2 0.000% 100.030% ( 2900, 4400 ] 3 0.000% 100.030% ** Level 1 read latency histogram (micros): Count: 76289 Average: 2.8806 StdDev: 18.36 Min: 0 Median: 1.6293 Max: 2435 Percentiles: P50: 1.63 P75: 2.14 P99: 6.87 P99.9: 159.04 P99.99: 793.76 ------------------------------------------------------ [ 0, 1 ] 8752 11.472% 11.472% ## ( 1, 2 ] 46704 61.220% 72.692% ############ ( 2, 3 ] 12594 16.508% 89.200% ### ( 3, 4 ] 4044 5.301% 94.501% # ( 4, 6 ] 3311 4.340% 98.841% # ( 6, 10 ] 560 0.734% 99.575% ( 10, 15 ] 136 0.178% 99.754% ( 15, 22 ] 13 0.017% 99.771% ( 22, 34 ] 1 0.001% 99.772% ( 34, 51 ] 41 0.054% 99.826% ( 51, 76 ] 23 0.030% 99.856% ( 76, 110 ] 19 0.025% 99.881% ( 110, 170 ] 18 0.024% 99.904% ( 170, 250 ] 29 0.038% 99.942% ( 250, 380 ] 12 0.016% 99.958% ( 380, 580 ] 17 0.022% 99.980% ( 580, 870 ] 10 0.013% 99.993% ( 870, 1300 ] 2 0.003% 99.996% ( 1300, 1900 ] 2 0.003% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 28300 Average: 2.0130 StdDev: 7.10 Min: 1 Median: 1.3743 Max: 900 Percentiles: P50: 1.37 P75: 1.79 P99: 5.37 P99.9: 9.59 P99.99: 183.60 ------------------------------------------------------ [ 0, 1 ] 7788 27.519% 27.519% ###### ( 1, 2 ] 16997 60.060% 87.580% ############ ( 2, 3 ] 1989 7.028% 94.608% # ( 3, 4 ] 865 3.057% 97.664% # ( 4, 6 ] 551 1.947% 99.611% ( 6, 10 ] 91 0.322% 99.933% ( 10, 15 ] 11 0.039% 99.972% ( 15, 22 ] 3 0.011% 99.982% ( 34, 51 ] 1 0.004% 99.986% ( 110, 170 ] 1 0.004% 99.989% ( 170, 250 ] 1 0.004% 99.993% ( 580, 870 ] 1 0.004% 99.996% ( 870, 1300 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 215792 Average: 3.3102 StdDev: 32.33 Min: 0 Median: 1.6408 Max: 4778 Percentiles: P50: 1.64 P75: 2.18 P99: 8.89 P99.9: 202.78 P99.99: 718.54 ------------------------------------------------------ [ 0, 1 ] 23434 10.860% 10.860% ## ( 1, 2 ] 131813 61.083% 71.943% ############ ( 2, 3 ] 36101 16.730% 88.672% ### ( 3, 4 ] 11300 5.237% 93.909% # ( 4, 6 ] 8466 3.923% 97.832% # ( 6, 10 ] 3484 1.615% 99.447% ( 10, 15 ] 432 0.200% 99.647% ( 15, 22 ] 122 0.057% 99.703% ( 22, 34 ] 28 0.013% 99.716% ( 34, 51 ] 8 0.004% 99.720% ( 51, 76 ] 5 0.002% 99.722% ( 76, 110 ] 131 0.061% 99.783% ( 110, 170 ] 210 0.097% 99.880% ( 170, 250 ] 103 0.048% 99.928% ( 250, 380 ] 67 0.031% 99.959% ( 380, 580 ] 54 0.025% 99.984% ( 580, 870 ] 26 0.012% 99.996% ( 870, 1300 ] 11 0.005% 100.001% ( 1300, 1900 ] 4 0.002% 100.003% ( 1900, 2900 ] 6 0.003% 100.006% ( 2900, 4400 ] 8 0.004% 100.010% ( 4400, 6600 ] 1 0.000% 100.010% ** Level 4 read latency histogram (micros): Count: 35 Average: 1254.7143 StdDev: 953.45 Min: 56 Median: 1146.4286 Max: 4112 Percentiles: P50: 1146.43 P75: 2056.25 P99: 4112.00 P99.9: 4112.00 P99.99: 4112.00 ------------------------------------------------------ ( 51, 76 ] 1 2.857% 2.857% # ( 76, 110 ] 1 2.857% 5.714% # ( 110, 170 ] 3 8.571% 14.286% ## ( 170, 250 ] 3 8.571% 22.857% ## ( 250, 380 ] 2 5.714% 28.571% # ( 380, 580 ] 2 5.714% 34.286% # ( 580, 870 ] 1 2.857% 37.143% # ( 870, 1300 ] 7 20.000% 57.143% #### ( 1300, 1900 ] 5 14.286% 71.429% ### ( 1900, 2900 ] 8 22.857% 94.286% ##### ( 2900, 4400 ] 2 5.714% 100.000% # ** DB Stats ** Uptime(secs): 9.1 total, 3.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 325.98 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.939 H:M:S, 10.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 324.95 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.370 H:M:S, 12.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/08/11-14:52:44 ... thread 0: (1000000,4000000) ops and (335301.0,336822.7) ops/second in (2.982395,11.875684) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/13 2.52 GB 18.9 2.4 0.0 2.4 6.1 3.7 0.0 1.6 121.9 312.4 20.16 18.79 63 0.320 2494K 2663 0.0 0.0 L1 9/9 942.41 MB 0.0 1.2 1.1 0.1 1.2 1.1 0.1 1.1 237.7 237.0 5.04 4.89 2 2.520 1216K 3660 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 0.3 1.8 235.5 235.5 0.47 0.45 1 0.473 113K 22 0.0 0.0 L3 39/0 995.36 MB 1.0 1.2 0.4 0.8 1.2 0.4 0.0 3.3 216.1 216.0 5.60 4.91 4 1.399 1199K 524 0.0 0.0 L4 128/0 9.72 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1036/0 85.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1227/22 99.88 GB 0.0 4.9 1.5 3.3 8.6 5.3 0.5 2.3 159.1 281.8 31.27 29.04 70 0.447 5024K 6869 0.0 0.0 Int 0/0 0.00 KB 0.0 4.9 1.5 3.3 8.6 5.3 0.4 2.3 159.1 281.8 31.27 29.04 70 0.447 5024K 6869 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.9 1.5 3.3 4.9 1.5 0.0 0.0 242.6 242.2 20.51 19.43 16 1.282 5024K 6869 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 357.2 10.76 9.61 54 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.0 total, 11.9 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.60 GB write, 731.86 MB/s write, 4.86 GB read, 413.29 MB/s read, 31.3 seconds Interval compaction: 8.60 GB write, 741.89 MB/s write, 4.86 GB read, 418.96 MB/s read, 31.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 13, interval: 13 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 967863 Average: 2.1884 StdDev: 13.77 Min: 0 Median: 1.4626 Max: 4059 Percentiles: P50: 1.46 P75: 1.86 P99: 5.57 P99.9: 9.88 P99.99: 21.15 ------------------------------------------------------ [ 0, 1 ] 200331 20.698% 20.698% #### ( 1, 2 ] 613065 63.342% 84.040% ############# ( 2, 3 ] 94727 9.787% 93.828% ## ( 3, 4 ] 33161 3.426% 97.254% # ( 4, 6 ] 21543 2.226% 99.480% ( 6, 10 ] 4192 0.433% 99.913% ( 10, 15 ] 662 0.068% 99.981% ( 15, 22 ] 97 0.010% 99.991% ( 22, 34 ] 15 0.002% 99.993% ( 34, 51 ] 8 0.001% 99.994% ( 51, 76 ] 10 0.001% 99.995% ( 76, 110 ] 64 0.007% 100.001% ( 110, 170 ] 95 0.010% 100.011% ( 170, 250 ] 43 0.004% 100.015% ( 250, 380 ] 43 0.004% 100.020% ( 380, 580 ] 20 0.002% 100.022% ( 580, 870 ] 14 0.001% 100.023% ( 870, 1300 ] 10 0.001% 100.024% ( 1300, 1900 ] 4 0.000% 100.025% ( 1900, 2900 ] 4 0.000% 100.025% ( 2900, 4400 ] 9 0.001% 100.026% ** Level 1 read latency histogram (micros): Count: 86131 Average: 2.8953 StdDev: 18.45 Min: 0 Median: 1.5898 Max: 2435 Percentiles: P50: 1.59 P75: 2.00 P99: 7.20 P99.9: 185.77 P99.99: 801.11 ------------------------------------------------------ [ 0, 1 ] 11976 13.904% 13.904% ### ( 1, 2 ] 52713 61.201% 75.105% ############ ( 2, 3 ] 12854 14.924% 90.029% ### ( 3, 4 ] 4141 4.808% 94.837% # ( 4, 6 ] 3388 3.934% 98.770% # ( 6, 10 ] 659 0.765% 99.536% ( 10, 15 ] 147 0.171% 99.706% ( 15, 22 ] 14 0.016% 99.723% ( 22, 34 ] 1 0.001% 99.724% ( 34, 51 ] 41 0.048% 99.771% ( 51, 76 ] 28 0.033% 99.804% ( 76, 110 ] 25 0.029% 99.833% ( 110, 170 ] 49 0.057% 99.890% ( 170, 250 ] 45 0.052% 99.942% ( 250, 380 ] 13 0.015% 99.957% ( 380, 580 ] 20 0.023% 99.980% ( 580, 870 ] 11 0.013% 99.993% ( 870, 1300 ] 3 0.003% 99.997% ( 1300, 1900 ] 2 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 28300 Average: 2.0130 StdDev: 7.10 Min: 1 Median: 1.3743 Max: 900 Percentiles: P50: 1.37 P75: 1.79 P99: 5.37 P99.9: 9.59 P99.99: 183.60 ------------------------------------------------------ [ 0, 1 ] 7788 27.519% 27.519% ###### ( 1, 2 ] 16997 60.060% 87.580% ############ ( 2, 3 ] 1989 7.028% 94.608% # ( 3, 4 ] 865 3.057% 97.664% # ( 4, 6 ] 551 1.947% 99.611% ( 6, 10 ] 91 0.322% 99.933% ( 10, 15 ] 11 0.039% 99.972% ( 15, 22 ] 3 0.011% 99.982% ( 34, 51 ] 1 0.004% 99.986% ( 110, 170 ] 1 0.004% 99.989% ( 170, 250 ] 1 0.004% 99.993% ( 580, 870 ] 1 0.004% 99.996% ( 870, 1300 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 215792 Average: 3.3102 StdDev: 32.33 Min: 0 Median: 1.6408 Max: 4778 Percentiles: P50: 1.64 P75: 2.18 P99: 8.89 P99.9: 202.78 P99.99: 718.54 ------------------------------------------------------ [ 0, 1 ] 23434 10.860% 10.860% ## ( 1, 2 ] 131813 61.083% 71.943% ############ ( 2, 3 ] 36101 16.730% 88.672% ### ( 3, 4 ] 11300 5.237% 93.909% # ( 4, 6 ] 8466 3.923% 97.832% # ( 6, 10 ] 3484 1.615% 99.447% ( 10, 15 ] 432 0.200% 99.647% ( 15, 22 ] 122 0.057% 99.703% ( 22, 34 ] 28 0.013% 99.716% ( 34, 51 ] 8 0.004% 99.720% ( 51, 76 ] 5 0.002% 99.722% ( 76, 110 ] 131 0.061% 99.783% ( 110, 170 ] 210 0.097% 99.880% ( 170, 250 ] 103 0.048% 99.928% ( 250, 380 ] 67 0.031% 99.959% ( 380, 580 ] 54 0.025% 99.984% ( 580, 870 ] 26 0.012% 99.996% ( 870, 1300 ] 11 0.005% 100.001% ( 1300, 1900 ] 4 0.002% 100.003% ( 1900, 2900 ] 6 0.003% 100.006% ( 2900, 4400 ] 8 0.004% 100.010% ( 4400, 6600 ] 1 0.000% 100.010% ** Level 4 read latency histogram (micros): Count: 35 Average: 1254.7143 StdDev: 953.45 Min: 56 Median: 1146.4286 Max: 4112 Percentiles: P50: 1146.43 P75: 2056.25 P99: 4112.00 P99.9: 4112.00 P99.99: 4112.00 ------------------------------------------------------ ( 51, 76 ] 1 2.857% 2.857% # ( 76, 110 ] 1 2.857% 5.714% # ( 110, 170 ] 3 8.571% 14.286% ## ( 170, 250 ] 3 8.571% 22.857% ## ( 250, 380 ] 2 5.714% 28.571% # ( 380, 580 ] 2 5.714% 34.286% # ( 580, 870 ] 1 2.857% 37.143% # ( 870, 1300 ] 7 20.000% 57.143% #### ( 1300, 1900 ] 5 14.286% 71.429% ### ( 1900, 2900 ] 8 22.857% 94.286% ##### ( 2900, 4400 ] 2 5.714% 100.000% # ** DB Stats ** Uptime(secs): 12.0 total, 3.0 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 326.97 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.306 H:M:S, 10.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 330.00 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.367 H:M:S, 12.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/08/11-14:52:49 ... thread 0: (1000000,5000000) ops and (198503.4,295623.9) ops/second in (5.037696,16.913380) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/17 3.50 GB 37.8 3.4 0.0 3.4 8.2 4.7 0.0 1.7 130.2 309.1 27.09 25.33 83 0.326 3582K 3832 0.0 0.0 L1 9/9 942.41 MB 0.0 1.2 1.1 0.1 1.2 1.1 0.1 1.1 237.7 237.0 5.04 4.89 2 2.520 1216K 3660 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 0.3 1.8 235.5 235.5 0.47 0.45 1 0.473 113K 22 0.0 0.0 L3 39/0 995.36 MB 1.0 1.2 0.4 0.8 1.2 0.4 0.0 3.3 216.1 216.0 5.60 4.91 4 1.399 1199K 524 0.0 0.0 L4 128/0 9.72 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1036/0 85.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1231/26 100.87 GB 0.0 5.9 1.5 4.4 10.6 6.2 0.5 2.2 158.3 285.0 38.20 35.58 90 0.424 6111K 8038 0.0 0.0 Int 0/0 0.00 KB 0.0 5.9 1.5 4.4 10.6 6.2 0.4 2.2 158.3 285.0 38.20 35.58 90 0.424 6111K 8038 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.9 1.5 4.4 5.9 1.5 0.0 0.0 246.7 246.3 24.51 23.37 20 1.226 6111K 8038 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 354.3 13.69 12.21 70 0.196 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 17.1 total, 16.9 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.63 GB write, 637.63 MB/s write, 5.91 GB read, 354.09 MB/s read, 38.2 seconds Interval compaction: 10.63 GB write, 643.77 MB/s write, 5.91 GB read, 357.49 MB/s read, 38.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 7, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 7, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 7, total-stops: 13, interval: 20 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1467616 Average: 2.1935 StdDev: 12.89 Min: 0 Median: 1.4722 Max: 4059 Percentiles: P50: 1.47 P75: 1.86 P99: 5.55 P99.9: 9.95 P99.99: 20.47 ------------------------------------------------------ [ 0, 1 ] 285277 19.438% 19.438% #### ( 1, 2 ] 949900 64.724% 84.162% ############# ( 2, 3 ] 145638 9.923% 94.086% ## ( 3, 4 ] 49037 3.341% 97.427% # ( 4, 6 ] 29864 2.035% 99.462% ( 6, 10 ] 6509 0.444% 99.905% ( 10, 15 ] 1136 0.077% 99.983% ( 15, 22 ] 141 0.010% 99.992% ( 22, 34 ] 19 0.001% 99.994% ( 34, 51 ] 10 0.001% 99.994% ( 51, 76 ] 13 0.001% 99.995% ( 76, 110 ] 111 0.008% 100.003% ( 110, 170 ] 142 0.010% 100.012% ( 170, 250 ] 78 0.005% 100.018% ( 250, 380 ] 52 0.004% 100.021% ( 380, 580 ] 40 0.003% 100.024% ( 580, 870 ] 23 0.002% 100.025% ( 870, 1300 ] 17 0.001% 100.027% ( 1300, 1900 ] 5 0.000% 100.027% ( 1900, 2900 ] 7 0.000% 100.027% ( 2900, 4400 ] 11 0.001% 100.028% ** Level 1 read latency histogram (micros): Count: 168749 Average: 2.6280 StdDev: 14.04 Min: 0 Median: 1.5413 Max: 2435 Percentiles: P50: 1.54 P75: 1.93 P99: 7.59 P99.9: 128.80 P99.99: 607.19 ------------------------------------------------------ [ 0, 1 ] 26198 15.525% 15.525% ### ( 1, 2 ] 107485 63.695% 79.220% ############# ( 2, 3 ] 21312 12.629% 91.849% ### ( 3, 4 ] 6483 3.842% 95.691% # ( 4, 6 ] 4903 2.905% 98.597% # ( 6, 10 ] 1710 1.013% 99.610% ( 10, 15 ] 266 0.158% 99.768% ( 15, 22 ] 24 0.014% 99.782% ( 22, 34 ] 2 0.001% 99.783% ( 34, 51 ] 41 0.024% 99.807% ( 51, 76 ] 32 0.019% 99.826% ( 76, 110 ] 102 0.060% 99.887% ( 110, 170 ] 71 0.042% 99.929% ( 170, 250 ] 58 0.034% 99.963% ( 250, 380 ] 18 0.011% 99.974% ( 380, 580 ] 26 0.015% 99.989% ( 580, 870 ] 12 0.007% 99.996% ( 870, 1300 ] 3 0.002% 99.998% ( 1300, 1900 ] 2 0.001% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 28300 Average: 2.0130 StdDev: 7.10 Min: 1 Median: 1.3743 Max: 900 Percentiles: P50: 1.37 P75: 1.79 P99: 5.37 P99.9: 9.59 P99.99: 183.60 ------------------------------------------------------ [ 0, 1 ] 7788 27.519% 27.519% ###### ( 1, 2 ] 16997 60.060% 87.580% ############ ( 2, 3 ] 1989 7.028% 94.608% # ( 3, 4 ] 865 3.057% 97.664% # ( 4, 6 ] 551 1.947% 99.611% ( 6, 10 ] 91 0.322% 99.933% ( 10, 15 ] 11 0.039% 99.972% ( 15, 22 ] 3 0.011% 99.982% ( 34, 51 ] 1 0.004% 99.986% ( 110, 170 ] 1 0.004% 99.989% ( 170, 250 ] 1 0.004% 99.993% ( 580, 870 ] 1 0.004% 99.996% ( 870, 1300 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 215792 Average: 3.3102 StdDev: 32.33 Min: 0 Median: 1.6408 Max: 4778 Percentiles: P50: 1.64 P75: 2.18 P99: 8.89 P99.9: 202.78 P99.99: 718.54 ------------------------------------------------------ [ 0, 1 ] 23434 10.860% 10.860% ## ( 1, 2 ] 131813 61.083% 71.943% ############ ( 2, 3 ] 36101 16.730% 88.672% ### ( 3, 4 ] 11300 5.237% 93.909% # ( 4, 6 ] 8466 3.923% 97.832% # ( 6, 10 ] 3484 1.615% 99.447% ( 10, 15 ] 432 0.200% 99.647% ( 15, 22 ] 122 0.057% 99.703% ( 22, 34 ] 28 0.013% 99.716% ( 34, 51 ] 8 0.004% 99.720% ( 51, 76 ] 5 0.002% 99.722% ( 76, 110 ] 131 0.061% 99.783% ( 110, 170 ] 210 0.097% 99.880% ( 170, 250 ] 103 0.048% 99.928% ( 250, 380 ] 67 0.031% 99.959% ( 380, 580 ] 54 0.025% 99.984% ( 580, 870 ] 26 0.012% 99.996% ( 870, 1300 ] 11 0.005% 100.001% ( 1300, 1900 ] 4 0.002% 100.003% ( 1900, 2900 ] 6 0.003% 100.006% ( 2900, 4400 ] 8 0.004% 100.010% ( 4400, 6600 ] 1 0.000% 100.010% ** Level 4 read latency histogram (micros): Count: 35 Average: 1254.7143 StdDev: 953.45 Min: 56 Median: 1146.4286 Max: 4112 Percentiles: P50: 1146.43 P75: 2056.25 P99: 4112.00 P99.9: 4112.00 P99.99: 4112.00 ------------------------------------------------------ ( 51, 76 ] 1 2.857% 2.857% # ( 76, 110 ] 1 2.857% 5.714% # ( 110, 170 ] 3 8.571% 14.286% ## ( 170, 250 ] 3 8.571% 22.857% ## ( 250, 380 ] 2 5.714% 28.571% # ( 380, 580 ] 2 5.714% 34.286% # ( 580, 870 ] 1 2.857% 37.143% # ( 870, 1300 ] 7 20.000% 57.143% #### ( 1300, 1900 ] 5 14.286% 71.429% ### ( 1900, 2900 ] 8 22.857% 94.286% ##### ( 2900, 4400 ] 2 5.714% 100.000% # ** DB Stats ** Uptime(secs): 17.1 total, 5.0 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 288.15 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.642 H:M:S, 21.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 195.37 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.336 H:M:S, 46.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/08/11-14:52:59 ... thread 0: (1000000,6000000) ops and (100419.8,223284.3) ops/second in (9.958194,26.871574) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 8/4 2.03 GB 125.6 5.0 0.0 5.0 10.6 5.6 0.0 1.9 144.4 306.3 35.35 33.24 101 0.350 5180K 7024 0.0 0.0 L1 10/0 744.21 MB 74.4 4.4 3.4 1.0 4.4 3.4 0.1 1.3 224.6 222.0 20.18 18.94 3 6.726 4602K 53K 0.0 0.0 L2 27/9 1.89 GB 14.7 0.1 0.1 0.0 0.1 0.1 2.8 1.8 235.5 235.5 0.47 0.45 1 0.473 113K 22 0.0 0.0 L3 44/9 1.56 GB 1.6 1.2 0.4 0.8 1.2 0.4 0.6 3.3 216.1 216.0 5.60 4.91 4 1.399 1199K 524 0.0 0.0 L4 128/0 9.72 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1036/0 85.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1253/22 101.68 GB 0.0 10.7 3.9 6.8 16.2 9.4 3.5 2.9 177.9 269.9 61.59 57.54 109 0.565 11M 60K 0.0 0.0 Int 0/0 0.00 KB 0.0 10.7 3.9 6.8 16.2 9.4 3.5 2.9 177.9 269.9 61.59 57.54 109 0.565 11M 60K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.7 3.9 6.8 10.6 3.8 0.0 0.0 241.2 239.9 45.41 43.11 25 1.816 11M 60K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.6 5.6 0.0 0.0 0.0 354.3 16.18 14.43 84 0.193 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 27.0 total, 26.9 interval Flush(GB): cumulative 5.599, interval 5.599 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 16.24 GB write, 614.99 MB/s write, 10.70 GB read, 405.22 MB/s read, 61.6 seconds Interval compaction: 16.24 GB write, 618.72 MB/s write, 10.70 GB read, 407.68 MB/s read, 61.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 17, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 17, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 17, total-stops: 13, interval: 30 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2387240 Average: 2.4517 StdDev: 16.80 Min: 0 Median: 1.4688 Max: 8022 Percentiles: P50: 1.47 P75: 1.85 P99: 5.80 P99.9: 109.05 P99.99: 225.89 ------------------------------------------------------ [ 0, 1 ] 461893 19.348% 19.348% #### ( 1, 2 ] 1560975 65.388% 84.737% ############# ( 2, 3 ] 224573 9.407% 94.144% ## ( 3, 4 ] 75859 3.178% 97.322% # ( 4, 6 ] 44517 1.865% 99.186% ( 6, 10 ] 12839 0.538% 99.724% ( 10, 15 ] 2018 0.085% 99.809% ( 15, 22 ] 524 0.022% 99.831% ( 22, 34 ] 31 0.001% 99.832% ( 34, 51 ] 12 0.001% 99.832% ( 51, 76 ] 39 0.002% 99.834% ( 76, 110 ] 1618 0.068% 99.902% ( 110, 170 ] 1777 0.074% 99.976% ( 170, 250 ] 467 0.020% 99.996% ( 250, 380 ] 282 0.012% 100.008% ( 380, 580 ] 243 0.010% 100.018% ( 580, 870 ] 107 0.004% 100.022% ( 870, 1300 ] 39 0.002% 100.024% ( 1300, 1900 ] 11 0.000% 100.024% ( 1900, 2900 ] 12 0.001% 100.025% ( 2900, 4400 ] 16 0.001% 100.026% ( 4400, 6600 ] 1 0.000% 100.026% ( 6600, 9900 ] 2 0.000% 100.026% ** Level 1 read latency histogram (micros): Count: 401406 Average: 3.8064 StdDev: 65.12 Min: 0 Median: 1.6068 Max: 9630 Percentiles: P50: 1.61 P75: 2.08 P99: 8.11 P99.9: 126.61 P99.99: 228.80 ------------------------------------------------------ [ 0, 1 ] 52697 13.128% 13.128% ### ( 1, 2 ] 243911 60.764% 73.892% ############ ( 2, 3 ] 58026 14.456% 88.348% ### ( 3, 4 ] 21456 5.345% 93.693% # ( 4, 6 ] 18733 4.667% 98.360% # ( 6, 10 ] 4875 1.214% 99.574% ( 10, 15 ] 716 0.178% 99.753% ( 15, 22 ] 99 0.025% 99.778% ( 22, 34 ] 8 0.002% 99.780% ( 34, 51 ] 41 0.010% 99.790% ( 51, 76 ] 45 0.011% 99.801% ( 76, 110 ] 304 0.076% 99.877% ( 110, 170 ] 338 0.084% 99.961% ( 170, 250 ] 159 0.040% 100.000% ( 250, 380 ] 86 0.021% 100.022% ( 380, 580 ] 95 0.024% 100.046% ( 580, 870 ] 58 0.014% 100.060% ( 870, 1300 ] 26 0.006% 100.067% ( 1300, 1900 ] 20 0.005% 100.071% ( 1900, 2900 ] 13 0.003% 100.075% ( 2900, 4400 ] 32 0.008% 100.083% ( 4400, 6600 ] 14 0.003% 100.086% ( 6600, 9900 ] 10 0.002% 100.089% ** Level 2 read latency histogram (micros): Count: 28300 Average: 2.0130 StdDev: 7.10 Min: 1 Median: 1.3743 Max: 900 Percentiles: P50: 1.37 P75: 1.79 P99: 5.37 P99.9: 9.59 P99.99: 183.60 ------------------------------------------------------ [ 0, 1 ] 7788 27.519% 27.519% ###### ( 1, 2 ] 16997 60.060% 87.580% ############ ( 2, 3 ] 1989 7.028% 94.608% # ( 3, 4 ] 865 3.057% 97.664% # ( 4, 6 ] 551 1.947% 99.611% ( 6, 10 ] 91 0.322% 99.933% ( 10, 15 ] 11 0.039% 99.972% ( 15, 22 ] 3 0.011% 99.982% ( 34, 51 ] 1 0.004% 99.986% ( 110, 170 ] 1 0.004% 99.989% ( 170, 250 ] 1 0.004% 99.993% ( 580, 870 ] 1 0.004% 99.996% ( 870, 1300 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 223959 Average: 4.6026 StdDev: 53.89 Min: 0 Median: 1.6516 Max: 4920 Percentiles: P50: 1.65 P75: 2.23 P99: 9.53 P99.9: 493.53 P99.99: 1396.25 ------------------------------------------------------ [ 0, 1 ] 23607 10.541% 10.541% ## ( 1, 2 ] 135631 60.561% 71.101% ############ ( 2, 3 ] 37926 16.934% 88.036% ### ( 3, 4 ] 12273 5.480% 93.516% # ( 4, 6 ] 9057 4.044% 97.560% # ( 6, 10 ] 3660 1.634% 99.194% ( 10, 15 ] 465 0.208% 99.402% ( 15, 22 ] 124 0.055% 99.457% ( 22, 34 ] 28 0.013% 99.470% ( 34, 51 ] 8 0.004% 99.473% ( 51, 76 ] 5 0.002% 99.475% ( 76, 110 ] 151 0.067% 99.543% ( 110, 170 ] 289 0.129% 99.672% ( 170, 250 ] 211 0.094% 99.766% ( 250, 380 ] 199 0.089% 99.855% ( 380, 580 ] 178 0.079% 99.934% ( 580, 870 ] 87 0.039% 99.973% ( 870, 1300 ] 36 0.016% 99.989% ( 1300, 1900 ] 10 0.004% 99.994% ( 1900, 2900 ] 15 0.007% 100.000% ( 2900, 4400 ] 19 0.008% 100.009% ( 4400, 6600 ] 4 0.002% 100.011% ** Level 4 read latency histogram (micros): Count: 35 Average: 1254.7143 StdDev: 953.45 Min: 56 Median: 1146.4286 Max: 4112 Percentiles: P50: 1146.43 P75: 2056.25 P99: 4112.00 P99.9: 4112.00 P99.99: 4112.00 ------------------------------------------------------ ( 51, 76 ] 1 2.857% 2.857% # ( 76, 110 ] 1 2.857% 5.714% # ( 110, 170 ] 3 8.571% 14.286% ## ( 170, 250 ] 3 8.571% 22.857% ## ( 250, 380 ] 2 5.714% 28.571% # ( 380, 580 ] 2 5.714% 34.286% # ( 580, 870 ] 1 2.857% 37.143% # ( 870, 1300 ] 7 20.000% 57.143% #### ( 1300, 1900 ] 5 14.286% 71.429% ### ( 1900, 2900 ] 8 22.857% 94.286% ##### ( 2900, 4400 ] 2 5.714% 100.000% # ** DB Stats ** Uptime(secs): 27.0 total, 10.0 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 218.42 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:10.869 H:M:S, 40.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 98.83 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:7.226 H:M:S, 72.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/08/11-14:53:03 ... thread 0: (1000000,7000000) ops and (280066.0,229944.3) ops/second in (3.570587,30.442161) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/14 3.01 GB 37.8 6.0 0.0 6.0 12.6 6.6 0.0 1.9 142.5 297.9 43.31 39.99 117 0.370 6267K 9915 0.0 0.0 L1 10/10 744.21 MB 0.0 4.4 3.4 1.0 4.4 3.4 0.1 1.3 224.6 222.0 20.18 18.94 3 6.726 4602K 53K 0.0 0.0 L2 7/7 901.46 MB 0.0 0.1 0.1 0.0 0.1 0.1 2.8 1.8 235.5 235.5 0.47 0.45 1 0.473 113K 22 0.0 0.0 L3 20/1 1.07 GB 1.0 2.3 1.4 0.9 2.3 1.4 0.6 1.7 151.5 151.2 15.45 10.25 24 0.644 2236K 1832 0.0 0.0 L4 151/0 9.55 GB 1.0 0.0 0.0 0.0 0.0 0.0 1.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1056/0 87.42 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1260/32 102.65 GB 0.0 12.8 4.9 8.0 19.4 11.4 6.7 2.9 165.7 249.7 79.41 69.62 145 0.548 13M 65K 0.0 0.0 Int 0/0 0.00 KB 0.0 12.8 4.9 8.0 19.4 11.4 6.7 2.9 165.7 249.7 79.41 69.62 145 0.548 13M 65K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.8 4.9 8.0 12.8 4.8 0.0 0.0 220.1 219.0 59.76 52.61 47 1.272 13M 65K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 343.1 19.65 17.01 98 0.200 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 30.6 total, 30.4 interval Flush(GB): cumulative 6.583, interval 6.583 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 19.36 GB write, 647.89 MB/s write, 12.85 GB read, 429.86 MB/s read, 79.4 seconds Interval compaction: 19.36 GB write, 651.35 MB/s write, 12.85 GB read, 432.16 MB/s read, 79.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 17, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 17, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 40, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 17, total-stops: 40, interval: 57 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2705587 Average: 2.5152 StdDev: 18.07 Min: 0 Median: 1.4903 Max: 8022 Percentiles: P50: 1.49 P75: 1.87 P99: 5.91 P99.9: 106.50 P99.99: 242.96 ------------------------------------------------------ [ 0, 1 ] 486798 17.992% 17.992% #### ( 1, 2 ] 1766272 65.282% 83.275% ############# ( 2, 3 ] 275286 10.175% 93.449% ## ( 3, 4 ] 94190 3.481% 96.931% # ( 4, 6 ] 58726 2.171% 99.101% ( 6, 10 ] 16982 0.628% 99.729% ( 10, 15 ] 2507 0.093% 99.822% ( 15, 22 ] 569 0.021% 99.843% ( 22, 34 ] 35 0.001% 99.844% ( 34, 51 ] 12 0.000% 99.844% ( 51, 76 ] 40 0.001% 99.846% ( 76, 110 ] 1637 0.061% 99.906% ( 110, 170 ] 1805 0.067% 99.973% ( 170, 250 ] 506 0.019% 99.992% ( 250, 380 ] 336 0.012% 100.004% ( 380, 580 ] 302 0.011% 100.015% ( 580, 870 ] 154 0.006% 100.021% ( 870, 1300 ] 69 0.003% 100.024% ( 1300, 1900 ] 27 0.001% 100.025% ( 1900, 2900 ] 19 0.001% 100.025% ( 2900, 4400 ] 20 0.001% 100.026% ( 4400, 6600 ] 1 0.000% 100.026% ( 6600, 9900 ] 2 0.000% 100.026% ** Level 1 read latency histogram (micros): Count: 719363 Average: 5.1436 StdDev: 82.23 Min: 0 Median: 1.7076 Max: 9630 Percentiles: P50: 1.71 P75: 2.43 P99: 8.76 P99.9: 152.98 P99.99: 418.73 ------------------------------------------------------ [ 0, 1 ] 62092 8.632% 8.632% ## ( 1, 2 ] 420542 58.460% 67.092% ############ ( 2, 3 ] 132393 18.404% 85.496% #### ( 3, 4 ] 45934 6.385% 91.881% # ( 4, 6 ] 43000 5.978% 97.859% # ( 6, 10 ] 11901 1.654% 99.513% ( 10, 15 ] 1693 0.235% 99.749% ( 15, 22 ] 255 0.035% 99.784% ( 22, 34 ] 34 0.005% 99.789% ( 34, 51 ] 50 0.007% 99.796% ( 51, 76 ] 51 0.007% 99.803% ( 76, 110 ] 373 0.052% 99.855% ( 110, 170 ] 456 0.063% 99.918% ( 170, 250 ] 248 0.034% 99.953% ( 250, 380 ] 211 0.029% 99.982% ( 380, 580 ] 305 0.042% 100.024% ( 580, 870 ] 252 0.035% 100.059% ( 870, 1300 ] 159 0.022% 100.081% ( 1300, 1900 ] 95 0.013% 100.095% ( 1900, 2900 ] 88 0.012% 100.107% ( 2900, 4400 ] 142 0.020% 100.127% ( 4400, 6600 ] 41 0.006% 100.132% ( 6600, 9900 ] 13 0.002% 100.134% ** Level 2 read latency histogram (micros): Count: 28300 Average: 2.0130 StdDev: 7.10 Min: 1 Median: 1.3743 Max: 900 Percentiles: P50: 1.37 P75: 1.79 P99: 5.37 P99.9: 9.59 P99.99: 183.60 ------------------------------------------------------ [ 0, 1 ] 7788 27.519% 27.519% ###### ( 1, 2 ] 16997 60.060% 87.580% ############ ( 2, 3 ] 1989 7.028% 94.608% # ( 3, 4 ] 865 3.057% 97.664% # ( 4, 6 ] 551 1.947% 99.611% ( 6, 10 ] 91 0.322% 99.933% ( 10, 15 ] 11 0.039% 99.972% ( 15, 22 ] 3 0.011% 99.982% ( 34, 51 ] 1 0.004% 99.986% ( 110, 170 ] 1 0.004% 99.989% ( 170, 250 ] 1 0.004% 99.993% ( 580, 870 ] 1 0.004% 99.996% ( 870, 1300 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 254758 Average: 7.4189 StdDev: 93.45 Min: 0 Median: 1.6682 Max: 5589 Percentiles: P50: 1.67 P75: 2.29 P99: 11.49 P99.9: 1088.66 P99.99: 3630.99 ------------------------------------------------------ [ 0, 1 ] 25103 9.854% 9.854% ## ( 1, 2 ] 153068 60.084% 69.937% ############ ( 2, 3 ] 44219 17.357% 87.295% ### ( 3, 4 ] 14746 5.788% 93.083% # ( 4, 6 ] 10663 4.186% 97.268% # ( 6, 10 ] 4249 1.668% 98.936% ( 10, 15 ] 545 0.214% 99.150% ( 15, 22 ] 132 0.052% 99.202% ( 22, 34 ] 30 0.012% 99.214% ( 34, 51 ] 11 0.004% 99.218% ( 51, 76 ] 5 0.002% 99.220% ( 76, 110 ] 161 0.063% 99.283% ( 110, 170 ] 327 0.128% 99.412% ( 170, 250 ] 301 0.118% 99.530% ( 250, 380 ] 300 0.118% 99.648% ( 380, 580 ] 336 0.132% 99.779% ( 580, 870 ] 233 0.091% 99.871% ( 870, 1300 ] 146 0.057% 99.928% ( 1300, 1900 ] 74 0.029% 99.957% ( 1900, 2900 ] 46 0.018% 99.975% ( 2900, 4400 ] 77 0.030% 100.005% ( 4400, 6600 ] 12 0.005% 100.010% ** Level 4 read latency histogram (micros): Count: 35 Average: 1254.7143 StdDev: 953.45 Min: 56 Median: 1146.4286 Max: 4112 Percentiles: P50: 1146.43 P75: 2056.25 P99: 4112.00 P99.9: 4112.00 P99.99: 4112.00 ------------------------------------------------------ ( 51, 76 ] 1 2.857% 2.857% # ( 76, 110 ] 1 2.857% 5.714% # ( 110, 170 ] 3 8.571% 14.286% ## ( 170, 250 ] 3 8.571% 22.857% ## ( 250, 380 ] 2 5.714% 28.571% # ( 380, 580 ] 2 5.714% 34.286% # ( 580, 870 ] 1 2.857% 37.143% # ( 870, 1300 ] 7 20.000% 57.143% #### ( 1300, 1900 ] 5 14.286% 71.429% ### ( 1900, 2900 ] 8 22.857% 94.286% ##### ( 2900, 4400 ] 2 5.714% 100.000% # ** Level 5 read latency histogram (micros): Count: 100 Average: 1267.3800 StdDev: 1340.77 Min: 77 Median: 568.2353 Max: 6166 Percentiles: P50: 568.24 P75: 1850.00 P99: 6050.00 P99.9: 6166.00 P99.99: 6166.00 ------------------------------------------------------ ( 76, 110 ] 4 4.000% 4.000% # ( 110, 170 ] 6 6.000% 10.000% # ( 170, 250 ] 9 9.000% 19.000% ## ( 250, 380 ] 15 15.000% 34.000% ### ( 380, 580 ] 17 17.000% 51.000% ### ( 580, 870 ] 5 5.000% 56.000% # ( 870, 1300 ] 8 8.000% 64.000% ## ( 1300, 1900 ] 12 12.000% 76.000% ## ( 1900, 2900 ] 7 7.000% 83.000% # ( 2900, 4400 ] 13 13.000% 96.000% ### ( 4400, 6600 ] 4 4.000% 100.000% # ** DB Stats ** Uptime(secs): 30.6 total, 3.6 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 225.09 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:11.769 H:M:S, 38.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 275.64 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.900 H:M:S, 25.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 4 num-running-flushes: 1 2024/08/11-14:53:06 ... thread 0: (1000000,8000000) ops and (286964.4,235801.0) ops/second in (3.484753,33.926914) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/10 3.99 GB 169.9 7.1 0.0 7.1 14.7 7.6 0.0 1.9 144.5 297.6 50.56 46.80 135 0.374 7418K 11K 0.0 0.0 L1 10/10 744.21 MB 0.0 4.4 3.4 1.0 4.4 3.4 0.1 1.3 224.6 222.0 20.18 18.94 3 6.726 4602K 53K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 2.8 1.8 235.5 235.5 0.47 0.45 1 0.473 113K 22 0.0 0.0 L3 21/0 892.67 MB 0.9 3.3 2.2 1.0 3.3 2.2 0.6 1.5 166.1 165.7 20.27 14.68 25 0.811 2362K 1832 0.0 0.0 L4 163/0 9.64 GB 1.0 0.0 0.0 0.0 0.0 0.0 2.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1068/0 88.40 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1280/20 103.63 GB 0.0 15.0 5.7 9.2 22.5 13.2 8.8 3.0 167.4 251.4 91.47 80.86 164 0.558 14M 66K 0.0 0.0 Int 0/0 0.00 KB 0.0 15.0 5.7 9.2 22.5 13.2 8.7 3.0 167.4 251.4 91.47 80.86 164 0.558 14M 66K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 15.0 5.7 9.2 14.9 5.7 0.0 0.0 222.1 221.1 68.96 61.33 52 1.326 14M 66K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 344.2 22.51 19.53 112 0.201 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 34.1 total, 33.9 interval Flush(GB): cumulative 7.567, interval 7.567 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 22.45 GB write, 674.45 MB/s write, 14.96 GB read, 449.26 MB/s read, 91.5 seconds Interval compaction: 22.45 GB write, 677.68 MB/s write, 14.96 GB read, 451.41 MB/s read, 91.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 19, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 19, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 43, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 19, total-stops: 43, interval: 62 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3169618 Average: 2.5117 StdDev: 17.11 Min: 0 Median: 1.5065 Max: 8022 Percentiles: P50: 1.51 P75: 1.89 P99: 5.93 P99.9: 97.90 P99.99: 220.36 ------------------------------------------------------ [ 0, 1 ] 533274 16.825% 16.825% ### ( 1, 2 ] 2076067 65.499% 82.324% ############# ( 2, 3 ] 339735 10.718% 93.042% ## ( 3, 4 ] 117490 3.707% 96.749% # ( 4, 6 ] 73871 2.331% 99.079% ( 6, 10 ] 21310 0.672% 99.752% ( 10, 15 ] 2926 0.092% 99.844% ( 15, 22 ] 602 0.019% 99.863% ( 22, 34 ] 38 0.001% 99.864% ( 34, 51 ] 15 0.000% 99.865% ( 51, 76 ] 42 0.001% 99.866% ( 76, 110 ] 1677 0.053% 99.919% ( 110, 170 ] 1901 0.060% 99.979% ( 170, 250 ] 564 0.018% 99.997% ( 250, 380 ] 387 0.012% 100.009% ( 380, 580 ] 338 0.011% 100.020% ( 580, 870 ] 172 0.005% 100.025% ( 870, 1300 ] 75 0.002% 100.027% ( 1300, 1900 ] 29 0.001% 100.028% ( 1900, 2900 ] 21 0.001% 100.029% ( 2900, 4400 ] 20 0.001% 100.030% ( 4400, 6600 ] 1 0.000% 100.030% ( 6600, 9900 ] 2 0.000% 100.030% ** Level 1 read latency histogram (micros): Count: 804887 Average: 4.9130 StdDev: 77.89 Min: 0 Median: 1.7122 Max: 9630 Percentiles: P50: 1.71 P75: 2.43 P99: 8.75 P99.9: 151.93 P99.99: 424.21 ------------------------------------------------------ [ 0, 1 ] 65318 8.115% 8.115% ## ( 1, 2 ] 473346 58.809% 66.924% ############ ( 2, 3 ] 150985 18.759% 85.683% #### ( 3, 4 ] 50535 6.279% 91.961% # ( 4, 6 ] 47319 5.879% 97.840% # ( 6, 10 ] 13564 1.685% 99.525% ( 10, 15 ] 1849 0.230% 99.755% ( 15, 22 ] 276 0.034% 99.789% ( 22, 34 ] 35 0.004% 99.794% ( 34, 51 ] 51 0.006% 99.800% ( 51, 76 ] 51 0.006% 99.806% ( 76, 110 ] 396 0.049% 99.856% ( 110, 170 ] 511 0.063% 99.919% ( 170, 250 ] 275 0.034% 99.953% ( 250, 380 ] 225 0.028% 99.981% ( 380, 580 ] 319 0.040% 100.021% ( 580, 870 ] 255 0.032% 100.053% ( 870, 1300 ] 161 0.020% 100.073% ( 1300, 1900 ] 95 0.012% 100.084% ( 1900, 2900 ] 89 0.011% 100.095% ( 2900, 4400 ] 142 0.018% 100.113% ( 4400, 6600 ] 41 0.005% 100.118% ( 6600, 9900 ] 13 0.002% 100.120% ** Level 2 read latency histogram (micros): Count: 28300 Average: 2.0130 StdDev: 7.10 Min: 1 Median: 1.3743 Max: 900 Percentiles: P50: 1.37 P75: 1.79 P99: 5.37 P99.9: 9.59 P99.99: 183.60 ------------------------------------------------------ [ 0, 1 ] 7788 27.519% 27.519% ###### ( 1, 2 ] 16997 60.060% 87.580% ############ ( 2, 3 ] 1989 7.028% 94.608% # ( 3, 4 ] 865 3.057% 97.664% # ( 4, 6 ] 551 1.947% 99.611% ( 6, 10 ] 91 0.322% 99.933% ( 10, 15 ] 11 0.039% 99.972% ( 15, 22 ] 3 0.011% 99.982% ( 34, 51 ] 1 0.004% 99.986% ( 110, 170 ] 1 0.004% 99.989% ( 170, 250 ] 1 0.004% 99.993% ( 580, 870 ] 1 0.004% 99.996% ( 870, 1300 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 272577 Average: 7.1897 StdDev: 90.64 Min: 0 Median: 1.6749 Max: 5589 Percentiles: P50: 1.67 P75: 2.31 P99: 10.83 P99.9: 1042.65 P99.99: 3596.28 ------------------------------------------------------ [ 0, 1 ] 25842 9.481% 9.481% ## ( 1, 2 ] 163659 60.041% 69.522% ############ ( 2, 3 ] 48521 17.801% 87.323% #### ( 3, 4 ] 15631 5.735% 93.057% # ( 4, 6 ] 11545 4.236% 97.293% # ( 6, 10 ] 4557 1.672% 98.965% ( 10, 15 ] 579 0.212% 99.177% ( 15, 22 ] 136 0.050% 99.227% ( 22, 34 ] 35 0.013% 99.240% ( 34, 51 ] 11 0.004% 99.244% ( 51, 76 ] 6 0.002% 99.246% ( 76, 110 ] 170 0.062% 99.308% ( 110, 170 ] 341 0.125% 99.434% ( 170, 250 ] 316 0.116% 99.549% ( 250, 380 ] 307 0.113% 99.662% ( 380, 580 ] 349 0.128% 99.790% ( 580, 870 ] 240 0.088% 99.878% ( 870, 1300 ] 148 0.054% 99.932% ( 1300, 1900 ] 74 0.027% 99.960% ( 1900, 2900 ] 47 0.017% 99.977% ( 2900, 4400 ] 77 0.028% 100.005% ( 4400, 6600 ] 12 0.004% 100.010% ** Level 4 read latency histogram (micros): Count: 35 Average: 1254.7143 StdDev: 953.45 Min: 56 Median: 1146.4286 Max: 4112 Percentiles: P50: 1146.43 P75: 2056.25 P99: 4112.00 P99.9: 4112.00 P99.99: 4112.00 ------------------------------------------------------ ( 51, 76 ] 1 2.857% 2.857% # ( 76, 110 ] 1 2.857% 5.714% # ( 110, 170 ] 3 8.571% 14.286% ## ( 170, 250 ] 3 8.571% 22.857% ## ( 250, 380 ] 2 5.714% 28.571% # ( 380, 580 ] 2 5.714% 34.286% # ( 580, 870 ] 1 2.857% 37.143% # ( 870, 1300 ] 7 20.000% 57.143% #### ( 1300, 1900 ] 5 14.286% 71.429% ### ( 1900, 2900 ] 8 22.857% 94.286% ##### ( 2900, 4400 ] 2 5.714% 100.000% # ** Level 5 read latency histogram (micros): Count: 160 Average: 1083.5000 StdDev: 1217.50 Min: 77 Median: 498.1818 Max: 6166 Percentiles: P50: 498.18 P75: 1617.65 P99: 5720.00 P99.9: 6166.00 P99.99: 6166.00 ------------------------------------------------------ ( 76, 110 ] 9 5.625% 5.625% # ( 110, 170 ] 23 14.375% 20.000% ### ( 170, 250 ] 13 8.125% 28.125% ## ( 250, 380 ] 22 13.750% 41.875% ### ( 380, 580 ] 22 13.750% 55.625% ### ( 580, 870 ] 10 6.250% 61.875% # ( 870, 1300 ] 12 7.500% 69.375% ## ( 1300, 1900 ] 17 10.625% 80.000% ## ( 1900, 2900 ] 13 8.125% 88.125% ## ( 2900, 4400 ] 15 9.375% 97.500% ## ( 4400, 6600 ] 4 2.500% 100.000% # ** DB Stats ** Uptime(secs): 34.1 total, 3.5 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 230.95 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:12.600 H:M:S, 37.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 282.43 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.831 H:M:S, 23.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/08/11-14:53:13 ... thread 0: (1000000,9000000) ops and (144119.6,220234.2) ops/second in (6.938682,40.865596) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/9 2.89 GB 163.6 8.1 0.0 8.1 16.7 8.5 0.0 1.9 145.1 297.7 57.30 53.14 155 0.370 8442K 12K 0.0 0.0 L1 12/0 1.34 GB 137.3 7.2 5.5 1.7 7.2 5.4 0.1 1.3 225.2 222.6 32.91 31.01 4 8.227 6770K 53K 0.0 0.0 L2 4/2 515.11 MB 2.6 0.1 0.1 0.0 0.1 0.1 4.2 1.8 235.5 235.5 0.47 0.45 1 0.473 113K 22 0.0 0.0 L3 22/6 1.61 GB 1.0 3.3 2.2 1.0 3.3 2.2 1.5 1.5 166.1 165.7 20.27 14.68 25 0.811 2362K 1832 0.0 0.0 L4 170/14 9.47 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.8 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1072/0 88.77 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1296/31 104.59 GB 0.0 18.8 7.8 10.9 27.2 16.3 11.7 3.2 173.1 251.1 110.95 99.28 185 0.600 17M 67K 0.0 0.0 Int 0/0 0.00 KB 0.0 18.8 7.8 10.9 27.2 16.3 11.7 3.2 173.1 251.1 110.95 99.28 185 0.600 17M 67K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 18.8 7.8 10.9 18.6 7.7 0.0 0.0 224.6 223.4 85.50 77.16 57 1.500 17M 67K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.6 8.6 0.0 0.0 0.0 344.2 25.44 22.12 128 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 41.0 total, 40.9 interval Flush(GB): cumulative 8.552, interval 8.552 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 27.20 GB write, 678.88 MB/s write, 18.75 GB read, 468.02 MB/s read, 110.9 seconds Interval compaction: 27.20 GB write, 681.59 MB/s write, 18.75 GB read, 469.89 MB/s read, 110.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 28, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 28, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 43, pending-compaction-bytes-delays: 6, pending-compaction-bytes-stops: 0, total-delays: 34, total-stops: 43, interval: 77 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3779610 Average: 2.4668 StdDev: 16.06 Min: 0 Median: 1.5012 Max: 8022 Percentiles: P50: 1.50 P75: 1.88 P99: 5.91 P99.9: 91.55 P99.99: 214.65 ------------------------------------------------------ [ 0, 1 ] 641074 16.961% 16.961% ### ( 1, 2 ] 2491493 65.919% 82.881% ############# ( 2, 3 ] 394144 10.428% 93.309% ## ( 3, 4 ] 135990 3.598% 96.907% # ( 4, 6 ] 82919 2.194% 99.101% ( 6, 10 ] 25312 0.670% 99.770% ( 10, 15 ] 3272 0.087% 99.857% ( 15, 22 ] 666 0.018% 99.875% ( 22, 34 ] 50 0.001% 99.876% ( 34, 51 ] 19 0.001% 99.876% ( 51, 76 ] 55 0.001% 99.878% ( 76, 110 ] 1831 0.048% 99.926% ( 110, 170 ] 2062 0.055% 99.981% ( 170, 250 ] 620 0.016% 99.997% ( 250, 380 ] 427 0.011% 100.009% ( 380, 580 ] 374 0.010% 100.018% ( 580, 870 ] 193 0.005% 100.024% ( 870, 1300 ] 80 0.002% 100.026% ( 1300, 1900 ] 32 0.001% 100.027% ( 1900, 2900 ] 22 0.001% 100.027% ( 2900, 4400 ] 20 0.001% 100.028% ( 4400, 6600 ] 1 0.000% 100.028% ( 6600, 9900 ] 2 0.000% 100.028% ** Level 1 read latency histogram (micros): Count: 1034232 Average: 4.5844 StdDev: 69.89 Min: 0 Median: 1.6687 Max: 9630 Percentiles: P50: 1.67 P75: 2.29 P99: 8.73 P99.9: 183.35 P99.99: 487.37 ------------------------------------------------------ [ 0, 1 ] 97452 9.423% 9.423% ## ( 1, 2 ] 627577 60.680% 70.103% ############ ( 2, 3 ] 176818 17.097% 87.200% ### ( 3, 4 ] 58224 5.630% 92.829% # ( 4, 6 ] 52474 5.074% 97.903% # ( 6, 10 ] 16617 1.607% 99.510% ( 10, 15 ] 1997 0.193% 99.703% ( 15, 22 ] 314 0.030% 99.733% ( 22, 34 ] 38 0.004% 99.737% ( 34, 51 ] 52 0.005% 99.742% ( 51, 76 ] 73 0.007% 99.749% ( 76, 110 ] 696 0.067% 99.816% ( 110, 170 ] 798 0.077% 99.893% ( 170, 250 ] 412 0.040% 99.933% ( 250, 380 ] 368 0.036% 99.969% ( 380, 580 ] 409 0.040% 100.008% ( 580, 870 ] 310 0.030% 100.038% ( 870, 1300 ] 199 0.019% 100.058% ( 1300, 1900 ] 105 0.010% 100.068% ( 1900, 2900 ] 92 0.009% 100.077% ( 2900, 4400 ] 144 0.014% 100.091% ( 4400, 6600 ] 41 0.004% 100.095% ( 6600, 9900 ] 13 0.001% 100.096% ** Level 2 read latency histogram (micros): Count: 28300 Average: 2.0130 StdDev: 7.10 Min: 1 Median: 1.3743 Max: 900 Percentiles: P50: 1.37 P75: 1.79 P99: 5.37 P99.9: 9.59 P99.99: 183.60 ------------------------------------------------------ [ 0, 1 ] 7788 27.519% 27.519% ###### ( 1, 2 ] 16997 60.060% 87.580% ############ ( 2, 3 ] 1989 7.028% 94.608% # ( 3, 4 ] 865 3.057% 97.664% # ( 4, 6 ] 551 1.947% 99.611% ( 6, 10 ] 91 0.322% 99.933% ( 10, 15 ] 11 0.039% 99.972% ( 15, 22 ] 3 0.011% 99.982% ( 34, 51 ] 1 0.004% 99.986% ( 110, 170 ] 1 0.004% 99.989% ( 170, 250 ] 1 0.004% 99.993% ( 580, 870 ] 1 0.004% 99.996% ( 870, 1300 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 272865 Average: 7.4618 StdDev: 91.47 Min: 0 Median: 1.6756 Max: 5589 Percentiles: P50: 1.68 P75: 2.31 P99: 12.46 P99.9: 1066.32 P99.99: 3595.72 ------------------------------------------------------ [ 0, 1 ] 25842 9.471% 9.471% ## ( 1, 2 ] 163686 59.988% 69.459% ############ ( 2, 3 ] 48554 17.794% 87.253% #### ( 3, 4 ] 15648 5.735% 92.987% # ( 4, 6 ] 11558 4.236% 97.223% # ( 6, 10 ] 4563 1.672% 98.895% ( 10, 15 ] 579 0.212% 99.108% ( 15, 22 ] 136 0.050% 99.157% ( 22, 34 ] 35 0.013% 99.170% ( 34, 51 ] 11 0.004% 99.174% ( 51, 76 ] 6 0.002% 99.177% ( 76, 110 ] 180 0.066% 99.242% ( 110, 170 ] 363 0.133% 99.376% ( 170, 250 ] 360 0.132% 99.507% ( 250, 380 ] 348 0.128% 99.635% ( 380, 580 ] 386 0.141% 99.776% ( 580, 870 ] 265 0.097% 99.874% ( 870, 1300 ] 158 0.058% 99.931% ( 1300, 1900 ] 77 0.028% 99.960% ( 1900, 2900 ] 47 0.017% 99.977% ( 2900, 4400 ] 77 0.028% 100.005% ( 4400, 6600 ] 12 0.004% 100.010% ** Level 4 read latency histogram (micros): Count: 35 Average: 1254.7143 StdDev: 953.45 Min: 56 Median: 1146.4286 Max: 4112 Percentiles: P50: 1146.43 P75: 2056.25 P99: 4112.00 P99.9: 4112.00 P99.99: 4112.00 ------------------------------------------------------ ( 51, 76 ] 1 2.857% 2.857% # ( 76, 110 ] 1 2.857% 5.714% # ( 110, 170 ] 3 8.571% 14.286% ## ( 170, 250 ] 3 8.571% 22.857% ## ( 250, 380 ] 2 5.714% 28.571% # ( 380, 580 ] 2 5.714% 34.286% # ( 580, 870 ] 1 2.857% 37.143% # ( 870, 1300 ] 7 20.000% 57.143% #### ( 1300, 1900 ] 5 14.286% 71.429% ### ( 1900, 2900 ] 8 22.857% 94.286% ##### ( 2900, 4400 ] 2 5.714% 100.000% # ** Level 5 read latency histogram (micros): Count: 180 Average: 1066.0778 StdDev: 1190.27 Min: 75 Median: 501.7391 Max: 6166 Percentiles: P50: 501.74 P75: 1633.33 P99: 5610.00 P99.9: 6166.00 P99.99: 6166.00 ------------------------------------------------------ ( 51, 76 ] 1 0.556% 0.556% ( 76, 110 ] 12 6.667% 7.222% # ( 110, 170 ] 24 13.333% 20.556% ### ( 170, 250 ] 15 8.333% 28.889% ## ( 250, 380 ] 24 13.333% 42.222% ### ( 380, 580 ] 23 12.778% 55.000% ### ( 580, 870 ] 13 7.222% 62.222% # ( 870, 1300 ] 13 7.222% 69.444% # ( 1300, 1900 ] 18 10.000% 79.444% ## ( 1900, 2900 ] 17 9.444% 88.889% ## ( 2900, 4400 ] 16 8.889% 97.778% ## ( 4400, 6600 ] 4 2.222% 100.000% ** DB Stats ** Uptime(secs): 41.0 total, 6.9 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 215.88 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:16.905 H:M:S, 41.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 141.84 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.305 H:M:S, 62.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 7 num-running-flushes: 1 2024/08/11-14:53:18 ... thread 0: (1000000,10000000) ops and (208865.2,219041.9) ops/second in (4.787776,45.653372) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/16 3.81 GB 37.8 9.7 0.0 9.7 19.1 9.5 0.0 2.0 148.1 293.2 66.75 61.54 171 0.390 10M 17K 0.0 0.0 L1 12/12 1.34 GB 0.0 7.2 5.5 1.7 7.2 5.4 0.1 1.3 225.2 222.6 32.91 31.01 4 8.227 6770K 53K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 4.2 1.8 235.5 235.5 0.47 0.45 1 0.473 113K 22 0.0 0.0 L3 17/0 981.94 MB 1.0 4.5 2.7 1.7 4.4 2.7 1.5 1.6 172.1 171.5 26.53 20.03 29 0.915 3581K 11K 0.0 0.0 L4 167/0 9.62 GB 1.0 1.7 1.1 0.6 1.7 1.1 2.8 1.5 178.6 177.2 9.86 8.06 14 0.704 1788K 14K 0.0 0.0 L5 1084/0 89.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1299/28 105.48 GB 0.0 23.2 9.5 13.7 32.5 18.8 12.7 3.4 173.9 244.0 136.52 121.08 219 0.623 22M 95K 0.0 0.0 Int 0/0 0.00 KB 0.0 23.2 9.5 13.7 32.5 18.8 12.6 3.4 173.9 244.0 136.52 121.08 219 0.623 22M 95K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 23.2 9.5 13.7 23.1 9.4 0.0 0.0 220.1 218.8 107.86 96.55 78 1.383 22M 95K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 9.5 9.5 0.0 0.0 0.0 338.5 28.66 24.53 141 0.203 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 45.8 total, 45.7 interval Flush(GB): cumulative 9.474, interval 9.474 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 32.53 GB write, 726.92 MB/s write, 23.18 GB read, 518.10 MB/s read, 136.5 seconds Interval compaction: 32.53 GB write, 729.52 MB/s write, 23.18 GB read, 519.94 MB/s read, 136.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 28, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 28, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 46, pending-compaction-bytes-delays: 18, pending-compaction-bytes-stops: 0, total-delays: 46, total-stops: 46, interval: 92 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 4203692 Average: 2.5437 StdDev: 16.63 Min: 0 Median: 1.5272 Max: 8022 Percentiles: P50: 1.53 P75: 1.91 P99: 6.00 P99.9: 92.00 P99.99: 226.66 ------------------------------------------------------ [ 0, 1 ] 660716 15.718% 15.718% ### ( 1, 2 ] 2733546 65.027% 80.745% ############# ( 2, 3 ] 481718 11.459% 92.204% ## ( 3, 4 ] 169541 4.033% 96.237% # ( 4, 6 ] 116233 2.765% 99.002% # ( 6, 10 ] 31915 0.759% 99.762% ( 10, 15 ] 4047 0.096% 99.858% ( 15, 22 ] 733 0.017% 99.875% ( 22, 34 ] 56 0.001% 99.877% ( 34, 51 ] 21 0.000% 99.877% ( 51, 76 ] 59 0.001% 99.879% ( 76, 110 ] 1924 0.046% 99.924% ( 110, 170 ] 2232 0.053% 99.977% ( 170, 250 ] 752 0.018% 99.995% ( 250, 380 ] 520 0.012% 100.008% ( 380, 580 ] 473 0.011% 100.019% ( 580, 870 ] 245 0.006% 100.025% ( 870, 1300 ] 102 0.002% 100.027% ( 1300, 1900 ] 41 0.001% 100.028% ( 1900, 2900 ] 25 0.001% 100.029% ( 2900, 4400 ] 25 0.001% 100.029% ( 4400, 6600 ] 1 0.000% 100.029% ( 6600, 9900 ] 2 0.000% 100.029% ** Level 1 read latency histogram (micros): Count: 1327289 Average: 4.5898 StdDev: 67.96 Min: 0 Median: 1.7183 Max: 9630 Percentiles: P50: 1.72 P75: 2.46 P99: 8.87 P99.9: 161.45 P99.99: 453.13 ------------------------------------------------------ [ 0, 1 ] 104719 7.890% 7.890% ## ( 1, 2 ] 778075 58.621% 66.511% ############ ( 2, 3 ] 246194 18.549% 85.060% #### ( 3, 4 ] 86023 6.481% 91.541% # ( 4, 6 ] 81461 6.137% 97.678% # ( 6, 10 ] 24445 1.842% 99.520% ( 10, 15 ] 3016 0.227% 99.747% ( 15, 22 ] 418 0.031% 99.779% ( 22, 34 ] 41 0.003% 99.782% ( 34, 51 ] 57 0.004% 99.786% ( 51, 76 ] 86 0.006% 99.793% ( 76, 110 ] 715 0.054% 99.846% ( 110, 170 ] 830 0.063% 99.909% ( 170, 250 ] 458 0.035% 99.943% ( 250, 380 ] 438 0.033% 99.976% ( 380, 580 ] 493 0.037% 100.014% ( 580, 870 ] 406 0.031% 100.044% ( 870, 1300 ] 261 0.020% 100.064% ( 1300, 1900 ] 130 0.010% 100.074% ( 1900, 2900 ] 126 0.009% 100.083% ( 2900, 4400 ] 177 0.013% 100.096% ( 4400, 6600 ] 50 0.004% 100.100% ( 6600, 9900 ] 13 0.001% 100.101% ** Level 2 read latency histogram (micros): Count: 28300 Average: 2.0130 StdDev: 7.10 Min: 1 Median: 1.3743 Max: 900 Percentiles: P50: 1.37 P75: 1.79 P99: 5.37 P99.9: 9.59 P99.99: 183.60 ------------------------------------------------------ [ 0, 1 ] 7788 27.519% 27.519% ###### ( 1, 2 ] 16997 60.060% 87.580% ############ ( 2, 3 ] 1989 7.028% 94.608% # ( 3, 4 ] 865 3.057% 97.664% # ( 4, 6 ] 551 1.947% 99.611% ( 6, 10 ] 91 0.322% 99.933% ( 10, 15 ] 11 0.039% 99.972% ( 15, 22 ] 3 0.011% 99.982% ( 34, 51 ] 1 0.004% 99.986% ( 110, 170 ] 1 0.004% 99.989% ( 170, 250 ] 1 0.004% 99.993% ( 580, 870 ] 1 0.004% 99.996% ( 870, 1300 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 665949 Average: 6.0430 StdDev: 73.14 Min: 0 Median: 1.8091 Max: 7916 Percentiles: P50: 1.81 P75: 2.69 P99: 9.38 P99.9: 509.70 P99.99: 797.15 ------------------------------------------------------ [ 0, 1 ] 35011 5.257% 5.257% # ( 1, 2 ] 368260 55.299% 60.556% ########### ( 2, 3 ] 139491 20.946% 81.502% #### ( 3, 4 ] 56508 8.485% 89.987% ## ( 4, 6 ] 50222 7.541% 97.529% ## ( 6, 10 ] 11596 1.741% 99.270% ( 10, 15 ] 1431 0.215% 99.485% ( 15, 22 ] 246 0.037% 99.522% ( 22, 34 ] 46 0.007% 99.529% ( 34, 51 ] 12 0.002% 99.531% ( 51, 76 ] 22 0.003% 99.534% ( 76, 110 ] 274 0.041% 99.575% ( 110, 170 ] 595 0.089% 99.664% ( 170, 250 ] 589 0.088% 99.753% ( 250, 380 ] 554 0.083% 99.836% ( 380, 580 ] 657 0.099% 99.935% ( 580, 870 ] 492 0.074% 100.009% ( 870, 1300 ] 268 0.040% 100.049% ( 1300, 1900 ] 114 0.017% 100.066% ( 1900, 2900 ] 91 0.014% 100.080% ( 2900, 4400 ] 108 0.016% 100.096% ( 4400, 6600 ] 18 0.003% 100.099% ( 6600, 9900 ] 1 0.000% 100.099% ** Level 4 read latency histogram (micros): Count: 23461 Average: 5.6435 StdDev: 67.97 Min: 1 Median: 1.8070 Max: 4112 Percentiles: P50: 1.81 P75: 2.62 P99: 7.85 P99.9: 1086.52 P99.99: 2861.54 ------------------------------------------------------ [ 0, 1 ] 630 2.685% 2.685% # ( 1, 2 ] 13756 58.633% 61.319% ############ ( 2, 3 ] 5149 21.947% 83.266% #### ( 3, 4 ] 1832 7.809% 91.075% ## ( 4, 6 ] 1772 7.553% 98.628% ## ( 6, 10 ] 189 0.806% 99.433% ( 10, 15 ] 29 0.124% 99.557% ( 15, 22 ] 2 0.009% 99.565% ( 22, 34 ] 3 0.013% 99.578% ( 34, 51 ] 1 0.004% 99.582% ( 51, 76 ] 6 0.026% 99.608% ( 76, 110 ] 3 0.013% 99.621% ( 110, 170 ] 9 0.038% 99.659% ( 170, 250 ] 27 0.115% 99.774% ( 250, 380 ] 6 0.026% 99.800% ( 380, 580 ] 9 0.038% 99.838% ( 580, 870 ] 9 0.038% 99.876% ( 870, 1300 ] 11 0.047% 99.923% ( 1300, 1900 ] 7 0.030% 99.953% ( 1900, 2900 ] 9 0.038% 99.991% ( 2900, 4400 ] 2 0.009% 100.000% ** Level 5 read latency histogram (micros): Count: 240 Average: 1006.2083 StdDev: 1153.57 Min: 75 Median: 461.4815 Max: 6166 Percentiles: P50: 461.48 P75: 1510.00 P99: 5544.00 P99.9: 6166.00 P99.99: 6166.00 ------------------------------------------------------ ( 51, 76 ] 1 0.417% 0.417% ( 76, 110 ] 18 7.500% 7.917% ## ( 110, 170 ] 37 15.417% 23.333% ### ( 170, 250 ] 23 9.583% 32.917% ## ( 250, 380 ] 30 12.500% 45.417% ### ( 380, 580 ] 27 11.250% 56.667% ## ( 580, 870 ] 19 7.917% 64.583% ## ( 870, 1300 ] 18 7.500% 72.083% ## ( 1300, 1900 ] 20 8.333% 80.417% ## ( 1900, 2900 ] 23 9.583% 90.000% ## ( 2900, 4400 ] 19 7.917% 97.917% ## ( 4400, 6600 ] 5 2.083% 100.000% ** DB Stats ** Uptime(secs): 45.8 total, 4.8 interval Cumulative writes: 0 writes, 10M keys, 0 commit groups, 0.0 writes per commit group, ingest: 9.61 GB, 214.81 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:18.819 H:M:S, 41.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 205.57 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.914 H:M:S, 40.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 5.637 micros/op 177393 ops/sec 60.070 seconds 10655999 operations; 171.9 MB/s Microseconds per write: Count: 10655999 Average: 5.6372 StdDev: 294.78 Min: 0 Median: 1.7453 Max: 578527 Percentiles: P50: 1.75 P75: 2.76 P99: 8.74 P99.9: 1148.38 P99.99: 2683.17 ------------------------------------------------------ [ 0, 1 ] 1354005 12.707% 12.707% ### ( 1, 2 ] 5332363 50.041% 62.747% ########## ( 2, 3 ] 1727269 16.209% 78.957% ### ( 3, 4 ] 1487413 13.958% 92.915% ### ( 4, 6 ] 576331 5.409% 98.324% # ( 6, 10 ] 105078 0.986% 99.310% ( 10, 15 ] 31595 0.296% 99.606% ( 15, 22 ] 7893 0.074% 99.680% ( 22, 34 ] 1533 0.014% 99.695% ( 34, 51 ] 174 0.002% 99.696% ( 51, 76 ] 18 0.000% 99.697% ( 76, 110 ] 7620 0.072% 99.768% ( 110, 170 ] 2843 0.027% 99.795% ( 170, 250 ] 254 0.002% 99.797% ( 250, 380 ] 133 0.001% 99.798% ( 380, 580 ] 16 0.000% 99.799% ( 580, 870 ] 13 0.000% 99.799% ( 870, 1300 ] 16670 0.156% 99.955% ( 1300, 1900 ] 37 0.000% 99.956% ( 1900, 2900 ] 4693 0.044% 100.000% ( 2900, 4400 ] 23 0.000% 100.000% ( 4400, 6600 ] 5 0.000% 100.000% ( 6600, 9900 ] 2 0.000% 100.000% ( 9900, 14000 ] 1 0.000% 100.000% ( 22000, 33000 ] 1 0.000% 100.000% ( 33000, 50000 ] 1 0.000% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% ( 110000, 170000 ] 3 0.000% 100.000% ( 170000, 250000 ] 9 0.000% 100.000% ( 250000, 380000 ] 1 0.000% 100.000% ( 570000, 860000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 209610 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 10655999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 10996990968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 540 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 30930483 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 32245658801 rocksdb.compact.write.bytes COUNT : 32045965492 rocksdb.flush.write.bytes COUNT : 10965699786 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 59 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 279 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 10877142016 rocksdb.memtable.garbage.bytes.at.flush COUNT : 4116480 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 32344677829 rocksdb.non.last.level.read.count COUNT : 7815265 rocksdb.block.checksum.compute.count COUNT : 7814244 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.407123 P95 : 3.890894 P99 : 7.051944 P100 : 578525.000000 COUNT : 10655999 SUM : 55185020 rocksdb.compaction.times.micros P50 : 952727.272727 P95 : 4253333.333333 P99 : 15137897.000000 P100 : 15137897.000000 COUNT : 84 SUM : 120608021 rocksdb.compaction.times.cpu_micros P50 : 932121.212121 P95 : 4244000.000000 P99 : 14048679.000000 P100 : 14048679.000000 COUNT : 84 SUM : 108761557 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 716.030928 P95 : 1209.113636 P99 : 8151.000000 P100 : 42277.000000 COUNT : 153 SUM : 153897 rocksdb.compaction.outfile.sync.micros P50 : 861.594203 P95 : 15066.666667 P99 : 47416.000000 P100 : 71405.000000 COUNT : 352 SUM : 1121459 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 98.972973 P95 : 426.666667 P99 : 1556.000000 P100 : 3530.000000 COUNT : 272 SUM : 47777 rocksdb.table.open.io.micros P50 : 1092.771084 P95 : 6477.777778 P99 : 9540.000000 P100 : 12401.000000 COUNT : 540 SUM : 985722 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.610646 P95 : 7.105870 P99 : 9.963340 P100 : 15361.000000 COUNT : 7812628 SUM : 42317579 rocksdb.read.block.get.micros P50 : 350.728477 P95 : 3250.000000 P99 : 4330.000000 P100 : 6452.000000 COUNT : 1080 SUM : 815269 rocksdb.write.raw.block.micros P50 : 0.512491 P95 : 0.973734 P99 : 2.857585 P100 : 12780.000000 COUNT : 10416870 SUM : 19508806 rocksdb.numfiles.in.singlecompaction P50 : 3.250000 P95 : 6.750000 P99 : 25.300000 P100 : 34.000000 COUNT : 145 SUM : 576 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.561492 P95 : 1169.244977 P99 : 2487.413830 P100 : 578358.000000 COUNT : 196019 SUM : 30946370 rocksdb.sst.read.micros P50 : 1.573935 P95 : 4.109700 P99 : 8.055339 P100 : 9630.000000 COUNT : 7815371 SUM : 26427409 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 10655999 SUM : 10996990968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 210000.000000 P95 : 346107.142857 P99 : 536433.333333 P100 : 687306.000000 COUNT : 153 SUM : 30800139 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 351 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0