[2024-11-21 05:59:00.923881] Starting SPDK v25.01-pre git sha1 557f022f6 / DPDK 23.11.0 initialization... [2024-11-21 05:59:00.924148] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2695459 ] Set seed to 1732165140917208 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-21 05:59:01.042552] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-21 05:59:01.066184] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-21 05:59:01.066226] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2695459' to capture a snapshot of events at runtime. [2024-11-21 05:59:01.066236] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-21 05:59:01.066247] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-21 05:59:01.066256] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2695459 for offline analysis/debug. [2024-11-21 05:59:01.066814] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Thu Nov 21 05:59:05 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/21-05:59:19 ... thread 0: (1000000,1000000) ops and (71932.5,71932.5) ops/second in (13.901917,13.901917) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 5/3 566.72 MB 18.9 0.2 0.0 0.2 1.1 0.9 0.0 1.3 38.8 174.6 6.49 3.47 13 0.499 255K 235 0.0 0.0 L1 4/3 199.90 MB 0.0 0.6 0.4 0.2 0.6 0.4 0.0 1.5 130.1 129.9 4.55 2.74 2 2.276 601K 907 0.0 0.0 L2 2/1 145.78 MB 0.7 5.4 1.1 4.3 5.3 1.0 0.2 4.9 109.1 107.9 50.27 25.84 17 2.957 1121K 0 0.0 0.0 L3 17/2 1.05 GB 1.0 7.5 5.3 2.2 7.5 5.3 0.5 1.4 117.5 117.4 65.24 36.56 45 1.450 6263K 2303 0.0 0.0 L4 159/0 9.49 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1120/0 92.03 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1307/9 103.47 GB 0.0 13.7 6.7 6.9 14.5 7.5 12.4 16.8 110.6 117.0 126.56 68.61 77 1.644 8242K 3445 0.0 0.0 Int 0/0 0.00 KB 0.0 13.7 6.7 6.9 14.5 7.5 12.4 16.8 110.6 117.0 126.56 68.61 77 1.644 8242K 3445 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.7 6.7 6.9 13.6 6.7 0.0 0.0 115.4 114.8 121.25 66.15 65 1.865 8242K 3445 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 166.2 5.31 2.46 12 0.442 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 14.1 total, 13.9 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.46 GB write, 1051.95 MB/s write, 13.67 GB read, 994.24 MB/s read, 126.6 seconds Interval compaction: 14.46 GB write, 1064.97 MB/s write, 13.67 GB read, 1006.55 MB/s read, 126.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 16, pending-compaction-bytes-delays: 32, pending-compaction-bytes-stops: 0, total-delays: 32, total-stops: 16, interval: 47 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 203994 Average: 11.5458 StdDev: 146.29 Min: 1 Median: 1.7932 Max: 25357 Percentiles: P50: 1.79 P75: 2.66 P99: 11.82 P99.9: 2209.89 P99.99: 3970.01 ------------------------------------------------------ [ 0, 1 ] 10844 5.316% 5.316% # ( 1, 2 ] 114916 56.333% 61.649% ########### ( 2, 3 ] 41139 20.167% 81.816% #### ( 3, 4 ] 16676 8.175% 89.990% ## ( 4, 6 ] 14769 7.240% 97.230% # ( 6, 10 ] 3403 1.668% 98.898% ( 10, 15 ] 571 0.280% 99.178% ( 15, 22 ] 83 0.041% 99.219% ( 22, 34 ] 4 0.002% 99.221% ( 34, 51 ] 5 0.002% 99.224% ( 51, 76 ] 3 0.001% 99.225% ( 76, 110 ] 52 0.025% 99.250% ( 110, 170 ] 139 0.068% 99.319% ( 170, 250 ] 119 0.058% 99.377% ( 250, 380 ] 119 0.058% 99.435% ( 380, 580 ] 185 0.091% 99.526% ( 580, 870 ] 260 0.127% 99.653% ( 870, 1300 ] 288 0.141% 99.795% ( 1300, 1900 ] 172 0.084% 99.879% ( 1900, 2900 ] 142 0.070% 99.949% ( 2900, 4400 ] 120 0.059% 100.007% ( 4400, 6600 ] 18 0.009% 100.016% ( 6600, 9900 ] 1 0.000% 100.017% ( 22000, 33000 ] 1 0.000% 100.017% ** Level 1 read latency histogram (micros): Count: 308117 Average: 9.5141 StdDev: 124.05 Min: 1 Median: 2.0492 Max: 30845 Percentiles: P50: 2.05 P75: 3.25 P99: 12.47 P99.9: 1138.91 P99.99: 1681.84 ------------------------------------------------------ [ 0, 1 ] 7954 2.581% 2.581% # ( 1, 2 ] 142560 46.268% 48.850% ######### ( 2, 3 ] 72024 23.376% 72.225% ##### ( 3, 4 ] 34798 11.294% 83.519% ## ( 4, 6 ] 37038 12.021% 95.540% ## ( 6, 10 ] 9814 3.185% 98.725% # ( 10, 15 ] 1716 0.557% 99.282% ( 15, 22 ] 236 0.077% 99.358% ( 22, 34 ] 36 0.012% 99.370% ( 34, 51 ] 22 0.007% 99.377% ( 51, 76 ] 24 0.008% 99.385% ( 76, 110 ] 39 0.013% 99.398% ( 110, 170 ] 110 0.036% 99.433% ( 170, 250 ] 156 0.051% 99.484% ( 250, 380 ] 298 0.097% 99.581% ( 380, 580 ] 360 0.117% 99.698% ( 580, 870 ] 400 0.130% 99.827% ( 870, 1300 ] 358 0.116% 99.944% ( 1300, 1900 ] 225 0.073% 100.017% ( 1900, 2900 ] 125 0.041% 100.057% ( 2900, 4400 ] 70 0.023% 100.080% ( 4400, 6600 ] 6 0.002% 100.082% ( 6600, 9900 ] 3 0.001% 100.083% ( 14000, 22000 ] 2 0.001% 100.083% ( 22000, 33000 ] 1 0.000% 100.084% ** Level 2 read latency histogram (micros): Count: 2164525 Average: 12.3702 StdDev: 163.83 Min: 0 Median: 2.0055 Max: 24055 Percentiles: P50: 2.01 P75: 3.11 P99: 9.89 P99.9: 666.00 P99.99: 958.01 ------------------------------------------------------ [ 0, 1 ] 35065 1.620% 1.620% ( 1, 2 ] 1044380 48.250% 49.870% ########## ( 2, 3 ] 516476 23.861% 73.731% ##### ( 3, 4 ] 251190 11.605% 85.336% ## ( 4, 6 ] 238011 10.996% 96.332% ## ( 6, 10 ] 59365 2.743% 99.074% # ( 10, 15 ] 11760 0.543% 99.618% ( 15, 22 ] 1577 0.073% 99.690% ( 22, 34 ] 171 0.008% 99.698% ( 34, 51 ] 221 0.010% 99.709% ( 51, 76 ] 156 0.007% 99.716% ( 76, 110 ] 140 0.006% 99.722% ( 110, 170 ] 332 0.015% 99.738% ( 170, 250 ] 516 0.024% 99.761% ( 250, 380 ] 821 0.038% 99.799% ( 380, 580 ] 1550 0.072% 99.871% ( 580, 870 ] 2126 0.098% 99.969% ( 870, 1300 ] 2211 0.102% 100.071% ( 1300, 1900 ] 1623 0.075% 100.146% ( 1900, 2900 ] 1715 0.079% 100.225% ( 2900, 4400 ] 1653 0.076% 100.302% ( 4400, 6600 ] 408 0.019% 100.321% ( 6600, 9900 ] 65 0.003% 100.324% ( 9900, 14000 ] 13 0.001% 100.324% ( 14000, 22000 ] 14 0.001% 100.325% ( 22000, 33000 ] 1 0.000% 100.325% ** Level 3 read latency histogram (micros): Count: 577310 Average: 17.3936 StdDev: 190.04 Min: 1 Median: 1.9385 Max: 27920 Percentiles: P50: 1.94 P75: 2.94 P99: 93.45 P99.9: 2566.26 P99.99: 3571.34 ------------------------------------------------------ [ 0, 1 ] 13377 2.317% 2.317% ( 1, 2 ] 293312 50.807% 53.124% ########## ( 2, 3 ] 134731 23.338% 76.462% ##### ( 3, 4 ] 61194 10.600% 87.061% ## ( 4, 6 ] 52235 9.048% 96.109% ## ( 6, 10 ] 13614 2.358% 98.468% ( 10, 15 ] 2520 0.437% 98.904% ( 15, 22 ] 351 0.061% 98.965% ( 22, 34 ] 39 0.007% 98.972% ( 34, 51 ] 76 0.013% 98.985% ( 51, 76 ] 53 0.009% 98.994% ( 76, 110 ] 68 0.012% 99.006% ( 110, 170 ] 216 0.037% 99.043% ( 170, 250 ] 320 0.055% 99.099% ( 250, 380 ] 428 0.074% 99.173% ( 380, 580 ] 710 0.123% 99.296% ( 580, 870 ] 1116 0.193% 99.489% ( 870, 1300 ] 1095 0.190% 99.679% ( 1300, 1900 ] 764 0.132% 99.811% ( 1900, 2900 ] 771 0.134% 99.945% ( 2900, 4400 ] 586 0.102% 100.046% ( 4400, 6600 ] 109 0.019% 100.065% ( 6600, 9900 ] 14 0.002% 100.067% ( 9900, 14000 ] 4 0.001% 100.068% ( 14000, 22000 ] 2 0.000% 100.068% ( 22000, 33000 ] 1 0.000% 100.069% ** Level 4 read latency histogram (micros): Count: 4128 Average: 1117.1611 StdDev: 1303.03 Min: 81 Median: 686.9599 Max: 23325 Percentiles: P50: 686.96 P75: 1404.28 P99: 6153.16 P99.9: 9565.60 P99.99: 23325.00 ------------------------------------------------------ ( 76, 110 ] 110 2.665% 2.665% # ( 110, 170 ] 417 10.102% 12.766% ## ( 170, 250 ] 388 9.399% 22.166% ## ( 250, 380 ] 376 9.109% 31.274% ## ( 380, 580 ] 534 12.936% 44.210% ### ( 580, 870 ] 648 15.698% 59.908% ### ( 870, 1300 ] 554 13.421% 73.328% ### ( 1300, 1900 ] 397 9.617% 82.946% ## ( 1900, 2900 ] 280 6.783% 89.729% # ( 2900, 4400 ] 311 7.534% 97.263% ## ( 4400, 6600 ] 90 2.180% 99.443% ( 6600, 9900 ] 21 0.509% 99.952% ( 22000, 33000 ] 2 0.048% 100.000% ** Level 5 read latency histogram (micros): Count: 37808 Average: 1117.4175 StdDev: 1286.06 Min: 73 Median: 684.0299 Max: 25027 Percentiles: P50: 684.03 P75: 1390.57 P99: 6037.18 P99.9: 9756.97 P99.99: 24235.20 ------------------------------------------------------ ( 51, 76 ] 1 0.003% 0.003% ( 76, 110 ] 858 2.269% 2.272% ( 110, 170 ] 3801 10.053% 12.325% ## ( 170, 250 ] 3418 9.040% 21.366% ## ( 250, 380 ] 3698 9.781% 31.147% ## ( 380, 580 ] 5037 13.323% 44.469% ### ( 580, 870 ] 5829 15.417% 59.887% ### ( 870, 1300 ] 5197 13.746% 73.633% ### ( 1300, 1900 ] 3425 9.059% 82.691% ## ( 1900, 2900 ] 2697 7.133% 89.825% # ( 2900, 4400 ] 2852 7.543% 97.368% ## ( 4400, 6600 ] 829 2.193% 99.561% ( 6600, 9900 ] 134 0.354% 99.915% ( 9900, 14000 ] 16 0.042% 99.958% ( 14000, 22000 ] 11 0.029% 99.987% ( 22000, 33000 ] 6 0.016% 100.003% ** DB Stats ** Uptime(secs): 14.1 total, 13.9 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 69.92 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.875 H:M:S, 70.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 70.78 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:9.875 H:M:S, 71.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/21-05:59:22 ... thread 0: (1000000,2000000) ops and (281128.9,114554.1) ops/second in (3.557087,17.459004) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 8/6 1.11 GB 12.6 1.0 0.0 1.0 2.8 1.8 0.0 1.6 81.7 221.1 13.10 9.24 31 0.423 1087K 1074 0.0 0.0 L1 3/3 63.95 MB 0.0 1.1 0.7 0.4 1.1 0.7 0.0 1.5 162.1 161.8 7.22 5.29 3 2.406 1187K 2010 0.0 0.0 L2 2/1 175.48 MB 0.5 5.4 1.1 4.3 5.3 1.0 0.7 4.9 109.1 107.9 50.27 25.84 17 2.957 1121K 0 0.0 0.0 L3 23/6 1.35 GB 1.0 8.4 5.7 2.7 8.4 5.7 0.6 1.5 122.4 122.2 70.40 40.85 50 1.408 7224K 5147 0.0 0.0 L4 162/1 9.67 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1120/0 92.03 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1318/17 104.39 GB 0.0 16.0 7.5 8.5 17.7 9.2 13.2 9.9 115.9 128.3 140.99 81.22 101 1.396 10M 8231 0.0 0.0 Int 0/0 0.00 KB 0.0 16.0 7.5 8.5 17.7 9.2 13.2 9.9 115.9 128.3 140.99 81.22 101 1.396 10M 8231 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 16.0 7.5 8.5 15.9 7.4 0.0 0.0 123.4 122.9 132.36 76.14 74 1.789 10M 8231 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 211.8 8.63 5.08 27 0.319 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 17.6 total, 17.5 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 17.67 GB write, 1025.91 MB/s write, 15.95 GB read, 926.53 MB/s read, 141.0 seconds Interval compaction: 17.67 GB write, 1036.03 MB/s write, 15.95 GB read, 935.67 MB/s read, 141.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 16, pending-compaction-bytes-delays: 32, pending-compaction-bytes-stops: 0, total-delays: 32, total-stops: 16, interval: 47 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 494160 Average: 6.9540 StdDev: 96.96 Min: 1 Median: 1.6523 Max: 25357 Percentiles: P50: 1.65 P75: 2.25 P99: 9.36 P99.9: 1077.22 P99.99: 2622.04 ------------------------------------------------------ [ 0, 1 ] 52220 10.567% 10.567% ## ( 1, 2 ] 298724 60.451% 71.018% ############ ( 2, 3 ] 80133 16.216% 87.234% ### ( 3, 4 ] 30076 6.086% 93.321% # ( 4, 6 ] 23919 4.840% 98.161% # ( 6, 10 ] 4942 1.000% 99.161% ( 10, 15 ] 913 0.185% 99.346% ( 15, 22 ] 122 0.025% 99.370% ( 22, 34 ] 8 0.002% 99.372% ( 34, 51 ] 19 0.004% 99.376% ( 51, 76 ] 13 0.003% 99.379% ( 76, 110 ] 192 0.039% 99.417% ( 110, 170 ] 701 0.142% 99.559% ( 170, 250 ] 529 0.107% 99.666% ( 250, 380 ] 321 0.065% 99.731% ( 380, 580 ] 350 0.071% 99.802% ( 580, 870 ] 334 0.068% 99.870% ( 870, 1300 ] 313 0.063% 99.933% ( 1300, 1900 ] 175 0.035% 99.968% ( 1900, 2900 ] 149 0.030% 99.999% ( 2900, 4400 ] 127 0.026% 100.024% ( 4400, 6600 ] 18 0.004% 100.028% ( 6600, 9900 ] 1 0.000% 100.028% ( 22000, 33000 ] 1 0.000% 100.028% ** Level 1 read latency histogram (micros): Count: 509564 Average: 7.2288 StdDev: 99.13 Min: 1 Median: 1.9104 Max: 30845 Percentiles: P50: 1.91 P75: 2.91 P99: 9.92 P99.9: 837.05 P99.99: 1353.73 ------------------------------------------------------ [ 0, 1 ] 15838 3.108% 3.108% # ( 1, 2 ] 262455 51.506% 54.614% ########## ( 2, 3 ] 114434 22.457% 77.071% #### ( 3, 4 ] 50161 9.844% 86.915% ## ( 4, 6 ] 49704 9.754% 96.669% ## ( 6, 10 ] 12110 2.377% 99.046% ( 10, 15 ] 2172 0.426% 99.472% ( 15, 22 ] 268 0.053% 99.525% ( 22, 34 ] 44 0.009% 99.533% ( 34, 51 ] 78 0.015% 99.549% ( 51, 76 ] 39 0.008% 99.556% ( 76, 110 ] 93 0.018% 99.575% ( 110, 170 ] 224 0.044% 99.618% ( 170, 250 ] 249 0.049% 99.667% ( 250, 380 ] 361 0.071% 99.738% ( 380, 580 ] 430 0.084% 99.823% ( 580, 870 ] 445 0.087% 99.910% ( 870, 1300 ] 387 0.076% 99.986% ( 1300, 1900 ] 235 0.046% 100.032% ( 1900, 2900 ] 136 0.027% 100.059% ( 2900, 4400 ] 78 0.015% 100.074% ( 4400, 6600 ] 6 0.001% 100.075% ( 6600, 9900 ] 3 0.001% 100.076% ( 14000, 22000 ] 2 0.000% 100.076% ( 22000, 33000 ] 1 0.000% 100.076% ** Level 2 read latency histogram (micros): Count: 2253238 Average: 12.1166 StdDev: 161.16 Min: 0 Median: 1.9938 Max: 24055 Percentiles: P50: 1.99 P75: 3.08 P99: 9.89 P99.9: 665.82 P99.99: 963.42 ------------------------------------------------------ [ 0, 1 ] 37220 1.652% 1.652% ( 1, 2 ] 1096232 48.651% 50.303% ########## ( 2, 3 ] 535059 23.746% 74.049% ##### ( 3, 4 ] 258622 11.478% 85.527% ## ( 4, 6 ] 243885 10.824% 96.351% ## ( 6, 10 ] 61339 2.722% 99.073% # ( 10, 15 ] 12053 0.535% 99.608% ( 15, 22 ] 1613 0.072% 99.680% ( 22, 34 ] 174 0.008% 99.688% ( 34, 51 ] 228 0.010% 99.698% ( 51, 76 ] 159 0.007% 99.705% ( 76, 110 ] 163 0.007% 99.712% ( 110, 170 ] 431 0.019% 99.731% ( 170, 250 ] 619 0.027% 99.759% ( 250, 380 ] 914 0.041% 99.799% ( 380, 580 ] 1626 0.072% 99.871% ( 580, 870 ] 2189 0.097% 99.968% ( 870, 1300 ] 2240 0.099% 100.068% ( 1300, 1900 ] 1632 0.072% 100.140% ( 1900, 2900 ] 1729 0.077% 100.217% ( 2900, 4400 ] 1669 0.074% 100.291% ( 4400, 6600 ] 409 0.018% 100.309% ( 6600, 9900 ] 65 0.003% 100.312% ( 9900, 14000 ] 13 0.001% 100.313% ( 14000, 22000 ] 14 0.001% 100.313% ( 22000, 33000 ] 1 0.000% 100.313% ** Level 3 read latency histogram (micros): Count: 627600 Average: 17.3971 StdDev: 184.36 Min: 1 Median: 1.9333 Max: 27920 Percentiles: P50: 1.93 P75: 2.93 P99: 229.13 P99.9: 2499.75 P99.99: 3530.86 ------------------------------------------------------ [ 0, 1 ] 14641 2.333% 2.333% ( 1, 2 ] 320535 51.073% 53.406% ########## ( 2, 3 ] 145981 23.260% 76.666% ##### ( 3, 4 ] 64962 10.351% 87.017% ## ( 4, 6 ] 55160 8.789% 95.806% ## ( 6, 10 ] 14737 2.348% 98.154% ( 10, 15 ] 2641 0.421% 98.575% ( 15, 22 ] 381 0.061% 98.636% ( 22, 34 ] 43 0.007% 98.643% ( 34, 51 ] 82 0.013% 98.656% ( 51, 76 ] 57 0.009% 98.665% ( 76, 110 ] 284 0.045% 98.710% ( 110, 170 ] 1154 0.184% 98.894% ( 170, 250 ] 901 0.144% 99.037% ( 250, 380 ] 748 0.119% 99.157% ( 380, 580 ] 1018 0.162% 99.319% ( 580, 870 ] 1262 0.201% 99.520% ( 870, 1300 ] 1138 0.181% 99.701% ( 1300, 1900 ] 776 0.124% 99.825% ( 1900, 2900 ] 786 0.125% 99.950% ( 2900, 4400 ] 595 0.095% 100.045% ( 4400, 6600 ] 110 0.018% 100.062% ( 6600, 9900 ] 14 0.002% 100.065% ( 9900, 14000 ] 4 0.001% 100.065% ( 14000, 22000 ] 2 0.000% 100.066% ( 22000, 33000 ] 1 0.000% 100.066% ** Level 4 read latency histogram (micros): Count: 8265 Average: 701.8361 StdDev: 1040.75 Min: 76 Median: 310.0497 Max: 23325 Percentiles: P50: 310.05 P75: 777.52 P99: 5157.91 P99.9: 8915.50 P99.99: 23325.00 ------------------------------------------------------ ( 51, 76 ] 1 0.012% 0.012% ( 76, 110 ] 635 7.683% 7.695% ## ( 110, 170 ] 1823 22.057% 29.752% #### ( 170, 250 ] 1255 15.185% 44.936% ### ( 250, 380 ] 906 10.962% 55.898% ## ( 380, 580 ] 993 12.015% 67.913% ## ( 580, 870 ] 860 10.405% 78.318% ## ( 870, 1300 ] 627 7.586% 85.904% ## ( 1300, 1900 ] 417 5.045% 90.950% # ( 1900, 2900 ] 300 3.630% 94.580% # ( 2900, 4400 ] 334 4.041% 98.621% # ( 4400, 6600 ] 91 1.101% 99.722% ( 6600, 9900 ] 21 0.254% 99.976% ( 22000, 33000 ] 2 0.024% 100.000% ** Level 5 read latency histogram (micros): Count: 73330 Average: 713.1447 StdDev: 1039.78 Min: 73 Median: 321.0977 Max: 25027 Percentiles: P50: 321.10 P75: 792.98 P99: 5113.37 P99.9: 8857.54 P99.99: 19576.00 ------------------------------------------------------ ( 51, 76 ] 4 0.005% 0.005% ( 76, 110 ] 4994 6.810% 6.816% # ( 110, 170 ] 16583 22.614% 29.430% ##### ( 170, 250 ] 10630 14.496% 43.926% ### ( 250, 380 ] 8144 11.106% 55.032% ## ( 380, 580 ] 9017 12.296% 67.329% ## ( 580, 870 ] 7661 10.447% 77.776% ## ( 870, 1300 ] 5820 7.937% 85.713% ## ( 1300, 1900 ] 3620 4.937% 90.649% # ( 1900, 2900 ] 2866 3.908% 94.557% # ( 2900, 4400 ] 2986 4.072% 98.629% # ( 4400, 6600 ] 841 1.147% 99.776% ( 6600, 9900 ] 134 0.183% 99.959% ( 9900, 14000 ] 16 0.022% 99.981% ( 14000, 22000 ] 11 0.015% 99.996% ( 22000, 33000 ] 6 0.008% 100.004% ** DB Stats ** Uptime(secs): 17.6 total, 3.6 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 111.63 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.875 H:M:S, 56.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 276.69 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/11/21-05:59:27 ... thread 0: (1000000,3000000) ops and (232432.6,137859.2) ops/second in (4.302323,21.761327) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/10 2.09 GB 88.1 1.8 0.0 1.8 4.6 2.8 0.0 1.6 89.5 228.2 20.42 15.13 49 0.417 1855K 1809 0.0 0.0 L1 3/3 63.95 MB 0.0 1.1 0.7 0.4 1.1 0.7 0.0 1.5 162.1 161.8 7.22 5.29 3 2.406 1187K 2010 0.0 0.0 L2 1/0 46.72 MB 0.5 5.4 1.1 4.3 5.3 1.0 0.7 4.9 109.1 107.9 50.27 25.84 17 2.957 1121K 0 0.0 0.0 L3 18/0 990.20 MB 1.0 8.9 5.8 3.1 8.9 5.8 0.6 1.5 125.1 124.8 72.82 43.05 51 1.428 7725K 8034 0.0 0.0 L4 160/0 9.49 GB 1.0 1.8 0.5 1.3 1.8 0.5 5.9 3.5 152.3 151.5 12.07 8.66 7 1.725 1867K 9925 0.0 0.0 L5 1128/0 92.71 GB 0.9 0.0 0.0 0.0 0.0 0.0 6.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1324/13 105.36 GB 0.0 19.0 8.1 10.8 21.7 10.8 13.8 7.8 119.3 136.2 162.81 97.97 127 1.282 13M 21K 0.0 0.0 Int 0/0 0.00 KB 0.0 19.0 8.1 10.8 21.7 10.8 13.8 7.8 119.3 136.2 162.81 97.97 127 1.282 13M 21K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 19.0 8.1 10.8 18.9 8.0 0.0 0.0 129.4 128.8 150.08 89.99 85 1.766 13M 21K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 222.8 12.73 7.98 42 0.303 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 21.9 total, 21.8 interval Flush(GB): cumulative 2.769, interval 2.769 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 21.65 GB write, 1010.81 MB/s write, 18.97 GB read, 885.58 MB/s read, 162.8 seconds Interval compaction: 21.65 GB write, 1018.81 MB/s write, 18.97 GB read, 892.58 MB/s read, 162.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 20, pending-compaction-bytes-delays: 32, pending-compaction-bytes-stops: 0, total-delays: 32, total-stops: 20, interval: 51 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 895652 Average: 6.3487 StdDev: 83.68 Min: 0 Median: 1.6530 Max: 25357 Percentiles: P50: 1.65 P75: 2.23 P99: 9.31 P99.9: 820.52 P99.99: 1886.14 ------------------------------------------------------ [ 0, 1 ] 90625 10.118% 10.118% ## ( 1, 2 ] 547044 61.078% 71.196% ############ ( 2, 3 ] 146003 16.301% 87.497% ### ( 3, 4 ] 53544 5.978% 93.476% # ( 4, 6 ] 42163 4.708% 98.183% # ( 6, 10 ] 8834 0.986% 99.169% ( 10, 15 ] 1596 0.178% 99.348% ( 15, 22 ] 187 0.021% 99.369% ( 22, 34 ] 13 0.001% 99.370% ( 34, 51 ] 32 0.004% 99.374% ( 51, 76 ] 28 0.003% 99.377% ( 76, 110 ] 332 0.037% 99.414% ( 110, 170 ] 1341 0.150% 99.563% ( 170, 250 ] 1023 0.114% 99.678% ( 250, 380 ] 724 0.081% 99.758% ( 380, 580 ] 745 0.083% 99.842% ( 580, 870 ] 631 0.070% 99.912% ( 870, 1300 ] 463 0.052% 99.964% ( 1300, 1900 ] 241 0.027% 99.991% ( 1900, 2900 ] 197 0.022% 100.013% ( 2900, 4400 ] 176 0.020% 100.032% ( 4400, 6600 ] 22 0.002% 100.035% ( 6600, 9900 ] 1 0.000% 100.035% ( 22000, 33000 ] 1 0.000% 100.035% ** Level 1 read latency histogram (micros): Count: 529738 Average: 7.2385 StdDev: 98.18 Min: 1 Median: 1.9093 Max: 30845 Percentiles: P50: 1.91 P75: 2.90 P99: 9.94 P99.9: 842.57 P99.99: 1374.75 ------------------------------------------------------ [ 0, 1 ] 16401 3.096% 3.096% # ( 1, 2 ] 273256 51.583% 54.679% ########## ( 2, 3 ] 119017 22.467% 77.146% #### ( 3, 4 ] 52057 9.827% 86.973% ## ( 4, 6 ] 51415 9.706% 96.679% ## ( 6, 10 ] 12470 2.354% 99.033% ( 10, 15 ] 2235 0.422% 99.455% ( 15, 22 ] 271 0.051% 99.506% ( 22, 34 ] 44 0.008% 99.514% ( 34, 51 ] 79 0.015% 99.529% ( 51, 76 ] 41 0.008% 99.537% ( 76, 110 ] 102 0.019% 99.556% ( 110, 170 ] 265 0.050% 99.606% ( 170, 250 ] 276 0.052% 99.659% ( 250, 380 ] 394 0.074% 99.733% ( 380, 580 ] 457 0.086% 99.819% ( 580, 870 ] 473 0.089% 99.908% ( 870, 1300 ] 402 0.076% 99.984% ( 1300, 1900 ] 241 0.045% 100.030% ( 1900, 2900 ] 140 0.026% 100.056% ( 2900, 4400 ] 80 0.015% 100.071% ( 4400, 6600 ] 6 0.001% 100.072% ( 6600, 9900 ] 3 0.001% 100.073% ( 14000, 22000 ] 2 0.000% 100.073% ( 22000, 33000 ] 1 0.000% 100.074% ** Level 2 read latency histogram (micros): Count: 2253238 Average: 12.1166 StdDev: 161.16 Min: 0 Median: 1.9938 Max: 24055 Percentiles: P50: 1.99 P75: 3.08 P99: 9.89 P99.9: 665.82 P99.99: 963.42 ------------------------------------------------------ [ 0, 1 ] 37220 1.652% 1.652% ( 1, 2 ] 1096232 48.651% 50.303% ########## ( 2, 3 ] 535059 23.746% 74.049% ##### ( 3, 4 ] 258622 11.478% 85.527% ## ( 4, 6 ] 243885 10.824% 96.351% ## ( 6, 10 ] 61339 2.722% 99.073% # ( 10, 15 ] 12053 0.535% 99.608% ( 15, 22 ] 1613 0.072% 99.680% ( 22, 34 ] 174 0.008% 99.688% ( 34, 51 ] 228 0.010% 99.698% ( 51, 76 ] 159 0.007% 99.705% ( 76, 110 ] 163 0.007% 99.712% ( 110, 170 ] 431 0.019% 99.731% ( 170, 250 ] 619 0.027% 99.759% ( 250, 380 ] 914 0.041% 99.799% ( 380, 580 ] 1626 0.072% 99.871% ( 580, 870 ] 2189 0.097% 99.968% ( 870, 1300 ] 2240 0.099% 100.068% ( 1300, 1900 ] 1632 0.072% 100.140% ( 1900, 2900 ] 1729 0.077% 100.217% ( 2900, 4400 ] 1669 0.074% 100.291% ( 4400, 6600 ] 409 0.018% 100.309% ( 6600, 9900 ] 65 0.003% 100.312% ( 9900, 14000 ] 13 0.001% 100.313% ( 14000, 22000 ] 14 0.001% 100.313% ( 22000, 33000 ] 1 0.000% 100.313% ** Level 3 read latency histogram (micros): Count: 1140942 Average: 13.7988 StdDev: 161.80 Min: 1 Median: 1.9099 Max: 35315 Percentiles: P50: 1.91 P75: 2.87 P99: 122.72 P99.9: 1512.13 P99.99: 2326.53 ------------------------------------------------------ [ 0, 1 ] 23315 2.043% 2.043% ( 1, 2 ] 601305 52.703% 54.746% ########### ( 2, 3 ] 264737 23.203% 77.949% ##### ( 3, 4 ] 112232 9.837% 87.786% ## ( 4, 6 ] 95461 8.367% 96.153% ## ( 6, 10 ] 26455 2.319% 98.472% ( 10, 15 ] 4253 0.373% 98.844% ( 15, 22 ] 539 0.047% 98.892% ( 22, 34 ] 64 0.006% 98.897% ( 34, 51 ] 139 0.012% 98.909% ( 51, 76 ] 120 0.011% 98.920% ( 76, 110 ] 498 0.044% 98.964% ( 110, 170 ] 1955 0.171% 99.135% ( 170, 250 ] 1556 0.136% 99.271% ( 250, 380 ] 1455 0.128% 99.399% ( 380, 580 ] 1820 0.160% 99.558% ( 580, 870 ] 1982 0.174% 99.732% ( 870, 1300 ] 1576 0.138% 99.870% ( 1300, 1900 ] 959 0.084% 99.954% ( 1900, 2900 ] 954 0.084% 100.038% ( 2900, 4400 ] 765 0.067% 100.105% ( 4400, 6600 ] 128 0.011% 100.116% ( 6600, 9900 ] 22 0.002% 100.118% ( 9900, 14000 ] 6 0.001% 100.119% ( 14000, 22000 ] 5 0.000% 100.119% ( 22000, 33000 ] 2 0.000% 100.119% ( 33000, 50000 ] 1 0.000% 100.119% ** Level 4 read latency histogram (micros): Count: 12031 Average: 594.0539 StdDev: 917.34 Min: 1 Median: 259.6143 Max: 23325 Percentiles: P50: 259.61 P75: 640.46 P99: 4390.56 P99.9: 8395.35 P99.99: 23325.00 ------------------------------------------------------ [ 0, 1 ] 35 0.291% 0.291% ( 1, 2 ] 82 0.682% 0.972% ( 2, 3 ] 93 0.773% 1.745% ( 3, 4 ] 51 0.424% 2.169% ( 4, 6 ] 73 0.607% 2.776% ( 6, 10 ] 10 0.083% 2.859% ( 10, 15 ] 5 0.042% 2.901% ( 15, 22 ] 1 0.008% 2.909% ( 51, 76 ] 1 0.008% 2.917% ( 76, 110 ] 849 7.057% 9.974% # ( 110, 170 ] 2738 22.758% 32.732% ##### ( 170, 250 ] 1973 16.399% 49.131% ### ( 250, 380 ] 1413 11.745% 60.876% ## ( 380, 580 ] 1452 12.069% 72.945% ## ( 580, 870 ] 1186 9.858% 82.803% ## ( 870, 1300 ] 797 6.625% 89.427% # ( 1300, 1900 ] 463 3.848% 93.276% # ( 1900, 2900 ] 324 2.693% 95.969% # ( 2900, 4400 ] 367 3.050% 99.019% # ( 4400, 6600 ] 94 0.781% 99.801% ( 6600, 9900 ] 22 0.183% 99.983% ( 22000, 33000 ] 2 0.017% 100.000% ** Level 5 read latency histogram (micros): Count: 102789 Average: 626.4032 StdDev: 959.93 Min: 73 Median: 290.4713 Max: 45180 Percentiles: P50: 290.47 P75: 683.03 P99: 4467.60 P99.9: 8266.86 P99.99: 18751.38 ------------------------------------------------------ ( 51, 76 ] 7 0.007% 0.007% ( 76, 110 ] 6837 6.651% 6.658% # ( 110, 170 ] 24039 23.387% 30.045% ##### ( 170, 250 ] 16595 16.145% 46.190% ### ( 250, 380 ] 12582 12.241% 58.430% ## ( 380, 580 ] 13261 12.901% 71.332% ### ( 580, 870 ] 10618 10.330% 81.661% ## ( 870, 1300 ] 7323 7.124% 88.786% # ( 1300, 1900 ] 4136 4.024% 92.810% # ( 1900, 2900 ] 3147 3.062% 95.871% # ( 2900, 4400 ] 3191 3.104% 98.976% # ( 4400, 6600 ] 882 0.858% 99.834% ( 6600, 9900 ] 139 0.135% 99.969% ( 9900, 14000 ] 16 0.016% 99.984% ( 14000, 22000 ] 13 0.013% 99.997% ( 22000, 33000 ] 6 0.006% 100.003% ( 33000, 50000 ] 4 0.004% 100.007% ** DB Stats ** Uptime(secs): 21.9 total, 4.3 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 134.60 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:10.432 H:M:S, 47.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 228.76 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.557 H:M:S, 12.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/21-05:59:30 ... thread 0: (1000000,4000000) ops and (264803.4,156631.1) ops/second in (3.776386,25.537713) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/10 3.07 GB 188.8 2.8 0.0 2.8 6.5 3.7 0.0 1.7 103.3 243.1 27.46 21.66 69 0.398 2878K 2762 0.0 0.0 L1 3/3 63.95 MB 0.0 1.1 0.7 0.4 1.1 0.7 0.0 1.5 162.1 161.8 7.22 5.29 3 2.406 1187K 2010 0.0 0.0 L2 1/0 46.72 MB 0.5 5.4 1.1 4.3 5.3 1.0 0.7 4.9 109.1 107.9 50.27 25.84 17 2.957 1121K 0 0.0 0.0 L3 18/0 990.20 MB 1.0 8.9 5.8 3.1 8.9 5.8 0.6 1.5 125.1 124.8 72.82 43.05 51 1.428 7725K 8034 0.0 0.0 L4 160/0 9.49 GB 1.0 1.8 0.5 1.3 1.8 0.5 5.9 3.5 152.3 151.5 12.07 8.66 7 1.725 1867K 9925 0.0 0.0 L5 1128/0 92.71 GB 0.9 0.0 0.0 0.0 0.0 0.0 6.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1328/13 106.34 GB 0.0 20.0 8.1 11.8 23.6 11.8 13.8 6.3 120.3 142.4 169.84 104.50 147 1.155 14M 22K 0.0 0.0 Int 0/0 0.00 KB 0.0 20.0 8.1 11.8 23.6 11.8 13.8 6.3 120.3 142.4 169.84 104.50 147 1.155 14M 22K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 20.0 8.1 11.8 19.9 8.0 0.0 0.0 132.7 132.1 153.95 93.79 89 1.730 14M 22K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 241.9 15.89 10.71 58 0.274 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 25.7 total, 25.5 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 23.62 GB write, 940.72 MB/s write, 19.95 GB read, 794.71 MB/s read, 169.8 seconds Interval compaction: 23.62 GB write, 947.06 MB/s write, 19.95 GB read, 800.07 MB/s read, 169.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 3, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 3, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 20, pending-compaction-bytes-delays: 32, pending-compaction-bytes-stops: 0, total-delays: 35, total-stops: 20, interval: 54 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1186956 Average: 8.9699 StdDev: 80.48 Min: 0 Median: 1.6032 Max: 25357 Percentiles: P50: 1.60 P75: 2.10 P99: 174.54 P99.9: 730.11 P99.99: 1429.81 ------------------------------------------------------ [ 0, 1 ] 168292 14.178% 14.178% ### ( 1, 2 ] 704837 59.382% 73.560% ############ ( 2, 3 ] 166860 14.058% 87.618% ### ( 3, 4 ] 61016 5.141% 92.759% # ( 4, 6 ] 45771 3.856% 96.615% # ( 6, 10 ] 9773 0.823% 97.438% ( 10, 15 ] 1902 0.160% 97.598% ( 15, 22 ] 250 0.021% 97.620% ( 22, 34 ] 19 0.002% 97.621% ( 34, 51 ] 34 0.003% 97.624% ( 51, 76 ] 41 0.003% 97.627% ( 76, 110 ] 3394 0.286% 97.913% ( 110, 170 ] 12562 1.058% 98.972% ( 170, 250 ] 5905 0.497% 99.469% ( 250, 380 ] 2511 0.212% 99.681% ( 380, 580 ] 2072 0.175% 99.855% ( 580, 870 ] 1024 0.086% 99.942% ( 870, 1300 ] 520 0.044% 99.985% ( 1300, 1900 ] 251 0.021% 100.007% ( 1900, 2900 ] 221 0.019% 100.025% ( 2900, 4400 ] 190 0.016% 100.041% ( 4400, 6600 ] 22 0.002% 100.043% ( 6600, 9900 ] 1 0.000% 100.043% ( 22000, 33000 ] 1 0.000% 100.043% ** Level 1 read latency histogram (micros): Count: 530085 Average: 7.3652 StdDev: 98.47 Min: 1 Median: 1.9099 Max: 30845 Percentiles: P50: 1.91 P75: 2.91 P99: 10.38 P99.9: 843.81 P99.99: 1379.65 ------------------------------------------------------ [ 0, 1 ] 16401 3.094% 3.094% # ( 1, 2 ] 273256 51.549% 54.644% ########## ( 2, 3 ] 119017 22.452% 77.096% #### ( 3, 4 ] 52057 9.821% 86.916% ## ( 4, 6 ] 51415 9.699% 96.616% ## ( 6, 10 ] 12470 2.352% 98.968% ( 10, 15 ] 2235 0.422% 99.390% ( 15, 22 ] 271 0.051% 99.441% ( 22, 34 ] 44 0.008% 99.449% ( 34, 51 ] 79 0.015% 99.464% ( 51, 76 ] 41 0.008% 99.472% ( 76, 110 ] 153 0.029% 99.501% ( 110, 170 ] 442 0.083% 99.584% ( 170, 250 ] 343 0.065% 99.649% ( 250, 380 ] 417 0.079% 99.728% ( 380, 580 ] 480 0.091% 99.818% ( 580, 870 ] 477 0.090% 99.908% ( 870, 1300 ] 402 0.076% 99.984% ( 1300, 1900 ] 241 0.045% 100.029% ( 1900, 2900 ] 141 0.027% 100.056% ( 2900, 4400 ] 81 0.015% 100.071% ( 4400, 6600 ] 6 0.001% 100.072% ( 6600, 9900 ] 3 0.001% 100.073% ( 14000, 22000 ] 2 0.000% 100.073% ( 22000, 33000 ] 1 0.000% 100.074% ** Level 2 read latency histogram (micros): Count: 2253238 Average: 12.1166 StdDev: 161.16 Min: 0 Median: 1.9938 Max: 24055 Percentiles: P50: 1.99 P75: 3.08 P99: 9.89 P99.9: 665.82 P99.99: 963.42 ------------------------------------------------------ [ 0, 1 ] 37220 1.652% 1.652% ( 1, 2 ] 1096232 48.651% 50.303% ########## ( 2, 3 ] 535059 23.746% 74.049% ##### ( 3, 4 ] 258622 11.478% 85.527% ## ( 4, 6 ] 243885 10.824% 96.351% ## ( 6, 10 ] 61339 2.722% 99.073% # ( 10, 15 ] 12053 0.535% 99.608% ( 15, 22 ] 1613 0.072% 99.680% ( 22, 34 ] 174 0.008% 99.688% ( 34, 51 ] 228 0.010% 99.698% ( 51, 76 ] 159 0.007% 99.705% ( 76, 110 ] 163 0.007% 99.712% ( 110, 170 ] 431 0.019% 99.731% ( 170, 250 ] 619 0.027% 99.759% ( 250, 380 ] 914 0.041% 99.799% ( 380, 580 ] 1626 0.072% 99.871% ( 580, 870 ] 2189 0.097% 99.968% ( 870, 1300 ] 2240 0.099% 100.068% ( 1300, 1900 ] 1632 0.072% 100.140% ( 1900, 2900 ] 1729 0.077% 100.217% ( 2900, 4400 ] 1669 0.074% 100.291% ( 4400, 6600 ] 409 0.018% 100.309% ( 6600, 9900 ] 65 0.003% 100.312% ( 9900, 14000 ] 13 0.001% 100.313% ( 14000, 22000 ] 14 0.001% 100.313% ( 22000, 33000 ] 1 0.000% 100.313% ** Level 3 read latency histogram (micros): Count: 1144946 Average: 14.4328 StdDev: 162.10 Min: 1 Median: 1.9133 Max: 35315 Percentiles: P50: 1.91 P75: 2.88 P99: 168.49 P99.9: 1513.74 P99.99: 2328.50 ------------------------------------------------------ [ 0, 1 ] 23315 2.036% 2.036% ( 1, 2 ] 601305 52.518% 54.555% ########### ( 2, 3 ] 264737 23.122% 77.677% ##### ( 3, 4 ] 112232 9.802% 87.479% ## ( 4, 6 ] 95461 8.338% 95.817% ## ( 6, 10 ] 26455 2.311% 98.127% ( 10, 15 ] 4253 0.371% 98.499% ( 15, 22 ] 539 0.047% 98.546% ( 22, 34 ] 64 0.006% 98.551% ( 34, 51 ] 139 0.012% 98.564% ( 51, 76 ] 120 0.010% 98.574% ( 76, 110 ] 1030 0.090% 98.664% ( 110, 170 ] 3946 0.345% 99.009% ( 170, 250 ] 2383 0.208% 99.217% ( 250, 380 ] 1779 0.155% 99.372% ( 380, 580 ] 2061 0.180% 99.552% ( 580, 870 ] 2050 0.179% 99.731% ( 870, 1300 ] 1589 0.139% 99.870% ( 1300, 1900 ] 963 0.084% 99.954% ( 1900, 2900 ] 958 0.084% 100.038% ( 2900, 4400 ] 765 0.067% 100.105% ( 4400, 6600 ] 128 0.011% 100.116% ( 6600, 9900 ] 22 0.002% 100.118% ( 9900, 14000 ] 6 0.001% 100.118% ( 14000, 22000 ] 5 0.000% 100.119% ( 22000, 33000 ] 2 0.000% 100.119% ( 33000, 50000 ] 1 0.000% 100.119% ** Level 4 read latency histogram (micros): Count: 18543 Average: 444.4506 StdDev: 771.48 Min: 1 Median: 191.3891 Max: 23325 Percentiles: P50: 191.39 P75: 436.34 P99: 4125.15 P99.9: 7418.55 P99.99: 22801.35 ------------------------------------------------------ [ 0, 1 ] 35 0.189% 0.189% ( 1, 2 ] 160 0.863% 1.052% ( 2, 3 ] 400 2.157% 3.209% ( 3, 4 ] 247 1.332% 4.541% ( 4, 6 ] 273 1.472% 6.013% ( 6, 10 ] 39 0.210% 6.223% ( 10, 15 ] 14 0.076% 6.299% ( 15, 22 ] 1 0.005% 6.304% ( 51, 76 ] 2 0.011% 6.315% ( 76, 110 ] 1673 9.022% 15.337% ## ( 110, 170 ] 5596 30.179% 45.516% ###### ( 170, 250 ] 3110 16.772% 62.288% ### ( 250, 380 ] 1866 10.063% 72.351% ## ( 380, 580 ] 1744 9.405% 81.756% ## ( 580, 870 ] 1289 6.951% 88.707% # ( 870, 1300 ] 813 4.384% 93.092% # ( 1300, 1900 ] 466 2.513% 95.605% # ( 1900, 2900 ] 329 1.774% 97.379% ( 2900, 4400 ] 368 1.985% 99.364% ( 4400, 6600 ] 94 0.507% 99.871% ( 6600, 9900 ] 22 0.119% 99.989% ( 22000, 33000 ] 2 0.011% 100.000% ** Level 5 read latency histogram (micros): Count: 150798 Average: 488.1258 StdDev: 821.86 Min: 73 Median: 211.5567 Max: 45180 Percentiles: P50: 211.56 P75: 490.52 P99: 4186.73 P99.9: 7079.62 P99.99: 14566.28 ------------------------------------------------------ ( 51, 76 ] 12 0.008% 0.008% ( 76, 110 ] 13805 9.155% 9.163% ## ( 110, 170 ] 47981 31.818% 40.981% ###### ( 170, 250 ] 26183 17.363% 58.344% ### ( 250, 380 ] 16206 10.747% 69.090% ## ( 380, 580 ] 16127 10.694% 79.785% ## ( 580, 870 ] 11451 7.594% 87.378% ## ( 870, 1300 ] 7437 4.932% 92.310% # ( 1300, 1900 ] 4170 2.765% 95.076% # ( 1900, 2900 ] 3173 2.104% 97.180% ( 2900, 4400 ] 3200 2.122% 99.302% ( 4400, 6600 ] 882 0.585% 99.887% ( 6600, 9900 ] 139 0.092% 99.979% ( 9900, 14000 ] 16 0.011% 99.989% ( 14000, 22000 ] 13 0.009% 99.998% ( 22000, 33000 ] 6 0.004% 100.002% ( 33000, 50000 ] 4 0.003% 100.005% ** DB Stats ** Uptime(secs): 25.7 total, 3.8 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 153.11 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:10.736 H:M:S, 41.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 260.62 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.304 H:M:S, 8.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/21-05:59:35 ... thread 0: (1000000,5000000) ops and (237333.3,168060.5) ops/second in (4.213484,29.751197) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/16 3.07 GB 56.7 3.5 0.0 3.5 8.2 4.7 0.0 1.7 106.8 251.0 33.62 27.35 88 0.382 3646K 3490 0.0 0.0 L1 12/12 1.04 GB 0.0 2.2 1.7 0.5 2.2 1.7 0.0 1.3 134.7 134.3 16.63 10.77 4 4.158 2275K 6712 0.0 0.0 L2 1/0 46.72 MB 0.5 5.4 1.1 4.3 5.3 1.0 0.7 4.9 109.1 107.9 50.27 25.84 17 2.957 1121K 0 0.0 0.0 L3 18/0 990.20 MB 1.0 8.9 5.8 3.1 8.9 5.8 0.6 1.5 125.1 124.8 72.82 43.05 51 1.428 7725K 8034 0.0 0.0 L4 160/0 9.49 GB 1.0 1.8 0.5 1.3 1.8 0.5 5.9 3.5 152.3 151.5 12.07 8.66 7 1.725 1867K 9925 0.0 0.0 L5 1128/0 92.71 GB 0.9 0.0 0.0 0.0 0.0 0.0 6.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1338/28 107.32 GB 0.0 21.7 9.1 12.6 26.4 13.8 13.8 5.6 120.1 145.7 185.42 115.68 167 1.110 16M 28K 0.0 0.0 Int 0/0 0.00 KB 0.0 21.7 9.1 12.6 26.4 13.8 13.8 5.6 120.1 145.7 185.42 115.68 167 1.110 16M 28K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 21.7 9.1 12.6 21.6 9.0 0.0 0.0 133.9 133.3 166.30 102.16 93 1.788 16M 28K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 253.7 19.12 13.52 74 0.258 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 29.9 total, 29.8 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 26.38 GB write, 902.81 MB/s write, 21.74 GB read, 743.87 MB/s read, 185.4 seconds Interval compaction: 26.38 GB write, 908.04 MB/s write, 21.74 GB read, 748.18 MB/s read, 185.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 7, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 7, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 20, pending-compaction-bytes-delays: 32, pending-compaction-bytes-stops: 0, total-delays: 39, total-stops: 20, interval: 58 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1453942 Average: 9.9642 StdDev: 77.71 Min: 0 Median: 1.5854 Max: 25357 Percentiles: P50: 1.59 P75: 2.06 P99: 202.17 P99.9: 682.26 P99.99: 1224.00 ------------------------------------------------------ [ 0, 1 ] 231060 15.892% 15.892% ### ( 1, 2 ] 847145 58.265% 74.157% ############ ( 2, 3 ] 194118 13.351% 87.509% ### ( 3, 4 ] 72408 4.980% 92.489% # ( 4, 6 ] 50654 3.484% 95.973% # ( 6, 10 ] 10983 0.755% 96.728% ( 10, 15 ] 2245 0.154% 96.882% ( 15, 22 ] 305 0.021% 96.903% ( 22, 34 ] 26 0.002% 96.905% ( 34, 51 ] 37 0.003% 96.908% ( 51, 76 ] 43 0.003% 96.911% ( 76, 110 ] 5825 0.401% 97.311% ( 110, 170 ] 20781 1.429% 98.741% ( 170, 250 ] 9381 0.645% 99.386% ( 250, 380 ] 3929 0.270% 99.656% ( 380, 580 ] 3090 0.213% 99.868% ( 580, 870 ] 1299 0.089% 99.958% ( 870, 1300 ] 568 0.039% 99.997% ( 1300, 1900 ] 262 0.018% 100.015% ( 1900, 2900 ] 234 0.016% 100.031% ( 2900, 4400 ] 204 0.014% 100.045% ( 4400, 6600 ] 23 0.002% 100.047% ( 6600, 9900 ] 1 0.000% 100.047% ( 22000, 33000 ] 1 0.000% 100.047% ** Level 1 read latency histogram (micros): Count: 547676 Average: 7.6029 StdDev: 97.72 Min: 1 Median: 1.9199 Max: 30845 Percentiles: P50: 1.92 P75: 2.92 P99: 12.51 P99.9: 841.25 P99.99: 1382.74 ------------------------------------------------------ [ 0, 1 ] 17264 3.152% 3.152% # ( 1, 2 ] 278906 50.925% 54.078% ########## ( 2, 3 ] 124007 22.642% 76.720% ##### ( 3, 4 ] 54235 9.903% 86.623% ## ( 4, 6 ] 53601 9.787% 96.410% ## ( 6, 10 ] 12998 2.373% 98.783% ( 10, 15 ] 2369 0.433% 99.216% ( 15, 22 ] 323 0.059% 99.275% ( 22, 34 ] 45 0.008% 99.283% ( 34, 51 ] 82 0.015% 99.298% ( 51, 76 ] 41 0.007% 99.305% ( 76, 110 ] 310 0.057% 99.362% ( 110, 170 ] 916 0.167% 99.529% ( 170, 250 ] 552 0.101% 99.630% ( 250, 380 ] 499 0.091% 99.721% ( 380, 580 ] 529 0.097% 99.818% ( 580, 870 ] 501 0.091% 99.909% ( 870, 1300 ] 410 0.075% 99.984% ( 1300, 1900 ] 241 0.044% 100.028% ( 1900, 2900 ] 142 0.026% 100.054% ( 2900, 4400 ] 83 0.015% 100.069% ( 4400, 6600 ] 6 0.001% 100.070% ( 6600, 9900 ] 3 0.001% 100.071% ( 14000, 22000 ] 2 0.000% 100.071% ( 22000, 33000 ] 1 0.000% 100.071% ** Level 2 read latency histogram (micros): Count: 2253238 Average: 12.1166 StdDev: 161.16 Min: 0 Median: 1.9938 Max: 24055 Percentiles: P50: 1.99 P75: 3.08 P99: 9.89 P99.9: 665.82 P99.99: 963.42 ------------------------------------------------------ [ 0, 1 ] 37220 1.652% 1.652% ( 1, 2 ] 1096232 48.651% 50.303% ########## ( 2, 3 ] 535059 23.746% 74.049% ##### ( 3, 4 ] 258622 11.478% 85.527% ## ( 4, 6 ] 243885 10.824% 96.351% ## ( 6, 10 ] 61339 2.722% 99.073% # ( 10, 15 ] 12053 0.535% 99.608% ( 15, 22 ] 1613 0.072% 99.680% ( 22, 34 ] 174 0.008% 99.688% ( 34, 51 ] 228 0.010% 99.698% ( 51, 76 ] 159 0.007% 99.705% ( 76, 110 ] 163 0.007% 99.712% ( 110, 170 ] 431 0.019% 99.731% ( 170, 250 ] 619 0.027% 99.759% ( 250, 380 ] 914 0.041% 99.799% ( 380, 580 ] 1626 0.072% 99.871% ( 580, 870 ] 2189 0.097% 99.968% ( 870, 1300 ] 2240 0.099% 100.068% ( 1300, 1900 ] 1632 0.072% 100.140% ( 1900, 2900 ] 1729 0.077% 100.217% ( 2900, 4400 ] 1669 0.074% 100.291% ( 4400, 6600 ] 409 0.018% 100.309% ( 6600, 9900 ] 65 0.003% 100.312% ( 9900, 14000 ] 13 0.001% 100.313% ( 14000, 22000 ] 14 0.001% 100.313% ( 22000, 33000 ] 1 0.000% 100.313% ** Level 3 read latency histogram (micros): Count: 1149542 Average: 15.1263 StdDev: 162.35 Min: 1 Median: 1.9171 Max: 35315 Percentiles: P50: 1.92 P75: 2.90 P99: 205.10 P99.9: 1513.11 P99.99: 2328.62 ------------------------------------------------------ [ 0, 1 ] 23315 2.028% 2.028% ( 1, 2 ] 601305 52.308% 54.336% ########## ( 2, 3 ] 264737 23.030% 77.366% ##### ( 3, 4 ] 112232 9.763% 87.129% ## ( 4, 6 ] 95461 8.304% 95.434% ## ( 6, 10 ] 26455 2.301% 97.735% ( 10, 15 ] 4253 0.370% 98.105% ( 15, 22 ] 539 0.047% 98.152% ( 22, 34 ] 64 0.006% 98.157% ( 34, 51 ] 139 0.012% 98.170% ( 51, 76 ] 121 0.011% 98.180% ( 76, 110 ] 1725 0.150% 98.330% ( 110, 170 ] 6278 0.546% 98.876% ( 170, 250 ] 3242 0.282% 99.158% ( 250, 380 ] 2141 0.186% 99.345% ( 380, 580 ] 2319 0.202% 99.546% ( 580, 870 ] 2125 0.185% 99.731% ( 870, 1300 ] 1598 0.139% 99.870% ( 1300, 1900 ] 967 0.084% 99.954% ( 1900, 2900 ] 959 0.083% 100.038% ( 2900, 4400 ] 765 0.067% 100.104% ( 4400, 6600 ] 128 0.011% 100.115% ( 6600, 9900 ] 22 0.002% 100.117% ( 9900, 14000 ] 6 0.001% 100.118% ( 14000, 22000 ] 5 0.000% 100.118% ( 22000, 33000 ] 2 0.000% 100.118% ( 33000, 50000 ] 1 0.000% 100.118% ** Level 4 read latency histogram (micros): Count: 26266 Average: 366.4989 StdDev: 664.17 Min: 1 Median: 166.2726 Max: 23325 Percentiles: P50: 166.27 P75: 332.33 P99: 3813.54 P99.9: 6546.97 P99.99: 9806.01 ------------------------------------------------------ [ 0, 1 ] 35 0.133% 0.133% ( 1, 2 ] 202 0.769% 0.902% ( 2, 3 ] 529 2.014% 2.916% ( 3, 4 ] 349 1.329% 4.245% ( 4, 6 ] 377 1.435% 5.680% ( 6, 10 ] 49 0.187% 5.867% ( 10, 15 ] 14 0.053% 5.920% ( 15, 22 ] 1 0.004% 5.924% ( 51, 76 ] 4 0.015% 5.939% ( 76, 110 ] 2877 10.953% 16.893% ## ( 110, 170 ] 9272 35.300% 52.193% ####### ( 170, 250 ] 4482 17.064% 69.257% ### ( 250, 380 ] 2382 9.069% 78.326% ## ( 380, 580 ] 2151 8.189% 86.515% ## ( 580, 870 ] 1416 5.391% 91.906% # ( 870, 1300 ] 833 3.171% 95.077% # ( 1300, 1900 ] 472 1.797% 96.874% ( 1900, 2900 ] 333 1.268% 98.142% ( 2900, 4400 ] 370 1.409% 99.551% ( 4400, 6600 ] 94 0.358% 99.909% ( 6600, 9900 ] 22 0.084% 99.992% ( 22000, 33000 ] 2 0.008% 100.000% ** Level 5 read latency histogram (micros): Count: 206205 Average: 407.5211 StdDev: 718.74 Min: 73 Median: 180.4813 Max: 45180 Percentiles: P50: 180.48 P75: 375.34 P99: 3927.89 P99.9: 6512.19 P99.99: 12816.00 ------------------------------------------------------ ( 51, 76 ] 15 0.007% 0.007% ( 76, 110 ] 22717 11.017% 11.024% ## ( 110, 170 ] 75550 36.638% 47.662% ####### ( 170, 250 ] 36793 17.843% 65.505% #### ( 250, 380 ] 20306 9.847% 75.353% ## ( 380, 580 ] 19217 9.319% 84.672% ## ( 580, 870 ] 12383 6.005% 90.677% # ( 870, 1300 ] 7560 3.666% 94.343% # ( 1300, 1900 ] 4205 2.039% 96.383% ( 1900, 2900 ] 3200 1.552% 97.935% ( 2900, 4400 ] 3206 1.555% 99.489% ( 4400, 6600 ] 882 0.428% 99.917% ( 6600, 9900 ] 139 0.067% 99.984% ( 9900, 14000 ] 16 0.008% 99.992% ( 14000, 22000 ] 13 0.006% 99.999% ( 22000, 33000 ] 6 0.003% 100.001% ( 33000, 50000 ] 4 0.002% 100.003% ** DB Stats ** Uptime(secs): 29.9 total, 4.2 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 164.44 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:11.509 H:M:S, 38.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 233.58 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.773 H:M:S, 18.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/21-05:59:44 ... thread 0: (1000000,6000000) ops and (110758.5,154719.5) ops/second in (9.028650,38.779847) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/16 3.99 GB 150.4 5.7 0.0 5.7 11.4 5.6 0.0 2.0 129.8 258.0 45.13 38.36 108 0.418 5947K 10K 0.0 0.0 L1 12/12 1.04 GB 0.0 2.2 1.7 0.5 2.2 1.7 0.0 1.3 134.7 134.3 16.63 10.77 4 4.158 2275K 6712 0.0 0.0 L2 1/0 46.72 MB 0.5 5.4 1.1 4.3 5.3 1.0 0.7 4.9 109.1 107.9 50.27 25.84 17 2.957 1121K 0 0.0 0.0 L3 18/0 990.20 MB 1.0 8.9 5.8 3.1 8.9 5.8 0.6 1.5 125.1 124.8 72.82 43.05 51 1.428 7725K 8034 0.0 0.0 L4 160/0 9.49 GB 1.0 1.8 0.5 1.3 1.8 0.5 5.9 3.5 152.3 151.5 12.07 8.66 7 1.725 1867K 9925 0.0 0.0 L5 1128/0 92.71 GB 0.9 0.0 0.0 0.0 0.0 0.0 6.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1337/28 108.24 GB 0.0 24.0 9.1 14.8 29.5 14.7 13.8 5.2 124.5 153.5 196.93 126.68 187 1.053 18M 35K 0.0 0.0 Int 0/0 0.00 KB 0.0 24.0 9.1 14.8 29.5 14.7 13.8 5.2 124.5 153.5 196.93 126.68 187 1.053 18M 35K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 24.0 9.1 14.8 23.9 9.0 0.0 0.0 140.3 139.7 174.84 110.56 98 1.784 18M 35K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 262.3 22.09 16.12 89 0.248 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 39.0 total, 38.8 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 29.51 GB write, 775.82 MB/s write, 23.95 GB read, 629.65 MB/s read, 196.9 seconds Interval compaction: 29.51 GB write, 779.26 MB/s write, 23.95 GB read, 632.44 MB/s read, 196.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 17, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 17, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 20, pending-compaction-bytes-delays: 32, pending-compaction-bytes-stops: 0, total-delays: 49, total-stops: 20, interval: 68 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2129990 Average: 11.1569 StdDev: 71.26 Min: 0 Median: 1.5559 Max: 25357 Percentiles: P50: 1.56 P75: 2.00 P99: 215.10 P99.9: 574.97 P99.99: 854.79 ------------------------------------------------------ [ 0, 1 ] 399375 18.750% 18.750% #### ( 1, 2 ] 1197363 56.214% 74.965% ########### ( 2, 3 ] 253510 11.902% 86.867% ## ( 3, 4 ] 108462 5.092% 91.959% # ( 4, 6 ] 64945 3.049% 95.008% # ( 6, 10 ] 13833 0.649% 95.657% ( 10, 15 ] 2961 0.139% 95.796% ( 15, 22 ] 403 0.019% 95.815% ( 22, 34 ] 40 0.002% 95.817% ( 34, 51 ] 42 0.002% 95.819% ( 51, 76 ] 47 0.002% 95.821% ( 76, 110 ] 12614 0.592% 96.413% ( 110, 170 ] 45195 2.122% 98.535% ( 170, 250 ] 17561 0.824% 99.360% ( 250, 380 ] 6550 0.308% 99.667% ( 380, 580 ] 5088 0.239% 99.906% ( 580, 870 ] 1888 0.089% 99.995% ( 870, 1300 ] 637 0.030% 100.025% ( 1300, 1900 ] 270 0.013% 100.037% ( 1900, 2900 ] 255 0.012% 100.049% ( 2900, 4400 ] 212 0.010% 100.059% ( 4400, 6600 ] 24 0.001% 100.060% ( 6600, 9900 ] 1 0.000% 100.060% ( 22000, 33000 ] 1 0.000% 100.060% ** Level 1 read latency histogram (micros): Count: 582116 Average: 8.3090 StdDev: 96.83 Min: 1 Median: 1.9659 Max: 30845 Percentiles: P50: 1.97 P75: 3.10 P99: 115.10 P99.9: 826.08 P99.99: 1379.14 ------------------------------------------------------ [ 0, 1 ] 17653 3.033% 3.033% # ( 1, 2 ] 283046 48.624% 51.656% ########## ( 2, 3 ] 129440 22.236% 73.892% #### ( 3, 4 ] 63015 10.825% 84.717% ## ( 4, 6 ] 63619 10.929% 95.646% ## ( 6, 10 ] 14828 2.547% 98.194% # ( 10, 15 ] 2768 0.476% 98.669% ( 15, 22 ] 645 0.111% 98.780% ( 22, 34 ] 69 0.012% 98.792% ( 34, 51 ] 83 0.014% 98.806% ( 51, 76 ] 41 0.007% 98.813% ( 76, 110 ] 861 0.148% 98.961% ( 110, 170 ] 2667 0.458% 99.419% ( 170, 250 ] 1027 0.176% 99.596% ( 250, 380 ] 670 0.115% 99.711% ( 380, 580 ] 653 0.112% 99.823% ( 580, 870 ] 529 0.091% 99.914% ( 870, 1300 ] 412 0.071% 99.985% ( 1300, 1900 ] 241 0.041% 100.026% ( 1900, 2900 ] 143 0.025% 100.051% ( 2900, 4400 ] 83 0.014% 100.065% ( 4400, 6600 ] 6 0.001% 100.066% ( 6600, 9900 ] 3 0.001% 100.066% ( 9900, 14000 ] 1 0.000% 100.066% ( 14000, 22000 ] 2 0.000% 100.067% ( 22000, 33000 ] 1 0.000% 100.067% ** Level 2 read latency histogram (micros): Count: 2253238 Average: 12.1166 StdDev: 161.16 Min: 0 Median: 1.9938 Max: 24055 Percentiles: P50: 1.99 P75: 3.08 P99: 9.89 P99.9: 665.82 P99.99: 963.42 ------------------------------------------------------ [ 0, 1 ] 37220 1.652% 1.652% ( 1, 2 ] 1096232 48.651% 50.303% ########## ( 2, 3 ] 535059 23.746% 74.049% ##### ( 3, 4 ] 258622 11.478% 85.527% ## ( 4, 6 ] 243885 10.824% 96.351% ## ( 6, 10 ] 61339 2.722% 99.073% # ( 10, 15 ] 12053 0.535% 99.608% ( 15, 22 ] 1613 0.072% 99.680% ( 22, 34 ] 174 0.008% 99.688% ( 34, 51 ] 228 0.010% 99.698% ( 51, 76 ] 159 0.007% 99.705% ( 76, 110 ] 163 0.007% 99.712% ( 110, 170 ] 431 0.019% 99.731% ( 170, 250 ] 619 0.027% 99.759% ( 250, 380 ] 914 0.041% 99.799% ( 380, 580 ] 1626 0.072% 99.871% ( 580, 870 ] 2189 0.097% 99.968% ( 870, 1300 ] 2240 0.099% 100.068% ( 1300, 1900 ] 1632 0.072% 100.140% ( 1900, 2900 ] 1729 0.077% 100.217% ( 2900, 4400 ] 1669 0.074% 100.291% ( 4400, 6600 ] 409 0.018% 100.309% ( 6600, 9900 ] 65 0.003% 100.312% ( 9900, 14000 ] 13 0.001% 100.313% ( 14000, 22000 ] 14 0.001% 100.313% ( 22000, 33000 ] 1 0.000% 100.313% ** Level 3 read latency histogram (micros): Count: 1160312 Average: 16.5552 StdDev: 162.58 Min: 1 Median: 1.9261 Max: 35315 Percentiles: P50: 1.93 P75: 2.93 P99: 236.57 P99.9: 1508.88 P99.99: 2328.69 ------------------------------------------------------ [ 0, 1 ] 23315 2.009% 2.009% ( 1, 2 ] 601305 51.823% 53.832% ########## ( 2, 3 ] 264737 22.816% 76.648% ##### ( 3, 4 ] 112232 9.673% 86.321% ## ( 4, 6 ] 95461 8.227% 94.548% ## ( 6, 10 ] 26455 2.280% 96.828% ( 10, 15 ] 4253 0.367% 97.194% ( 15, 22 ] 539 0.046% 97.241% ( 22, 34 ] 64 0.006% 97.246% ( 34, 51 ] 139 0.012% 97.258% ( 51, 76 ] 122 0.011% 97.269% ( 76, 110 ] 3735 0.322% 97.591% ( 110, 170 ] 12233 1.054% 98.645% ( 170, 250 ] 4950 0.427% 99.072% ( 250, 380 ] 2694 0.232% 99.304% ( 380, 580 ] 2744 0.236% 99.540% ( 580, 870 ] 2221 0.191% 99.732% ( 870, 1300 ] 1615 0.139% 99.871% ( 1300, 1900 ] 970 0.084% 99.954% ( 1900, 2900 ] 961 0.083% 100.037% ( 2900, 4400 ] 765 0.066% 100.103% ( 4400, 6600 ] 128 0.011% 100.114% ( 6600, 9900 ] 22 0.002% 100.116% ( 9900, 14000 ] 6 0.001% 100.117% ( 14000, 22000 ] 5 0.000% 100.117% ( 22000, 33000 ] 2 0.000% 100.117% ( 33000, 50000 ] 1 0.000% 100.117% ** Level 4 read latency histogram (micros): Count: 43802 Average: 287.3767 StdDev: 527.67 Min: 1 Median: 154.7494 Max: 23325 Percentiles: P50: 154.75 P75: 237.95 P99: 3102.62 P99.9: 6136.55 P99.99: 9542.97 ------------------------------------------------------ [ 0, 1 ] 35 0.080% 0.080% ( 1, 2 ] 212 0.484% 0.564% ( 2, 3 ] 553 1.262% 1.826% ( 3, 4 ] 367 0.838% 2.664% ( 4, 6 ] 397 0.906% 3.571% ( 6, 10 ] 51 0.116% 3.687% ( 10, 15 ] 15 0.034% 3.721% ( 15, 22 ] 1 0.002% 3.724% ( 51, 76 ] 5 0.011% 3.735% ( 76, 110 ] 6201 14.157% 17.892% ### ( 110, 170 ] 18857 43.051% 60.942% ######### ( 170, 250 ] 7249 16.549% 77.492% ### ( 250, 380 ] 3246 7.411% 84.903% # ( 380, 580 ] 2866 6.543% 91.446% # ( 580, 870 ] 1594 3.639% 95.085% # ( 870, 1300 ] 853 1.947% 97.032% ( 1300, 1900 ] 476 1.087% 98.119% ( 1900, 2900 ] 336 0.767% 98.886% ( 2900, 4400 ] 370 0.845% 99.731% ( 4400, 6600 ] 94 0.215% 99.945% ( 6600, 9900 ] 22 0.050% 99.995% ( 22000, 33000 ] 2 0.005% 100.000% ** Level 5 read latency histogram (micros): Count: 333874 Average: 316.5666 StdDev: 580.52 Min: 72 Median: 159.2629 Max: 45180 Percentiles: P50: 159.26 P75: 253.92 P99: 3330.83 P99.9: 6186.26 P99.99: 9795.84 ------------------------------------------------------ ( 51, 76 ] 31 0.009% 0.009% ( 76, 110 ] 47326 14.175% 14.184% ### ( 110, 170 ] 145643 43.622% 57.806% ######### ( 170, 250 ] 56602 16.953% 74.759% ### ( 250, 380 ] 26651 7.982% 82.742% ## ( 380, 580 ] 24411 7.311% 90.053% # ( 580, 870 ] 13769 4.124% 94.177% # ( 870, 1300 ] 7727 2.314% 96.491% ( 1300, 1900 ] 4223 1.265% 97.756% ( 1900, 2900 ] 3230 0.967% 98.724% ( 2900, 4400 ] 3211 0.962% 99.686% ( 4400, 6600 ] 882 0.264% 99.950% ( 6600, 9900 ] 139 0.042% 99.991% ( 9900, 14000 ] 16 0.005% 99.996% ( 14000, 22000 ] 13 0.004% 100.000% ( 22000, 33000 ] 6 0.002% 100.002% ( 33000, 50000 ] 4 0.001% 100.003% ** DB Stats ** Uptime(secs): 39.0 total, 9.0 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 151.59 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:16.820 H:M:S, 43.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 109.01 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.311 H:M:S, 58.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 277.819 micros/op 14383 ops/sec 60.208 seconds 865996 operations; 13.9 MB/s (217999 of 217999 found) Microseconds per read: Count: 865996 Average: 277.8209 StdDev: 739.83 Min: 2 Median: 185.5336 Max: 54145 Percentiles: P50: 185.53 P75: 245.32 P99: 1824.38 P99.9: 11733.92 P99.99: 29862.63 ------------------------------------------------------ ( 1, 2 ] 8 0.001% 0.001% ( 2, 3 ] 37 0.004% 0.005% ( 3, 4 ] 49 0.006% 0.011% ( 4, 6 ] 93 0.011% 0.022% ( 6, 10 ] 292 0.034% 0.055% ( 10, 15 ] 2926 0.338% 0.393% ( 15, 22 ] 4270 0.493% 0.886% ( 22, 34 ] 8975 1.036% 1.923% ( 34, 51 ] 2768 0.320% 2.242% ( 51, 76 ] 396 0.046% 2.288% ( 76, 110 ] 13648 1.576% 3.864% ( 110, 170 ] 343284 39.640% 43.504% ######## ( 170, 250 ] 289705 33.453% 76.958% ####### ( 250, 380 ] 104017 12.011% 88.969% ## ( 380, 580 ] 49537 5.720% 94.689% # ( 580, 870 ] 22691 2.620% 97.309% # ( 870, 1300 ] 10535 1.217% 98.526% ( 1300, 1900 ] 4697 0.542% 99.068% ( 1900, 2900 ] 3205 0.370% 99.438% ( 2900, 4400 ] 2819 0.326% 99.764% ( 4400, 6600 ] 693 0.080% 99.844% ( 6600, 9900 ] 345 0.040% 99.884% ( 9900, 14000 ] 313 0.036% 99.920% ( 14000, 22000 ] 442 0.051% 99.971% ( 22000, 33000 ] 230 0.027% 99.998% ( 33000, 50000 ] 20 0.002% 100.000% ( 50000, 75000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 16174111 rocksdb.bloom.filter.full.positive COUNT : 1023446 rocksdb.bloom.filter.full.true.positive COUNT : 865557 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 439 rocksdb.memtable.miss COUNT : 865557 rocksdb.l0.hit COUNT : 30075 rocksdb.l1.hit COUNT : 7043 rocksdb.l2andup.hit COUNT : 828439 rocksdb.compaction.key.drop.new COUNT : 124088 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 6147113 rocksdb.number.keys.read COUNT : 865996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 6343820616 rocksdb.bytes.read COUNT : 865996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1706 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 37240138 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 27724734926 rocksdb.compact.write.bytes COUNT : 27591931853 rocksdb.flush.write.bytes COUNT : 6341863443 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 26 rocksdb.number.superversion_acquires COUNT : 1369 rocksdb.number.superversion_releases COUNT : 1328 rocksdb.number.superversion_cleanups COUNT : 341 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 335 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 6290333696 rocksdb.memtable.garbage.bytes.at.flush COUNT : 2094080 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 31163675715 rocksdb.non.last.level.read.count COUNT : 7392674 rocksdb.block.checksum.compute.count COUNT : 7749464 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 184.197145 P95 : 612.810959 P99 : 1823.804482 P100 : 54144.000000 COUNT : 865996 SUM : 239815859 rocksdb.db.write.micros P50 : 1.527103 P95 : 4.383053 P99 : 9.272215 P100 : 483188.000000 COUNT : 6147113 SUM : 52437065 rocksdb.compaction.times.micros P50 : 1294230.769231 P95 : 4072500.000000 P99 : 8166500.000000 P100 : 9414984.000000 COUNT : 99 SUM : 175806323 rocksdb.compaction.times.cpu_micros P50 : 998684.210526 P95 : 2412500.000000 P99 : 5483572.000000 P100 : 5483572.000000 COUNT : 99 SUM : 111512280 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 795.731707 P95 : 9363.750000 P99 : 14560.000000 P100 : 15080.000000 COUNT : 93 SUM : 174948 rocksdb.compaction.outfile.sync.micros P50 : 10127.777778 P95 : 190347.826087 P99 : 243634.782609 P100 : 265916.000000 COUNT : 383 SUM : 14485282 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 185.555556 P95 : 2663.888889 P99 : 4803.333333 P100 : 5645.000000 COUNT : 245 SUM : 158995 rocksdb.table.open.io.micros P50 : 7794.198895 P95 : 20367.527675 P99 : 26744.666667 P100 : 31032.000000 COUNT : 1706 SUM : 14243401 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.962372 P95 : 9.901241 P99 : 166.446958 P100 : 35319.000000 COUNT : 6714100 SUM : 97216726 rocksdb.read.block.get.micros P50 : 141.947673 P95 : 514.941326 P99 : 1691.573548 P100 : 45186.000000 COUNT : 1033817 SUM : 206503550 rocksdb.write.raw.block.micros P50 : 0.530402 P95 : 1.202226 P99 : 3.996072 P100 : 50289.000000 COUNT : 8220229 SUM : 21369919 rocksdb.numfiles.in.singlecompaction P50 : 3.076923 P95 : 4.900000 P99 : 11.700000 P100 : 24.000000 COUNT : 166 SUM : 610 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.557381 P95 : 1165.356236 P99 : 2550.892587 P100 : 482983.000000 COUNT : 256984 SUM : 37258846 rocksdb.sst.read.micros P50 : 1.970537 P95 : 162.409772 P99 : 510.058851 P100 : 45180.000000 COUNT : 7392881 SUM : 280128762 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 865996 SUM : 865996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 6147113 SUM : 6343820616 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 219600.000000 P95 : 525214.285714 P99 : 770100.000000 P100 : 791869.000000 COUNT : 93 SUM : 22962976 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 375 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0