[2024-11-16 22:39:01.588357] Starting SPDK v25.01-pre git sha1 83e8405e4 / DPDK 23.11.0 initialization... [2024-11-16 22:39:01.588620] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid443934 ] Set seed to 1731793141581654 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-16 22:39:01.712493] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-16 22:39:01.733588] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-16 22:39:01.733629] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 443934' to capture a snapshot of events at runtime. [2024-11-16 22:39:01.733639] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-16 22:39:01.733650] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-16 22:39:01.733658] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid443934 for offline analysis/debug. [2024-11-16 22:39:01.734207] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Sat Nov 16 22:39:06 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/16-22:39:13 ... thread 0: (1000000,1000000) ops and (126869.7,126869.7) ops/second in (7.882101,7.882101) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/10 920.03 MB 12.6 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 116.4 7.04 2.53 9 0.782 0 0 0.0 0.0 L2 1/0 128.78 MB 1.3 0.4 0.4 0.1 0.4 0.4 0.0 1.1 107.9 107.3 4.03 2.05 1 4.034 442K 2455 0.0 0.0 L3 29/8 2.65 GB 1.7 0.6 0.5 0.1 0.6 0.5 0.4 1.2 92.7 92.3 6.63 2.97 4 1.657 555K 235 0.0 0.0 L4 194/25 12.71 GB 1.2 5.2 2.3 2.9 5.1 2.2 0.3 2.2 107.2 104.9 49.71 24.49 22 2.260 1265K 0 0.0 0.0 L5 1096/0 90.12 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1331/43 106.50 GB 0.0 6.2 3.2 3.0 6.9 3.9 0.8 8.6 94.7 105.0 67.41 32.04 36 1.872 2262K 2690 0.0 0.0 Int 0/0 0.00 KB 0.0 6.2 3.2 3.0 6.9 3.9 0.8 8.6 94.7 105.0 67.41 32.04 36 1.872 2262K 2690 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.2 3.2 3.0 6.1 3.1 0.0 0.0 105.7 103.7 60.37 29.51 27 2.236 2262K 2690 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 116.4 7.04 2.53 9 0.782 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.1 total, 7.9 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.91 GB write, 875.73 MB/s write, 6.23 GB read, 789.26 MB/s read, 67.4 seconds Interval compaction: 6.91 GB write, 897.99 MB/s write, 6.23 GB read, 809.33 MB/s read, 67.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 23, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 23, interval: 23 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 177551 Average: 12.9895 StdDev: 142.41 Min: 1 Median: 1.8943 Max: 7577 Percentiles: P50: 1.89 P75: 2.90 P99: 13.49 P99.9: 2003.46 P99.99: 4016.85 ------------------------------------------------------ [ 0, 1 ] 6442 3.628% 3.628% # ( 1, 2 ] 92066 51.853% 55.482% ########## ( 2, 3 ] 38347 21.598% 77.079% #### ( 3, 4 ] 16763 9.441% 86.520% ## ( 4, 6 ] 17703 9.971% 96.491% ## ( 6, 10 ] 3990 2.247% 98.738% ( 10, 15 ] 666 0.375% 99.113% ( 15, 22 ] 60 0.034% 99.147% ( 22, 34 ] 4 0.002% 99.150% ( 34, 51 ] 2 0.001% 99.151% ( 51, 76 ] 9 0.005% 99.156% ( 76, 110 ] 1 0.001% 99.156% ( 110, 170 ] 3 0.002% 99.158% ( 170, 250 ] 46 0.026% 99.184% ( 250, 380 ] 111 0.063% 99.246% ( 380, 580 ] 247 0.139% 99.386% ( 580, 870 ] 337 0.190% 99.575% ( 870, 1300 ] 356 0.201% 99.776% ( 1300, 1900 ] 210 0.118% 99.894% ( 1900, 2900 ] 101 0.057% 99.951% ( 2900, 4400 ] 93 0.052% 100.003% ( 4400, 6600 ] 21 0.012% 100.015% ( 6600, 9900 ] 2 0.001% 100.016% ** Level 1 read latency histogram (micros): Count: 96952 Average: 12.9554 StdDev: 150.62 Min: 1 Median: 1.9775 Max: 7778 Percentiles: P50: 1.98 P75: 3.07 P99: 13.76 P99.9: 2745.81 P99.99: 5404.41 ------------------------------------------------------ [ 0, 1 ] 2828 2.917% 2.917% # ( 1, 2 ] 46699 48.167% 51.084% ########## ( 2, 3 ] 22561 23.270% 74.354% ##### ( 3, 4 ] 9531 9.831% 84.185% ## ( 4, 6 ] 10506 10.836% 95.021% ## ( 6, 10 ] 3365 3.471% 98.492% # ( 10, 15 ] 655 0.676% 99.168% ( 15, 22 ] 104 0.107% 99.275% ( 22, 34 ] 21 0.022% 99.297% ( 34, 51 ] 8 0.008% 99.305% ( 51, 76 ] 5 0.005% 99.310% ( 76, 110 ] 4 0.004% 99.314% ( 110, 170 ] 5 0.005% 99.319% ( 170, 250 ] 20 0.021% 99.340% ( 250, 380 ] 40 0.041% 99.381% ( 380, 580 ] 80 0.083% 99.464% ( 580, 870 ] 109 0.112% 99.576% ( 870, 1300 ] 137 0.141% 99.717% ( 1300, 1900 ] 106 0.109% 99.827% ( 1900, 2900 ] 84 0.087% 99.913% ( 2900, 4400 ] 67 0.069% 99.982% ( 4400, 6600 ] 16 0.017% 99.999% ( 6600, 9900 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 254024 Average: 11.7687 StdDev: 151.51 Min: 1 Median: 2.0495 Max: 9654 Percentiles: P50: 2.05 P75: 3.17 P99: 11.74 P99.9: 1796.70 P99.99: 3014.59 ------------------------------------------------------ [ 0, 1 ] 5639 2.220% 2.220% ( 1, 2 ] 118309 46.574% 48.794% ######### ( 2, 3 ] 61933 24.381% 73.175% ##### ( 3, 4 ] 27263 10.732% 83.907% ## ( 4, 6 ] 29886 11.765% 95.672% ## ( 6, 10 ] 7942 3.126% 98.799% # ( 10, 15 ] 1472 0.579% 99.378% ( 15, 22 ] 215 0.085% 99.463% ( 22, 34 ] 41 0.016% 99.479% ( 34, 51 ] 33 0.013% 99.492% ( 51, 76 ] 21 0.008% 99.500% ( 76, 110 ] 18 0.007% 99.507% ( 110, 170 ] 30 0.012% 99.519% ( 170, 250 ] 51 0.020% 99.539% ( 250, 380 ] 119 0.047% 99.586% ( 380, 580 ] 170 0.067% 99.653% ( 580, 870 ] 230 0.091% 99.743% ( 870, 1300 ] 244 0.096% 99.839% ( 1300, 1900 ] 186 0.073% 99.913% ( 1900, 2900 ] 183 0.072% 99.985% ( 2900, 4400 ] 178 0.070% 100.055% ( 4400, 6600 ] 46 0.018% 100.073% ( 6600, 9900 ] 8 0.003% 100.076% ** Level 3 read latency histogram (micros): Count: 467605 Average: 10.3320 StdDev: 133.36 Min: 1 Median: 1.9759 Max: 11045 Percentiles: P50: 1.98 P75: 3.03 P99: 9.97 P99.9: 1179.70 P99.99: 1838.77 ------------------------------------------------------ [ 0, 1 ] 10530 2.252% 2.252% ( 1, 2 ] 228789 48.928% 51.180% ########## ( 2, 3 ] 109962 23.516% 74.696% ##### ( 3, 4 ] 46275 9.896% 84.592% ## ( 4, 6 ] 52925 11.318% 95.910% ## ( 6, 10 ] 14573 3.117% 99.027% # ( 10, 15 ] 2297 0.491% 99.518% ( 15, 22 ] 383 0.082% 99.600% ( 22, 34 ] 43 0.009% 99.609% ( 34, 51 ] 31 0.007% 99.616% ( 51, 76 ] 18 0.004% 99.620% ( 76, 110 ] 11 0.002% 99.622% ( 110, 170 ] 21 0.004% 99.626% ( 170, 250 ] 52 0.011% 99.638% ( 250, 380 ] 144 0.031% 99.668% ( 380, 580 ] 281 0.060% 99.728% ( 580, 870 ] 465 0.099% 99.828% ( 870, 1300 ] 474 0.101% 99.929% ( 1300, 1900 ] 321 0.069% 99.998% ( 1900, 2900 ] 268 0.057% 100.055% ( 2900, 4400 ] 250 0.053% 100.109% ( 4400, 6600 ] 62 0.013% 100.122% ( 6600, 9900 ] 5 0.001% 100.123% ( 9900, 14000 ] 1 0.000% 100.123% ** Level 4 read latency histogram (micros): Count: 834773 Average: 15.1222 StdDev: 167.46 Min: 1 Median: 1.9927 Max: 8318 Percentiles: P50: 1.99 P75: 3.07 P99: 12.72 P99.9: 1393.37 P99.99: 1793.71 ------------------------------------------------------ [ 0, 1 ] 18050 2.162% 2.162% ( 1, 2 ] 402281 48.190% 50.353% ########## ( 2, 3 ] 199563 23.906% 74.259% ##### ( 3, 4 ] 84912 10.172% 84.431% ## ( 4, 6 ] 93653 11.219% 95.650% ## ( 6, 10 ] 25722 3.081% 98.731% # ( 10, 15 ] 4131 0.495% 99.226% ( 15, 22 ] 648 0.078% 99.304% ( 22, 34 ] 73 0.009% 99.312% ( 34, 51 ] 51 0.006% 99.318% ( 51, 76 ] 82 0.010% 99.328% ( 76, 110 ] 35 0.004% 99.333% ( 110, 170 ] 77 0.009% 99.342% ( 170, 250 ] 229 0.027% 99.369% ( 250, 380 ] 392 0.047% 99.416% ( 380, 580 ] 875 0.105% 99.521% ( 580, 870 ] 1439 0.172% 99.693% ( 870, 1300 ] 1550 0.186% 99.879% ( 1300, 1900 ] 1126 0.135% 100.014% ( 1900, 2900 ] 786 0.094% 100.108% ( 2900, 4400 ] 619 0.074% 100.182% ( 4400, 6600 ] 182 0.022% 100.204% ( 6600, 9900 ] 21 0.003% 100.207% ** Level 5 read latency histogram (micros): Count: 14958 Average: 1546.1293 StdDev: 1361.26 Min: 92 Median: 1061.1747 Max: 12778 Percentiles: P50: 1061.17 P75: 1996.70 P99: 6414.86 P99.9: 9382.46 P99.99: 9881.62 ------------------------------------------------------ ( 76, 110 ] 7 0.047% 0.047% ( 110, 170 ] 98 0.655% 0.702% ( 170, 250 ] 314 2.099% 2.801% ( 250, 380 ] 883 5.903% 8.704% # ( 380, 580 ] 1948 13.023% 21.728% ### ( 580, 870 ] 2893 19.341% 41.068% #### ( 870, 1300 ] 3005 20.090% 61.158% #### ( 1300, 1900 ] 1924 12.863% 74.021% ### ( 1900, 2900 ] 1515 10.128% 84.149% ## ( 2900, 4400 ] 1573 10.516% 94.665% ## ( 4400, 6600 ] 708 4.733% 99.398% # ( 6600, 9900 ] 89 0.595% 99.993% ( 9900, 14000 ] 1 0.007% 100.000% ** DB Stats ** Uptime(secs): 8.1 total, 7.9 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 121.74 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.944 H:M:S, 48.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 124.82 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.944 H:M:S, 50.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/11/16-22:39:18 ... thread 0: (1000000,2000000) ops and (197743.9,154569.7) ops/second in (5.057047,12.939148) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/4 1.48 GB 119.6 1.0 0.0 1.0 2.8 1.8 0.0 1.5 56.6 162.6 17.81 9.61 26 0.685 1023K 1282 0.0 0.0 L1 1/0 31.37 MB 3.1 0.5 0.5 0.0 0.5 0.5 0.0 1.0 75.8 75.6 6.32 2.27 1 6.320 486K 960 0.0 0.0 L2 1/1 128.77 MB 0.0 0.4 0.4 0.1 0.4 0.4 0.4 1.1 107.9 107.3 4.03 2.05 1 4.034 442K 2455 0.0 0.0 L3 18/2 1.06 GB 0.8 1.5 0.9 0.5 1.5 0.9 0.4 1.6 126.1 125.5 12.09 7.21 8 1.511 1069K 235 0.0 0.0 L4 166/0 9.73 GB 1.0 9.1 3.9 5.2 8.9 3.7 0.7 2.3 116.9 114.5 79.39 42.68 35 2.268 2590K 0 0.0 0.0 L5 1156/0 95.04 GB 1.0 0.0 0.0 0.0 0.0 0.0 4.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1352/7 107.46 GB 0.0 12.4 5.7 6.7 14.1 7.3 6.5 7.6 106.4 120.5 119.65 63.82 71 1.685 5611K 4932 0.0 0.0 Int 0/0 0.00 KB 0.0 12.4 5.7 6.7 14.1 7.3 6.5 7.6 106.4 120.5 119.65 63.82 71 1.685 5611K 4932 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.4 5.7 6.7 12.2 5.5 0.0 0.0 118.6 116.6 107.35 58.25 48 2.236 5611K 4932 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 153.7 12.30 5.57 23 0.535 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 13.1 total, 12.9 interval Flush(GB): cumulative 1.845, interval 1.845 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.07 GB write, 1096.68 MB/s write, 12.43 GB read, 968.76 MB/s read, 119.6 seconds Interval compaction: 14.07 GB write, 1113.67 MB/s write, 12.43 GB read, 983.76 MB/s read, 119.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 35, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 35, interval: 35 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 430302 Average: 15.1867 StdDev: 156.87 Min: 1 Median: 1.8115 Max: 16486 Percentiles: P50: 1.81 P75: 2.74 P99: 284.74 P99.9: 2316.77 P99.99: 3974.93 ------------------------------------------------------ [ 0, 1 ] 24190 5.622% 5.622% # ( 1, 2 ] 235313 54.686% 60.307% ########### ( 2, 3 ] 85247 19.811% 80.118% #### ( 3, 4 ] 35832 8.327% 88.445% ## ( 4, 6 ] 36278 8.431% 96.876% ## ( 6, 10 ] 7240 1.683% 98.559% ( 10, 15 ] 1171 0.272% 98.831% ( 15, 22 ] 122 0.028% 98.859% ( 22, 34 ] 9 0.002% 98.861% ( 34, 51 ] 11 0.003% 98.864% ( 51, 76 ] 14 0.003% 98.867% ( 76, 110 ] 30 0.007% 98.874% ( 110, 170 ] 154 0.036% 98.910% ( 170, 250 ] 277 0.064% 98.974% ( 250, 380 ] 419 0.097% 99.072% ( 380, 580 ] 760 0.177% 99.248% ( 580, 870 ] 1113 0.259% 99.507% ( 870, 1300 ] 1030 0.239% 99.746% ( 1300, 1900 ] 536 0.125% 99.871% ( 1900, 2900 ] 304 0.071% 99.941% ( 2900, 4400 ] 293 0.068% 100.010% ( 4400, 6600 ] 58 0.013% 100.023% ( 6600, 9900 ] 3 0.001% 100.024% ( 14000, 22000 ] 1 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 203287 Average: 9.1211 StdDev: 111.97 Min: 1 Median: 1.9084 Max: 7778 Percentiles: P50: 1.91 P75: 2.86 P99: 10.84 P99.9: 1594.06 P99.99: 3359.55 ------------------------------------------------------ [ 0, 1 ] 5263 2.589% 2.589% # ( 1, 2 ] 106095 52.190% 54.779% ########## ( 2, 3 ] 47543 23.387% 78.166% ##### ( 3, 4 ] 18175 8.941% 87.106% ## ( 4, 6 ] 18382 9.042% 96.149% ## ( 6, 10 ] 5655 2.782% 98.931% # ( 10, 15 ] 844 0.415% 99.346% ( 15, 22 ] 125 0.061% 99.407% ( 22, 34 ] 26 0.013% 99.420% ( 34, 51 ] 29 0.014% 99.434% ( 51, 76 ] 11 0.005% 99.440% ( 76, 110 ] 14 0.007% 99.447% ( 110, 170 ] 56 0.028% 99.474% ( 170, 250 ] 97 0.048% 99.522% ( 250, 380 ] 120 0.059% 99.581% ( 380, 580 ] 199 0.098% 99.679% ( 580, 870 ] 216 0.106% 99.785% ( 870, 1300 ] 170 0.084% 99.869% ( 1300, 1900 ] 130 0.064% 99.933% ( 1900, 2900 ] 94 0.046% 99.979% ( 2900, 4400 ] 74 0.036% 100.015% ( 4400, 6600 ] 17 0.008% 100.024% ( 6600, 9900 ] 1 0.000% 100.024% ** Level 2 read latency histogram (micros): Count: 286749 Average: 11.4232 StdDev: 147.61 Min: 1 Median: 2.0556 Max: 9654 Percentiles: P50: 2.06 P75: 3.18 P99: 11.60 P99.9: 1787.68 P99.99: 3094.08 ------------------------------------------------------ [ 0, 1 ] 6013 2.097% 2.097% ( 1, 2 ] 133465 46.544% 48.641% ######### ( 2, 3 ] 70047 24.428% 73.069% ##### ( 3, 4 ] 30736 10.719% 83.788% ## ( 4, 6 ] 34043 11.872% 95.660% ## ( 6, 10 ] 9060 3.160% 98.820% # ( 10, 15 ] 1619 0.565% 99.384% ( 15, 22 ] 243 0.085% 99.469% ( 22, 34 ] 44 0.015% 99.484% ( 34, 51 ] 36 0.013% 99.497% ( 51, 76 ] 31 0.011% 99.508% ( 76, 110 ] 18 0.006% 99.514% ( 110, 170 ] 33 0.012% 99.525% ( 170, 250 ] 55 0.019% 99.545% ( 250, 380 ] 133 0.046% 99.591% ( 380, 580 ] 190 0.066% 99.657% ( 580, 870 ] 257 0.090% 99.747% ( 870, 1300 ] 271 0.095% 99.841% ( 1300, 1900 ] 207 0.072% 99.914% ( 1900, 2900 ] 195 0.068% 99.982% ( 2900, 4400 ] 188 0.066% 100.047% ( 4400, 6600 ] 50 0.017% 100.065% ( 6600, 9900 ] 8 0.003% 100.067% ** Level 3 read latency histogram (micros): Count: 931841 Average: 8.5822 StdDev: 114.95 Min: 1 Median: 1.9562 Max: 11335 Percentiles: P50: 1.96 P75: 2.96 P99: 9.72 P99.9: 837.12 P99.99: 1293.66 ------------------------------------------------------ [ 0, 1 ] 19484 2.091% 2.091% ( 1, 2 ] 466870 50.102% 52.193% ########## ( 2, 3 ] 221252 23.744% 75.936% ##### ( 3, 4 ] 90983 9.764% 85.700% ## ( 4, 6 ] 98387 10.558% 96.258% ## ( 6, 10 ] 27448 2.946% 99.204% # ( 10, 15 ] 3862 0.414% 99.618% ( 15, 22 ] 558 0.060% 99.678% ( 22, 34 ] 58 0.006% 99.685% ( 34, 51 ] 72 0.008% 99.692% ( 51, 76 ] 56 0.006% 99.698% ( 76, 110 ] 38 0.004% 99.702% ( 110, 170 ] 106 0.011% 99.714% ( 170, 250 ] 162 0.017% 99.731% ( 250, 380 ] 316 0.034% 99.765% ( 380, 580 ] 539 0.058% 99.823% ( 580, 870 ] 810 0.087% 99.910% ( 870, 1300 ] 758 0.081% 99.991% ( 1300, 1900 ] 468 0.050% 100.041% ( 1900, 2900 ] 388 0.042% 100.083% ( 2900, 4400 ] 347 0.037% 100.120% ( 4400, 6600 ] 89 0.010% 100.130% ( 6600, 9900 ] 10 0.001% 100.131% ( 9900, 14000 ] 3 0.000% 100.131% ** Level 4 read latency histogram (micros): Count: 1243628 Average: 14.6645 StdDev: 159.01 Min: 1 Median: 1.9861 Max: 14471 Percentiles: P50: 1.99 P75: 3.03 P99: 14.04 P99.9: 1270.78 P99.99: 1683.27 ------------------------------------------------------ [ 0, 1 ] 25434 2.045% 2.045% ( 1, 2 ] 604768 48.629% 50.674% ########## ( 2, 3 ] 298575 24.008% 74.683% ##### ( 3, 4 ] 124574 10.017% 84.700% ## ( 4, 6 ] 135967 10.933% 95.633% ## ( 6, 10 ] 37279 2.998% 98.631% # ( 10, 15 ] 5684 0.457% 99.088% ( 15, 22 ] 825 0.066% 99.154% ( 22, 34 ] 88 0.007% 99.161% ( 34, 51 ] 104 0.008% 99.169% ( 51, 76 ] 119 0.010% 99.179% ( 76, 110 ] 176 0.014% 99.193% ( 110, 170 ] 679 0.055% 99.248% ( 170, 250 ] 804 0.065% 99.312% ( 250, 380 ] 957 0.077% 99.389% ( 380, 580 ] 1682 0.135% 99.525% ( 580, 870 ] 2521 0.203% 99.727% ( 870, 1300 ] 2305 0.185% 99.913% ( 1300, 1900 ] 1507 0.121% 100.034% ( 1900, 2900 ] 1004 0.081% 100.115% ( 2900, 4400 ] 823 0.066% 100.181% ( 4400, 6600 ] 219 0.018% 100.198% ( 6600, 9900 ] 27 0.002% 100.200% ( 9900, 14000 ] 2 0.000% 100.201% ( 14000, 22000 ] 1 0.000% 100.201% ** Level 5 read latency histogram (micros): Count: 37092 Average: 1023.8278 StdDev: 1139.50 Min: 73 Median: 658.3354 Max: 17013 Percentiles: P50: 658.34 P75: 1220.58 P99: 5873.14 P99.9: 8832.22 P99.99: 14581.60 ------------------------------------------------------ ( 51, 76 ] 1 0.003% 0.003% ( 76, 110 ] 781 2.106% 2.108% ( 110, 170 ] 3585 9.665% 11.773% ## ( 170, 250 ] 3162 8.525% 20.298% ## ( 250, 380 ] 3636 9.803% 30.101% ## ( 380, 580 ] 5609 15.122% 45.223% ### ( 580, 870 ] 6560 17.686% 62.908% #### ( 870, 1300 ] 5501 14.831% 77.739% ### ( 1300, 1900 ] 3010 8.115% 85.854% ## ( 1900, 2900 ] 2246 6.055% 91.909% # ( 2900, 4400 ] 2081 5.610% 97.520% # ( 4400, 6600 ] 820 2.211% 99.730% ( 6600, 9900 ] 93 0.251% 99.981% ( 9900, 14000 ] 3 0.008% 99.989% ( 14000, 22000 ] 4 0.011% 100.000% ** DB Stats ** Uptime(secs): 13.1 total, 5.1 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 149.78 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.299 H:M:S, 40.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 194.62 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.355 H:M:S, 26.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/16-22:39:22 ... thread 0: (1000000,3000000) ops and (286861.9,182646.8) ops/second in (3.485998,16.425146) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/4 2.40 GB 220.2 2.0 0.0 2.0 4.8 2.8 0.0 1.7 82.4 194.8 25.22 16.27 45 0.560 2110K 2387 0.0 0.0 L1 1/0 31.37 MB 3.1 0.5 0.5 0.0 0.5 0.5 0.0 1.0 75.8 75.6 6.32 2.27 1 6.320 486K 960 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.4 0.1 0.4 0.4 0.4 1.1 107.9 107.3 4.03 2.05 1 4.034 442K 2455 0.0 0.0 L3 19/0 976.28 MB 1.0 1.9 1.1 0.8 1.9 1.1 0.4 1.7 136.7 136.1 13.98 8.90 9 1.553 1462K 1484 0.0 0.0 L4 166/0 9.64 GB 1.0 9.5 4.1 5.4 9.3 3.9 0.7 2.3 119.2 116.6 81.90 44.93 39 2.100 2830K 0 0.0 0.0 L5 1160/0 95.35 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1359/4 108.37 GB 0.0 14.3 6.0 8.3 16.9 8.6 6.8 6.1 111.5 131.4 131.44 74.42 95 1.384 7331K 7286 0.0 0.0 Int 0/0 0.00 KB 0.0 14.3 6.0 8.3 16.9 8.6 6.8 6.1 111.5 131.4 131.44 74.42 95 1.384 7331K 7286 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.3 6.0 8.3 14.1 5.8 0.0 0.0 126.4 124.5 116.01 66.30 57 2.035 7331K 7286 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 183.7 15.43 8.12 38 0.406 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.6 total, 16.4 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 16.87 GB write, 1038.90 MB/s write, 14.32 GB read, 881.84 MB/s read, 131.4 seconds Interval compaction: 16.87 GB write, 1051.57 MB/s write, 14.32 GB read, 892.60 MB/s read, 131.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 35, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 35, interval: 35 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 697950 Average: 11.0821 StdDev: 126.25 Min: 0 Median: 1.6739 Max: 16486 Percentiles: P50: 1.67 P75: 2.42 P99: 113.49 P99.9: 1696.01 P99.99: 3548.86 ------------------------------------------------------ [ 0, 1 ] 87547 12.543% 12.543% ### ( 1, 2 ] 387956 55.585% 68.129% ########### ( 2, 3 ] 113440 16.253% 84.382% ### ( 3, 4 ] 46347 6.640% 91.022% # ( 4, 6 ] 44894 6.432% 97.455% # ( 6, 10 ] 8632 1.237% 98.691% ( 10, 15 ] 1584 0.227% 98.918% ( 15, 22 ] 172 0.025% 98.943% ( 22, 34 ] 11 0.002% 98.944% ( 34, 51 ] 20 0.003% 98.947% ( 51, 76 ] 20 0.003% 98.950% ( 76, 110 ] 276 0.040% 98.990% ( 110, 170 ] 1228 0.176% 99.166% ( 170, 250 ] 827 0.118% 99.284% ( 250, 380 ] 689 0.099% 99.383% ( 380, 580 ] 986 0.141% 99.524% ( 580, 870 ] 1205 0.173% 99.697% ( 870, 1300 ] 1059 0.152% 99.849% ( 1300, 1900 ] 544 0.078% 99.926% ( 1900, 2900 ] 313 0.045% 99.971% ( 2900, 4400 ] 301 0.043% 100.014% ( 4400, 6600 ] 59 0.008% 100.023% ( 6600, 9900 ] 3 0.000% 100.023% ( 9900, 14000 ] 1 0.000% 100.023% ( 14000, 22000 ] 1 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 210548 Average: 8.9074 StdDev: 110.05 Min: 1 Median: 1.8948 Max: 7778 Percentiles: P50: 1.89 P75: 2.84 P99: 10.57 P99.9: 1560.55 P99.99: 3344.84 ------------------------------------------------------ [ 0, 1 ] 5954 2.828% 2.828% # ( 1, 2 ] 111001 52.720% 55.548% ########### ( 2, 3 ] 48647 23.105% 78.653% ##### ( 3, 4 ] 18401 8.740% 87.392% ## ( 4, 6 ] 18545 8.808% 96.200% ## ( 6, 10 ] 5797 2.753% 98.954% # ( 10, 15 ] 850 0.404% 99.357% ( 15, 22 ] 126 0.060% 99.417% ( 22, 34 ] 26 0.012% 99.430% ( 34, 51 ] 29 0.014% 99.443% ( 51, 76 ] 11 0.005% 99.449% ( 76, 110 ] 17 0.008% 99.457% ( 110, 170 ] 67 0.032% 99.488% ( 170, 250 ] 98 0.047% 99.535% ( 250, 380 ] 126 0.060% 99.595% ( 380, 580 ] 200 0.095% 99.690% ( 580, 870 ] 216 0.103% 99.792% ( 870, 1300 ] 170 0.081% 99.873% ( 1300, 1900 ] 130 0.062% 99.935% ( 1900, 2900 ] 94 0.045% 99.980% ( 2900, 4400 ] 74 0.035% 100.015% ( 4400, 6600 ] 17 0.008% 100.023% ( 6600, 9900 ] 1 0.000% 100.023% ** Level 2 read latency histogram (micros): Count: 286762 Average: 11.4319 StdDev: 147.62 Min: 1 Median: 2.0557 Max: 9654 Percentiles: P50: 2.06 P75: 3.18 P99: 11.64 P99.9: 1787.65 P99.99: 3094.07 ------------------------------------------------------ [ 0, 1 ] 6013 2.097% 2.097% ( 1, 2 ] 133465 46.542% 48.639% ######### ( 2, 3 ] 70047 24.427% 73.066% ##### ( 3, 4 ] 30736 10.718% 83.784% ## ( 4, 6 ] 34043 11.872% 95.656% ## ( 6, 10 ] 9060 3.159% 98.815% # ( 10, 15 ] 1619 0.565% 99.380% ( 15, 22 ] 243 0.085% 99.464% ( 22, 34 ] 44 0.015% 99.480% ( 34, 51 ] 36 0.013% 99.492% ( 51, 76 ] 31 0.011% 99.503% ( 76, 110 ] 22 0.008% 99.511% ( 110, 170 ] 37 0.013% 99.524% ( 170, 250 ] 56 0.020% 99.543% ( 250, 380 ] 134 0.047% 99.590% ( 380, 580 ] 193 0.067% 99.657% ( 580, 870 ] 257 0.090% 99.747% ( 870, 1300 ] 271 0.095% 99.841% ( 1300, 1900 ] 207 0.072% 99.914% ( 1900, 2900 ] 195 0.068% 99.982% ( 2900, 4400 ] 188 0.066% 100.047% ( 4400, 6600 ] 50 0.017% 100.065% ( 6600, 9900 ] 8 0.003% 100.067% ** Level 3 read latency histogram (micros): Count: 992818 Average: 8.3116 StdDev: 111.52 Min: 1 Median: 1.9557 Max: 11335 Percentiles: P50: 1.96 P75: 2.96 P99: 9.71 P99.9: 801.44 P99.99: 1264.81 ------------------------------------------------------ [ 0, 1 ] 20624 2.077% 2.077% ( 1, 2 ] 497843 50.144% 52.222% ########## ( 2, 3 ] 235676 23.738% 75.960% ##### ( 3, 4 ] 97139 9.784% 85.744% ## ( 4, 6 ] 105197 10.596% 96.340% ## ( 6, 10 ] 28443 2.865% 99.205% # ( 10, 15 ] 4017 0.405% 99.609% ( 15, 22 ] 579 0.058% 99.668% ( 22, 34 ] 64 0.006% 99.674% ( 34, 51 ] 85 0.009% 99.683% ( 51, 76 ] 59 0.006% 99.689% ( 76, 110 ] 72 0.007% 99.696% ( 110, 170 ] 264 0.027% 99.722% ( 170, 250 ] 221 0.022% 99.745% ( 250, 380 ] 342 0.034% 99.779% ( 380, 580 ] 571 0.058% 99.837% ( 580, 870 ] 824 0.083% 99.920% ( 870, 1300 ] 761 0.077% 99.996% ( 1300, 1900 ] 468 0.047% 100.043% ( 1900, 2900 ] 390 0.039% 100.083% ( 2900, 4400 ] 347 0.035% 100.118% ( 4400, 6600 ] 89 0.009% 100.127% ( 6600, 9900 ] 10 0.001% 100.128% ( 9900, 14000 ] 3 0.000% 100.128% ** Level 4 read latency histogram (micros): Count: 1311261 Average: 15.3443 StdDev: 158.27 Min: 1 Median: 1.9897 Max: 15123 Percentiles: P50: 1.99 P75: 3.06 P99: 156.32 P99.9: 1256.47 P99.99: 1669.39 ------------------------------------------------------ [ 0, 1 ] 26619 2.030% 2.030% ( 1, 2 ] 635574 48.470% 50.500% ########## ( 2, 3 ] 312779 23.853% 74.354% ##### ( 3, 4 ] 130645 9.963% 84.317% ## ( 4, 6 ] 142465 10.865% 95.182% ## ( 6, 10 ] 38984 2.973% 98.155% # ( 10, 15 ] 5914 0.451% 98.606% ( 15, 22 ] 845 0.064% 98.670% ( 22, 34 ] 88 0.007% 98.677% ( 34, 51 ] 120 0.009% 98.686% ( 51, 76 ] 125 0.010% 98.696% ( 76, 110 ] 1062 0.081% 98.777% ( 110, 170 ] 3793 0.289% 99.066% ( 170, 250 ] 2114 0.161% 99.227% ( 250, 380 ] 1657 0.126% 99.354% ( 380, 580 ] 2258 0.172% 99.526% ( 580, 870 ] 2766 0.211% 99.737% ( 870, 1300 ] 2383 0.182% 99.918% ( 1300, 1900 ] 1525 0.116% 100.035% ( 1900, 2900 ] 1019 0.078% 100.112% ( 2900, 4400 ] 839 0.064% 100.176% ( 4400, 6600 ] 221 0.017% 100.193% ( 6600, 9900 ] 31 0.002% 100.196% ( 9900, 14000 ] 2 0.000% 100.196% ( 14000, 22000 ] 2 0.000% 100.196% ** Level 5 read latency histogram (micros): Count: 77006 Average: 612.4104 StdDev: 907.17 Min: 73 Median: 248.3400 Max: 19229 Percentiles: P50: 248.34 P75: 690.78 P99: 4821.66 P99.9: 7508.95 P99.99: 9842.73 ------------------------------------------------------ ( 51, 76 ] 1 0.001% 0.001% ( 76, 110 ] 6146 7.981% 7.982% ## ( 110, 170 ] 22022 28.598% 36.580% ###### ( 170, 250 ] 10554 13.705% 50.286% ### ( 250, 380 ] 7226 9.384% 59.669% ## ( 380, 580 ] 8817 11.450% 71.119% ## ( 580, 870 ] 7825 10.162% 81.281% ## ( 870, 1300 ] 5865 7.616% 88.897% ## ( 1300, 1900 ] 3120 4.052% 92.949% # ( 1900, 2900 ] 2368 3.075% 96.024% # ( 2900, 4400 ] 2135 2.773% 98.796% # ( 4400, 6600 ] 824 1.070% 99.866% ( 6600, 9900 ] 98 0.127% 99.994% ( 9900, 14000 ] 3 0.004% 99.997% ( 14000, 22000 ] 5 0.006% 100.004% ** DB Stats ** Uptime(secs): 16.6 total, 3.5 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 177.57 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.299 H:M:S, 31.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 282.33 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/16-22:39:25 ... thread 0: (1000000,4000000) ops and (295626.6,201940.7) ops/second in (3.382645,19.807791) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/4 3.38 GB 320.9 3.0 0.0 3.0 6.8 3.7 0.0 1.8 95.8 215.0 32.22 22.81 65 0.496 3134K 3422 0.0 0.0 L1 1/0 31.37 MB 3.1 0.5 0.5 0.0 0.5 0.5 0.0 1.0 75.8 75.6 6.32 2.27 1 6.320 486K 960 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.4 0.1 0.4 0.4 0.4 1.1 107.9 107.3 4.03 2.05 1 4.034 442K 2455 0.0 0.0 L3 19/0 976.28 MB 1.0 1.9 1.1 0.8 1.9 1.1 0.4 1.7 136.7 136.1 13.98 8.90 9 1.553 1462K 1484 0.0 0.0 L4 166/0 9.64 GB 1.0 9.5 4.1 5.4 9.3 3.9 0.7 2.3 119.2 116.6 81.90 44.93 39 2.100 2830K 0 0.0 0.0 L5 1160/0 95.35 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1363/4 109.35 GB 0.0 15.3 6.0 9.3 18.8 9.6 6.8 5.0 113.2 139.3 138.45 80.96 115 1.204 8355K 8321 0.0 0.0 Int 0/0 0.00 KB 0.0 15.3 6.0 9.3 18.8 9.6 6.8 5.0 113.2 139.3 138.45 80.96 115 1.204 8355K 8321 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 15.3 6.0 9.3 15.1 5.8 0.0 0.0 130.7 128.8 119.94 70.14 61 1.966 8355K 8321 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 207.6 18.51 10.82 54 0.343 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.0 total, 19.8 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 18.84 GB write, 963.97 MB/s write, 15.30 GB read, 783.14 MB/s read, 138.4 seconds Interval compaction: 18.84 GB write, 973.72 MB/s write, 15.30 GB read, 791.07 MB/s read, 138.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 35, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 35, interval: 35 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 962568 Average: 9.3084 StdDev: 108.72 Min: 0 Median: 1.5893 Max: 16486 Percentiles: P50: 1.59 P75: 2.15 P99: 118.55 P99.9: 1308.11 P99.99: 2943.28 ------------------------------------------------------ [ 0, 1 ] 164150 17.053% 17.053% ### ( 1, 2 ] 538160 55.909% 72.962% ########### ( 2, 3 ] 133462 13.865% 86.827% ### ( 3, 4 ] 54001 5.610% 92.437% # ( 4, 6 ] 50413 5.237% 97.675% # ( 6, 10 ] 9645 1.002% 98.677% ( 10, 15 ] 1867 0.194% 98.871% ( 15, 22 ] 217 0.023% 98.893% ( 22, 34 ] 14 0.001% 98.895% ( 34, 51 ] 22 0.002% 98.897% ( 51, 76 ] 20 0.002% 98.899% ( 76, 110 ] 584 0.061% 98.960% ( 110, 170 ] 2718 0.282% 99.242% ( 170, 250 ] 1729 0.180% 99.422% ( 250, 380 ] 1027 0.107% 99.528% ( 380, 580 ] 1222 0.127% 99.655% ( 580, 870 ] 1278 0.133% 99.788% ( 870, 1300 ] 1069 0.111% 99.899% ( 1300, 1900 ] 550 0.057% 99.956% ( 1900, 2900 ] 315 0.033% 99.989% ( 2900, 4400 ] 303 0.031% 100.021% ( 4400, 6600 ] 59 0.006% 100.027% ( 6600, 9900 ] 3 0.000% 100.027% ( 9900, 14000 ] 1 0.000% 100.027% ( 14000, 22000 ] 1 0.000% 100.027% ** Level 1 read latency histogram (micros): Count: 210600 Average: 8.9444 StdDev: 110.07 Min: 1 Median: 1.8950 Max: 7778 Percentiles: P50: 1.90 P75: 2.84 P99: 10.88 P99.9: 1560.31 P99.99: 3344.73 ------------------------------------------------------ [ 0, 1 ] 5954 2.827% 2.827% # ( 1, 2 ] 111001 52.707% 55.534% ########### ( 2, 3 ] 48647 23.099% 78.633% ##### ( 3, 4 ] 18401 8.737% 87.371% ## ( 4, 6 ] 18545 8.806% 96.177% ## ( 6, 10 ] 5797 2.753% 98.929% # ( 10, 15 ] 850 0.404% 99.333% ( 15, 22 ] 126 0.060% 99.393% ( 22, 34 ] 26 0.012% 99.405% ( 34, 51 ] 29 0.014% 99.419% ( 51, 76 ] 11 0.005% 99.424% ( 76, 110 ] 27 0.013% 99.437% ( 110, 170 ] 96 0.046% 99.482% ( 170, 250 ] 105 0.050% 99.532% ( 250, 380 ] 131 0.062% 99.594% ( 380, 580 ] 201 0.095% 99.690% ( 580, 870 ] 216 0.103% 99.792% ( 870, 1300 ] 170 0.081% 99.873% ( 1300, 1900 ] 130 0.062% 99.935% ( 1900, 2900 ] 94 0.045% 99.980% ( 2900, 4400 ] 74 0.035% 100.015% ( 4400, 6600 ] 17 0.008% 100.023% ( 6600, 9900 ] 1 0.000% 100.023% ** Level 2 read latency histogram (micros): Count: 286776 Average: 11.4409 StdDev: 147.62 Min: 1 Median: 2.0558 Max: 9654 Percentiles: P50: 2.06 P75: 3.18 P99: 11.68 P99.9: 1787.61 P99.99: 3094.06 ------------------------------------------------------ [ 0, 1 ] 6013 2.097% 2.097% ( 1, 2 ] 133465 46.540% 48.637% ######### ( 2, 3 ] 70047 24.426% 73.062% ##### ( 3, 4 ] 30736 10.718% 83.780% ## ( 4, 6 ] 34043 11.871% 95.651% ## ( 6, 10 ] 9060 3.159% 98.810% # ( 10, 15 ] 1619 0.565% 99.375% ( 15, 22 ] 243 0.085% 99.460% ( 22, 34 ] 44 0.015% 99.475% ( 34, 51 ] 36 0.013% 99.487% ( 51, 76 ] 31 0.011% 99.498% ( 76, 110 ] 23 0.008% 99.506% ( 110, 170 ] 46 0.016% 99.522% ( 170, 250 ] 59 0.021% 99.543% ( 250, 380 ] 134 0.047% 99.590% ( 380, 580 ] 193 0.067% 99.657% ( 580, 870 ] 258 0.090% 99.747% ( 870, 1300 ] 271 0.094% 99.841% ( 1300, 1900 ] 207 0.072% 99.914% ( 1900, 2900 ] 195 0.068% 99.982% ( 2900, 4400 ] 188 0.066% 100.047% ( 4400, 6600 ] 50 0.017% 100.065% ( 6600, 9900 ] 8 0.003% 100.067% ** Level 3 read latency histogram (micros): Count: 994114 Average: 8.4133 StdDev: 111.62 Min: 1 Median: 1.9570 Max: 11335 Percentiles: P50: 1.96 P75: 2.96 P99: 9.79 P99.9: 803.39 P99.99: 1266.00 ------------------------------------------------------ [ 0, 1 ] 20625 2.075% 2.075% ( 1, 2 ] 497862 50.081% 52.156% ########## ( 2, 3 ] 235724 23.712% 75.868% ##### ( 3, 4 ] 97414 9.799% 85.667% ## ( 4, 6 ] 105540 10.616% 96.283% ## ( 6, 10 ] 28491 2.866% 99.149% # ( 10, 15 ] 4026 0.405% 99.554% ( 15, 22 ] 583 0.059% 99.613% ( 22, 34 ] 64 0.006% 99.619% ( 34, 51 ] 85 0.009% 99.628% ( 51, 76 ] 59 0.006% 99.634% ( 76, 110 ] 168 0.017% 99.651% ( 110, 170 ] 529 0.053% 99.704% ( 170, 250 ] 330 0.033% 99.737% ( 250, 380 ] 374 0.038% 99.775% ( 380, 580 ] 605 0.061% 99.836% ( 580, 870 ] 832 0.084% 99.919% ( 870, 1300 ] 764 0.077% 99.996% ( 1300, 1900 ] 468 0.047% 100.043% ( 1900, 2900 ] 392 0.039% 100.083% ( 2900, 4400 ] 347 0.035% 100.117% ( 4400, 6600 ] 89 0.009% 100.126% ( 6600, 9900 ] 10 0.001% 100.127% ( 9900, 14000 ] 3 0.000% 100.128% ** Level 4 read latency histogram (micros): Count: 1318920 Average: 16.3316 StdDev: 158.68 Min: 1 Median: 1.9957 Max: 15123 Percentiles: P50: 2.00 P75: 3.11 P99: 207.10 P99.9: 1256.41 P99.99: 1670.57 ------------------------------------------------------ [ 0, 1 ] 26619 2.018% 2.018% ( 1, 2 ] 635574 48.189% 50.207% ########## ( 2, 3 ] 312790 23.716% 73.923% ##### ( 3, 4 ] 130743 9.913% 83.836% ## ( 4, 6 ] 142615 10.813% 94.649% ## ( 6, 10 ] 39006 2.957% 97.606% # ( 10, 15 ] 5917 0.449% 98.055% ( 15, 22 ] 845 0.064% 98.119% ( 22, 34 ] 88 0.007% 98.126% ( 34, 51 ] 120 0.009% 98.135% ( 51, 76 ] 125 0.009% 98.144% ( 76, 110 ] 2217 0.168% 98.312% ( 110, 170 ] 7462 0.566% 98.878% ( 170, 250 ] 3471 0.263% 99.141% ( 250, 380 ] 2207 0.167% 99.308% ( 380, 580 ] 2741 0.208% 99.516% ( 580, 870 ] 2899 0.220% 99.736% ( 870, 1300 ] 2406 0.182% 99.918% ( 1300, 1900 ] 1527 0.116% 100.034% ( 1900, 2900 ] 1021 0.077% 100.112% ( 2900, 4400 ] 840 0.064% 100.175% ( 4400, 6600 ] 221 0.017% 100.192% ( 6600, 9900 ] 31 0.002% 100.194% ( 9900, 14000 ] 2 0.000% 100.195% ( 14000, 22000 ] 2 0.000% 100.195% ** Level 5 read latency histogram (micros): Count: 120335 Average: 461.2165 StdDev: 757.88 Min: 73 Median: 185.4546 Max: 19229 Percentiles: P50: 185.45 P75: 461.98 P99: 4206.39 P99.9: 6553.72 P99.99: 9663.16 ------------------------------------------------------ ( 51, 76 ] 2 0.002% 0.002% ( 76, 110 ] 12960 10.770% 10.772% ## ( 110, 170 ] 43649 36.273% 47.045% ####### ( 170, 250 ] 18410 15.299% 62.343% ### ( 250, 380 ] 10450 8.684% 71.028% ## ( 380, 580 ] 11662 9.691% 80.719% ## ( 580, 870 ] 8646 7.185% 87.904% # ( 870, 1300 ] 5964 4.956% 92.860% # ( 1300, 1900 ] 3131 2.602% 95.462% # ( 1900, 2900 ] 2393 1.989% 97.450% ( 2900, 4400 ] 2141 1.779% 99.230% ( 4400, 6600 ] 824 0.685% 99.914% ( 6600, 9900 ] 98 0.081% 99.996% ( 9900, 14000 ] 3 0.002% 99.998% ( 14000, 22000 ] 5 0.004% 100.002% ** DB Stats ** Uptime(secs): 20.0 total, 3.4 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 196.74 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.299 H:M:S, 26.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 290.95 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/16-22:39:46 ... thread 0: (1000000,5000000) ops and (47309.8,122114.8) ops/second in (21.137272,40.945063) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/17 4.42 GB 50.3 4.2 0.0 4.2 9.0 4.8 0.0 1.9 107.4 228.6 40.49 30.54 87 0.465 4413K 4643 0.0 0.0 L1 1/1 31.37 MB 0.0 0.5 0.5 0.0 0.5 0.5 0.0 1.0 75.8 75.6 6.32 2.27 1 6.320 486K 960 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.4 0.1 0.4 0.4 0.4 1.1 107.9 107.3 4.03 2.05 1 4.034 442K 2455 0.0 0.0 L3 19/0 976.28 MB 1.0 1.9 1.1 0.8 1.9 1.1 0.4 1.7 136.7 136.1 13.98 8.90 9 1.553 1462K 1484 0.0 0.0 L4 166/0 9.64 GB 1.0 9.5 4.1 5.4 9.3 3.9 0.7 2.3 119.2 116.6 81.90 44.93 39 2.100 2830K 0 0.0 0.0 L5 1160/0 95.35 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1365/18 110.40 GB 0.0 16.5 6.0 10.5 21.1 10.6 6.8 4.4 115.4 147.3 146.72 88.69 137 1.071 9634K 9542 0.0 0.0 Int 0/0 0.00 KB 0.0 16.5 6.0 10.5 21.1 10.6 6.8 4.4 115.4 147.3 146.72 88.69 137 1.071 9634K 9542 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 16.5 6.0 10.5 16.3 5.8 0.0 0.0 135.7 133.9 124.79 74.87 66 1.891 9634K 9542 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.8 4.8 0.0 0.0 0.0 224.1 21.93 13.82 71 0.309 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 41.1 total, 40.9 interval Flush(GB): cumulative 4.799, interval 4.799 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 21.11 GB write, 525.40 MB/s write, 16.53 GB read, 411.46 MB/s read, 146.7 seconds Interval compaction: 21.11 GB write, 527.97 MB/s write, 16.53 GB read, 413.48 MB/s read, 146.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 10, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 10, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 35, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 10, total-stops: 35, interval: 45 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1536464 Average: 17.6529 StdDev: 96.54 Min: 0 Median: 1.7337 Max: 16486 Percentiles: P50: 1.73 P75: 3.11 P99: 212.46 P99.9: 1007.93 P99.99: 2121.96 ------------------------------------------------------ [ 0, 1 ] 252079 16.406% 16.406% ### ( 1, 2 ] 703532 45.789% 62.195% ######### ( 2, 3 ] 178795 11.637% 73.832% ## ( 3, 4 ] 161110 10.486% 84.318% ## ( 4, 6 ] 94232 6.133% 90.451% # ( 6, 10 ] 16574 1.079% 91.530% ( 10, 15 ] 3202 0.208% 91.738% ( 15, 22 ] 363 0.024% 91.762% ( 22, 34 ] 25 0.002% 91.763% ( 34, 51 ] 23 0.001% 91.765% ( 51, 76 ] 26 0.002% 91.767% ( 76, 110 ] 24814 1.615% 93.382% ( 110, 170 ] 78739 5.125% 98.506% # ( 170, 250 ] 14291 0.930% 99.436% ( 250, 380 ] 2768 0.180% 99.617% ( 380, 580 ] 2447 0.159% 99.776% ( 580, 870 ] 1555 0.101% 99.877% ( 870, 1300 ] 1099 0.072% 99.949% ( 1300, 1900 ] 560 0.036% 99.985% ( 1900, 2900 ] 344 0.022% 100.007% ( 2900, 4400 ] 337 0.022% 100.029% ( 4400, 6600 ] 59 0.004% 100.033% ( 6600, 9900 ] 3 0.000% 100.033% ( 9900, 14000 ] 1 0.000% 100.033% ( 14000, 22000 ] 1 0.000% 100.034% ** Level 1 read latency histogram (micros): Count: 210995 Average: 9.1904 StdDev: 110.16 Min: 1 Median: 1.8968 Max: 7778 Percentiles: P50: 1.90 P75: 2.85 P99: 13.18 P99.9: 1558.48 P99.99: 3343.93 ------------------------------------------------------ [ 0, 1 ] 5954 2.822% 2.822% # ( 1, 2 ] 111001 52.608% 55.430% ########### ( 2, 3 ] 48647 23.056% 78.486% ##### ( 3, 4 ] 18401 8.721% 87.207% ## ( 4, 6 ] 18545 8.789% 95.997% ## ( 6, 10 ] 5797 2.747% 98.744% # ( 10, 15 ] 850 0.403% 99.147% ( 15, 22 ] 126 0.060% 99.207% ( 22, 34 ] 26 0.012% 99.219% ( 34, 51 ] 29 0.014% 99.233% ( 51, 76 ] 11 0.005% 99.238% ( 76, 110 ] 150 0.071% 99.309% ( 110, 170 ] 324 0.154% 99.463% ( 170, 250 ] 138 0.065% 99.528% ( 250, 380 ] 135 0.064% 99.592% ( 380, 580 ] 205 0.097% 99.689% ( 580, 870 ] 219 0.104% 99.793% ( 870, 1300 ] 170 0.081% 99.873% ( 1300, 1900 ] 130 0.062% 99.935% ( 1900, 2900 ] 94 0.045% 99.980% ( 2900, 4400 ] 74 0.035% 100.015% ( 4400, 6600 ] 17 0.008% 100.023% ( 6600, 9900 ] 1 0.000% 100.023% ** Level 2 read latency histogram (micros): Count: 286860 Average: 11.4804 StdDev: 147.62 Min: 1 Median: 2.0564 Max: 9654 Percentiles: P50: 2.06 P75: 3.18 P99: 11.94 P99.9: 1787.36 P99.99: 3093.99 ------------------------------------------------------ [ 0, 1 ] 6013 2.096% 2.096% ( 1, 2 ] 133465 46.526% 48.622% ######### ( 2, 3 ] 70047 24.419% 73.041% ##### ( 3, 4 ] 30736 10.715% 83.755% ## ( 4, 6 ] 34043 11.867% 95.623% ## ( 6, 10 ] 9060 3.158% 98.781% # ( 10, 15 ] 1619 0.564% 99.346% ( 15, 22 ] 243 0.085% 99.430% ( 22, 34 ] 44 0.015% 99.446% ( 34, 51 ] 36 0.013% 99.458% ( 51, 76 ] 31 0.011% 99.469% ( 76, 110 ] 38 0.013% 99.482% ( 110, 170 ] 102 0.036% 99.518% ( 170, 250 ] 69 0.024% 99.542% ( 250, 380 ] 136 0.047% 99.589% ( 380, 580 ] 194 0.068% 99.657% ( 580, 870 ] 258 0.090% 99.747% ( 870, 1300 ] 271 0.094% 99.841% ( 1300, 1900 ] 207 0.072% 99.914% ( 1900, 2900 ] 195 0.068% 99.982% ( 2900, 4400 ] 188 0.066% 100.047% ( 4400, 6600 ] 50 0.017% 100.064% ( 6600, 9900 ] 8 0.003% 100.067% ** Level 3 read latency histogram (micros): Count: 1004044 Average: 9.7293 StdDev: 112.07 Min: 1 Median: 1.9669 Max: 11335 Percentiles: P50: 1.97 P75: 2.99 P99: 118.83 P99.9: 803.84 P99.99: 1266.74 ------------------------------------------------------ [ 0, 1 ] 20625 2.054% 2.054% ( 1, 2 ] 497862 49.586% 51.640% ########## ( 2, 3 ] 235731 23.478% 75.118% ##### ( 3, 4 ] 97479 9.709% 84.827% ## ( 4, 6 ] 105609 10.518% 95.345% ## ( 6, 10 ] 28498 2.838% 98.183% # ( 10, 15 ] 4027 0.401% 98.584% ( 15, 22 ] 583 0.058% 98.642% ( 22, 34 ] 64 0.006% 98.649% ( 34, 51 ] 85 0.008% 98.657% ( 51, 76 ] 59 0.006% 98.663% ( 76, 110 ] 2369 0.236% 98.899% ( 110, 170 ] 6880 0.685% 99.584% ( 170, 250 ] 1262 0.126% 99.710% ( 250, 380 ] 522 0.052% 99.762% ( 380, 580 ] 725 0.072% 99.834% ( 580, 870 ] 855 0.085% 99.919% ( 870, 1300 ] 768 0.076% 99.996% ( 1300, 1900 ] 468 0.047% 100.043% ( 1900, 2900 ] 394 0.039% 100.082% ( 2900, 4400 ] 347 0.035% 100.116% ( 4400, 6600 ] 89 0.009% 100.125% ( 6600, 9900 ] 10 0.001% 100.126% ( 9900, 14000 ] 3 0.000% 100.126% ** Level 4 read latency histogram (micros): Count: 1380127 Average: 21.8301 StdDev: 157.88 Min: 1 Median: 2.0891 Max: 15123 Percentiles: P50: 2.09 P75: 3.46 P99: 245.31 P99.9: 1247.14 P99.99: 1670.83 ------------------------------------------------------ [ 0, 1 ] 26619 1.929% 1.929% ( 1, 2 ] 635576 46.052% 47.981% ######### ( 2, 3 ] 312817 22.666% 70.647% ##### ( 3, 4 ] 131428 9.523% 80.169% ## ( 4, 6 ] 143431 10.393% 90.562% ## ( 6, 10 ] 39064 2.830% 93.392% # ( 10, 15 ] 5932 0.430% 93.822% ( 15, 22 ] 848 0.061% 93.884% ( 22, 34 ] 89 0.006% 93.890% ( 34, 51 ] 120 0.009% 93.899% ( 51, 76 ] 125 0.009% 93.908% ( 76, 110 ] 15437 1.119% 95.026% ( 110, 170 ] 46136 3.343% 98.369% # ( 170, 250 ] 9246 0.670% 99.039% ( 250, 380 ] 3172 0.230% 99.269% ( 380, 580 ] 3519 0.255% 99.524% ( 580, 870 ] 3068 0.222% 99.746% ( 870, 1300 ] 2417 0.175% 99.922% ( 1300, 1900 ] 1529 0.111% 100.032% ( 1900, 2900 ] 1024 0.074% 100.107% ( 2900, 4400 ] 843 0.061% 100.168% ( 4400, 6600 ] 221 0.016% 100.184% ( 6600, 9900 ] 31 0.002% 100.186% ( 9900, 14000 ] 2 0.000% 100.186% ( 14000, 22000 ] 2 0.000% 100.186% ** Level 5 read latency histogram (micros): Count: 460634 Average: 226.8580 StdDev: 416.02 Min: 73 Median: 142.0088 Max: 19229 Percentiles: P50: 142.01 P75: 168.22 P99: 2273.12 P99.9: 5642.48 P99.99: 8483.58 ------------------------------------------------------ ( 51, 76 ] 2 0.000% 0.000% ( 76, 110 ] 89664 19.465% 19.466% #### ( 110, 170 ] 263648 57.236% 76.702% ########### ( 170, 250 ] 50924 11.055% 87.757% ## ( 250, 380 ] 16060 3.486% 91.243% # ( 380, 580 ] 15911 3.454% 94.698% # ( 580, 870 ] 9679 2.101% 96.799% ( 870, 1300 ] 6073 1.318% 98.117% ( 1300, 1900 ] 3157 0.685% 98.803% ( 1900, 2900 ] 2438 0.529% 99.332% ( 2900, 4400 ] 2152 0.467% 99.799% ( 4400, 6600 ] 824 0.179% 99.978% ( 6600, 9900 ] 98 0.021% 99.999% ( 9900, 14000 ] 3 0.001% 100.000% ( 14000, 22000 ] 5 0.001% 100.001% ** DB Stats ** Uptime(secs): 41.1 total, 21.1 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 119.59 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:22.275 H:M:S, 54.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 46.56 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:16.977 H:M:S, 80.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 250.845 micros/op 15937 ops/sec 60.174 seconds 958996 operations; 15.4 MB/s (239999 of 239999 found) Microseconds per read: Count: 958996 Average: 250.8475 StdDev: 686.28 Min: 3 Median: 188.9224 Max: 65916 Percentiles: P50: 188.92 P75: 235.44 P99: 1258.19 P99.9: 11573.31 P99.99: 28648.46 ------------------------------------------------------ ( 2, 3 ] 1 0.000% 0.000% ( 3, 4 ] 4 0.000% 0.001% ( 4, 6 ] 61 0.006% 0.007% ( 6, 10 ] 256 0.027% 0.034% ( 10, 15 ] 119 0.012% 0.046% ( 15, 22 ] 366 0.038% 0.084% ( 22, 34 ] 16092 1.678% 1.762% ( 34, 51 ] 6389 0.666% 2.428% ( 51, 76 ] 1061 0.111% 2.539% ( 76, 110 ] 428 0.045% 2.584% ( 110, 170 ] 357202 37.247% 39.831% ####### ( 170, 250 ] 412291 42.992% 82.823% ######### ( 250, 380 ] 96675 10.081% 92.904% ## ( 380, 580 ] 37094 3.868% 96.772% # ( 580, 870 ] 14821 1.545% 98.317% ( 870, 1300 ] 7251 0.756% 99.074% ( 1300, 1900 ] 3327 0.347% 99.420% ( 1900, 2900 ] 2167 0.226% 99.646% ( 2900, 4400 ] 1547 0.161% 99.808% ( 4400, 6600 ] 504 0.053% 99.860% ( 6600, 9900 ] 210 0.022% 99.882% ( 9900, 14000 ] 419 0.044% 99.926% ( 14000, 22000 ] 519 0.054% 99.980% ( 22000, 33000 ] 159 0.017% 99.997% ( 33000, 50000 ] 28 0.003% 99.999% ( 50000, 75000 ] 5 0.001% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 19820050 rocksdb.bloom.filter.full.positive COUNT : 1152870 rocksdb.bloom.filter.full.true.positive COUNT : 958561 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 435 rocksdb.memtable.miss COUNT : 958561 rocksdb.l0.hit COUNT : 39603 rocksdb.l1.hit COUNT : 315 rocksdb.l2andup.hit COUNT : 918643 rocksdb.compaction.key.drop.new COUNT : 256575 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 5166906 rocksdb.number.keys.read COUNT : 958996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 5332246992 rocksdb.bytes.read COUNT : 958996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1604 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 40405238 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 19265052716 rocksdb.compact.write.bytes COUNT : 19000989733 rocksdb.flush.write.bytes COUNT : 5284646001 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 40 rocksdb.number.superversion_acquires COUNT : 954 rocksdb.number.superversion_releases COUNT : 929 rocksdb.number.superversion_cleanups COUNT : 243 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 229 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 5241999360 rocksdb.memtable.garbage.bytes.at.flush COUNT : 2011136 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 23588617861 rocksdb.non.last.level.read.count COUNT : 5555210 rocksdb.block.checksum.compute.count COUNT : 5832856 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 187.319440 P95 : 486.611715 P99 : 1257.401024 P100 : 65914.000000 COUNT : 958996 SUM : 239233957 rocksdb.db.write.micros P50 : 1.541512 P95 : 4.630476 P99 : 9.985225 P100 : 924099.000000 COUNT : 5166906 SUM : 53556976 rocksdb.compaction.times.micros P50 : 1822222.222222 P95 : 3840000.000000 P99 : 5048000.000000 P100 : 6320302.000000 COUNT : 66 SUM : 124790158 rocksdb.compaction.times.cpu_micros P50 : 1242424.242424 P95 : 1872424.242424 P99 : 2274092.000000 P100 : 2274092.000000 COUNT : 66 SUM : 74870052 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 777.727273 P95 : 177000.000000 P99 : 235400.000000 P100 : 239173.000000 COUNT : 73 SUM : 1332775 rocksdb.compaction.outfile.sync.micros P50 : 64375.000000 P95 : 249866.666667 P99 : 436050.000000 P100 : 867846.000000 COUNT : 241 SUM : 20903507 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 190.000000 P95 : 4633.750000 P99 : 6426.750000 P100 : 10356.000000 COUNT : 163 SUM : 153796 rocksdb.table.open.io.micros P50 : 9086.199095 P95 : 19745.454545 P99 : 21866.446281 P100 : 25758.000000 COUNT : 1604 SUM : 14676330 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.043470 P95 : 12.627051 P99 : 167.906812 P100 : 18508.000000 COUNT : 4666002 SUM : 70505031 rocksdb.read.block.get.micros P50 : 137.426818 P95 : 338.071283 P99 : 1179.031155 P100 : 22858.000000 COUNT : 1165257 SUM : 195516113 rocksdb.write.raw.block.micros P50 : 0.545664 P95 : 1.547220 P99 : 4.676605 P100 : 19510.000000 COUNT : 5882420 SUM : 17482585 rocksdb.numfiles.in.singlecompaction P50 : 3.385246 P95 : 4.985714 P99 : 15.630000 P100 : 18.000000 COUNT : 91 SUM : 344 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.595712 P95 : 2233.428138 P99 : 2767.671597 P100 : 923931.000000 COUNT : 140880 SUM : 40423203 rocksdb.sst.read.micros P50 : 2.443208 P95 : 164.729531 P99 : 480.540331 P100 : 19229.000000 COUNT : 5555281 SUM : 248843639 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 958996 SUM : 958996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 5166906 SUM : 5332246992 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 223090.909091 P95 : 889750.000000 P99 : 1137950.000000 P100 : 1177432.000000 COUNT : 73 SUM : 22330142 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 237 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0