[2024-11-19 19:24:30.475672] Starting SPDK v24.09.1-pre git sha1 b18e1bd62 / DPDK 22.11.4 initialization... [2024-11-19 19:24:30.475813] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1389480 ] Set seed to 1732040670472188 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-19 19:24:30.564251] app.c: 917:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-19 19:24:30.606760] app.c: 610:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-19 19:24:30.606807] app.c: 611:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1389480' to capture a snapshot of events at runtime. [2024-11-19 19:24:30.606818] app.c: 616:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-19 19:24:30.606828] app.c: 617:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-19 19:24:30.606836] app.c: 618:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1389480 for offline analysis/debug. [2024-11-19 19:24:30.606922] reactor.c: 990:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Tue Nov 19 19:24:34 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/19-19:24:43 ... thread 0: (1000000,1000000) ops and (118679.2,118679.2) ops/second in (8.426075,8.426075) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/12 2.06 GB 44.1 0.3 0.0 0.3 1.1 0.8 0.0 1.4 33.8 121.6 9.32 3.71 10 0.932 319K 369 0.0 0.0 L1 8/8 837.83 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 21/7 1.62 GB 0.9 1.3 0.8 0.5 1.3 0.8 0.0 1.6 110.3 108.7 11.89 6.03 5 2.378 415K 0 0.0 0.0 L4 195/21 10.59 GB 1.0 5.1 3.0 2.1 5.0 2.9 0.0 1.7 110.1 108.1 47.63 24.66 32 1.488 1090K 0 0.0 0.0 L5 1120/0 92.28 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1358/48 107.36 GB 0.0 6.7 3.8 2.9 7.4 4.5 2.0 9.2 99.8 110.0 68.84 34.41 47 1.465 1826K 369 0.0 0.0 Int 0/0 0.00 KB 0.0 6.7 3.8 2.9 7.4 4.5 2.0 9.2 99.8 110.0 68.84 34.41 47 1.465 1826K 369 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.7 3.8 2.9 6.6 3.7 0.0 0.0 111.1 109.2 61.87 31.96 38 1.628 1826K 369 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 117.4 6.97 2.45 9 0.775 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.6 total, 8.4 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.40 GB write, 876.84 MB/s write, 6.71 GB read, 795.67 MB/s read, 68.8 seconds Interval compaction: 7.40 GB write, 898.64 MB/s write, 6.71 GB read, 815.45 MB/s read, 68.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 27, pending-compaction-bytes-delays: 11, pending-compaction-bytes-stops: 0, total-delays: 11, total-stops: 27, interval: 38 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 306462 Average: 22.0796 StdDev: 242.83 Min: 1 Median: 1.9227 Max: 56552 Percentiles: P50: 1.92 P75: 2.91 P99: 631.41 P99.9: 3494.07 P99.99: 5376.09 ------------------------------------------------------ [ 0, 1 ] 9574 3.124% 3.124% # ( 1, 2 ] 155685 50.801% 53.925% ########## ( 2, 3 ] 70626 23.046% 76.970% ##### ( 3, 4 ] 28741 9.378% 86.349% ## ( 4, 6 ] 29196 9.527% 95.876% ## ( 6, 10 ] 7170 2.340% 98.215% ( 10, 15 ] 1148 0.375% 98.590% ( 15, 22 ] 151 0.049% 98.639% ( 22, 34 ] 13 0.004% 98.643% ( 34, 51 ] 8 0.003% 98.646% ( 51, 76 ] 13 0.004% 98.650% ( 76, 110 ] 6 0.002% 98.652% ( 110, 170 ] 48 0.016% 98.668% ( 170, 250 ] 109 0.036% 98.703% ( 250, 380 ] 256 0.084% 98.787% ( 380, 580 ] 495 0.162% 98.948% ( 580, 870 ] 899 0.293% 99.242% ( 870, 1300 ] 912 0.298% 99.539% ( 1300, 1900 ] 569 0.186% 99.725% ( 1900, 2900 ] 391 0.128% 99.853% ( 2900, 4400 ] 370 0.121% 99.973% ( 4400, 6600 ] 118 0.039% 100.012% ( 6600, 9900 ] 8 0.003% 100.014% ( 9900, 14000 ] 1 0.000% 100.015% ( 14000, 22000 ] 1 0.000% 100.015% ( 22000, 33000 ] 1 0.000% 100.015% ( 50000, 75000 ] 1 0.000% 100.016% ** Level 1 read latency histogram (micros): Count: 152 Average: 1495.4408 StdDev: 1277.30 Min: 148 Median: 1051.9231 Max: 6437 Percentiles: P50: 1051.92 P75: 2017.65 P99: 6042.67 P99.9: 6437.00 P99.99: 6437.00 ------------------------------------------------------ ( 110, 170 ] 1 0.658% 0.658% ( 170, 250 ] 1 0.658% 1.316% ( 250, 380 ] 13 8.553% 9.868% ## ( 380, 580 ] 22 14.474% 24.342% ### ( 580, 870 ] 28 18.421% 42.763% #### ( 870, 1300 ] 26 17.105% 59.868% ### ( 1300, 1900 ] 21 13.816% 73.684% ### ( 1900, 2900 ] 17 11.184% 84.868% ## ( 2900, 4400 ] 17 11.184% 96.053% ## ( 4400, 6600 ] 6 3.947% 100.000% # ** Level 2 read latency histogram (micros): Count: 91270 Average: 12.0961 StdDev: 157.50 Min: 1 Median: 1.9858 Max: 9485 Percentiles: P50: 1.99 P75: 2.98 P99: 9.98 P99.9: 3023.11 P99.99: 5345.04 ------------------------------------------------------ [ 0, 1 ] 1501 1.645% 1.645% ( 1, 2 ] 44771 49.053% 50.698% ########## ( 2, 3 ] 22613 24.776% 75.474% ##### ( 3, 4 ] 9305 10.195% 85.669% ## ( 4, 6 ] 9659 10.583% 96.252% ## ( 6, 10 ] 2524 2.765% 99.017% # ( 10, 15 ] 355 0.389% 99.406% ( 15, 22 ] 39 0.043% 99.449% ( 22, 34 ] 7 0.008% 99.457% ( 34, 51 ] 2 0.002% 99.459% ( 51, 76 ] 7 0.008% 99.466% ( 76, 110 ] 1 0.001% 99.468% ( 110, 170 ] 3 0.003% 99.471% ( 170, 250 ] 9 0.010% 99.481% ( 250, 380 ] 14 0.015% 99.496% ( 380, 580 ] 58 0.064% 99.560% ( 580, 870 ] 82 0.090% 99.649% ( 870, 1300 ] 103 0.113% 99.762% ( 1300, 1900 ] 60 0.066% 99.828% ( 1900, 2900 ] 59 0.065% 99.893% ( 2900, 4400 ] 82 0.090% 99.982% ( 4400, 6600 ] 16 0.018% 100.000% ( 6600, 9900 ] 2 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 858463 Average: 9.7048 StdDev: 155.89 Min: 1 Median: 1.9945 Max: 38754 Percentiles: P50: 1.99 P75: 2.97 P99: 9.57 P99.9: 772.97 P99.99: 1197.12 ------------------------------------------------------ [ 0, 1 ] 14277 1.663% 1.663% ( 1, 2 ] 417256 48.605% 50.268% ########## ( 2, 3 ] 217855 25.377% 75.645% ##### ( 3, 4 ] 88154 10.269% 85.914% ## ( 4, 6 ] 93008 10.834% 96.748% ## ( 6, 10 ] 21661 2.523% 99.272% # ( 10, 15 ] 3536 0.412% 99.684% ( 15, 22 ] 491 0.057% 99.741% ( 22, 34 ] 69 0.008% 99.749% ( 34, 51 ] 47 0.005% 99.754% ( 51, 76 ] 58 0.007% 99.761% ( 76, 110 ] 28 0.003% 99.764% ( 110, 170 ] 44 0.005% 99.769% ( 170, 250 ] 90 0.010% 99.780% ( 250, 380 ] 208 0.024% 99.804% ( 380, 580 ] 398 0.046% 99.851% ( 580, 870 ] 638 0.074% 99.925% ( 870, 1300 ] 735 0.086% 100.010% ( 1300, 1900 ] 498 0.058% 100.068% ( 1900, 2900 ] 438 0.051% 100.120% ( 2900, 4400 ] 419 0.049% 100.168% ( 4400, 6600 ] 102 0.012% 100.180% ( 6600, 9900 ] 26 0.003% 100.183% ( 9900, 14000 ] 1 0.000% 100.183% ( 14000, 22000 ] 6 0.001% 100.184% ( 22000, 33000 ] 2 0.000% 100.184% ( 33000, 50000 ] 2 0.000% 100.185% ** Level 4 read latency histogram (micros): Count: 695657 Average: 18.0505 StdDev: 239.73 Min: 1 Median: 2.0139 Max: 56432 Percentiles: P50: 2.01 P75: 3.01 P99: 14.76 P99.9: 1755.34 P99.99: 2489.98 ------------------------------------------------------ [ 0, 1 ] 11314 1.626% 1.626% ( 1, 2 ] 334072 48.023% 49.649% ########## ( 2, 3 ] 175974 25.296% 74.945% ##### ( 3, 4 ] 71271 10.245% 85.190% ## ( 4, 6 ] 74064 10.647% 95.837% ## ( 6, 10 ] 18955 2.725% 98.562% # ( 10, 15 ] 3204 0.461% 99.022% ( 15, 22 ] 400 0.057% 99.080% ( 22, 34 ] 54 0.008% 99.087% ( 34, 51 ] 58 0.008% 99.096% ( 51, 76 ] 71 0.010% 99.106% ( 76, 110 ] 33 0.005% 99.111% ( 110, 170 ] 98 0.014% 99.125% ( 170, 250 ] 227 0.033% 99.157% ( 250, 380 ] 452 0.065% 99.222% ( 380, 580 ] 976 0.140% 99.363% ( 580, 870 ] 1440 0.207% 99.570% ( 870, 1300 ] 1544 0.222% 99.792% ( 1300, 1900 ] 994 0.143% 99.934% ( 1900, 2900 ] 655 0.094% 100.029% ( 2900, 4400 ] 680 0.098% 100.126% ( 4400, 6600 ] 197 0.028% 100.155% ( 6600, 9900 ] 25 0.004% 100.158% ( 9900, 14000 ] 3 0.000% 100.159% ( 14000, 22000 ] 8 0.001% 100.160% ( 33000, 50000 ] 4 0.001% 100.160% ( 50000, 75000 ] 2 0.000% 100.161% ** Level 5 read latency histogram (micros): Count: 14478 Average: 1639.3363 StdDev: 1703.45 Min: 81 Median: 1104.6941 Max: 61948 Percentiles: P50: 1104.69 P75: 2178.11 P99: 6532.89 P99.9: 11962.89 P99.99: 46193.70 ------------------------------------------------------ ( 76, 110 ] 19 0.131% 0.131% ( 110, 170 ] 109 0.753% 0.884% ( 170, 250 ] 271 1.872% 2.756% ( 250, 380 ] 790 5.457% 8.212% # ( 380, 580 ] 1740 12.018% 20.231% ## ( 580, 870 ] 2731 18.863% 39.094% #### ( 870, 1300 ] 2893 19.982% 59.076% #### ( 1300, 1900 ] 1890 13.054% 72.130% ### ( 1900, 2900 ] 1494 10.319% 82.449% ## ( 2900, 4400 ] 1704 11.770% 94.219% ## ( 4400, 6600 ] 714 4.932% 99.150% # ( 6600, 9900 ] 105 0.725% 99.876% ( 9900, 14000 ] 7 0.048% 99.924% ( 14000, 22000 ] 5 0.035% 99.959% ( 22000, 33000 ] 3 0.021% 99.979% ( 33000, 50000 ] 2 0.014% 99.993% ( 50000, 75000 ] 2 0.014% 100.007% ** DB Stats ** Uptime(secs): 8.6 total, 8.4 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 113.93 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.347 H:M:S, 50.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 116.76 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.347 H:M:S, 51.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 9 num-running-flushes: 1 2024/11/19-19:24:47 ... thread 0: (1000000,2000000) ops and (258743.5,162721.9) ops/second in (3.864832,12.290907) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/12 3.10 GB 163.6 1.4 0.0 1.4 3.3 1.8 0.0 1.8 82.0 188.9 17.67 10.92 31 0.570 1471K 1605 0.0 0.0 L1 8/8 837.83 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 14/0 924.68 MB 0.9 1.3 0.8 0.5 1.3 0.8 0.0 1.6 110.3 108.7 11.89 6.03 5 2.378 415K 0 0.0 0.0 L4 185/0 9.67 GB 1.0 6.7 3.7 3.0 6.6 3.6 0.0 1.8 114.1 111.9 60.34 32.25 39 1.547 1494K 0 0.0 0.0 L5 1140/0 93.87 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1366/20 108.37 GB 0.0 9.4 4.5 4.9 11.1 6.2 3.6 6.0 107.3 126.6 89.90 49.20 75 1.199 3381K 1605 0.0 0.0 Int 0/0 0.00 KB 0.0 9.4 4.5 4.9 11.1 6.2 3.6 6.0 107.3 126.6 89.90 49.20 75 1.199 3381K 1605 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.4 4.5 4.9 9.3 4.4 0.0 0.0 121.8 119.9 79.20 43.86 49 1.616 3381K 1605 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 176.6 10.70 5.34 26 0.412 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.5 total, 12.3 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.12 GB write, 910.42 MB/s write, 9.42 GB read, 771.61 MB/s read, 89.9 seconds Interval compaction: 11.12 GB write, 925.94 MB/s write, 9.42 GB read, 784.76 MB/s read, 89.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 2, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 2, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 27, pending-compaction-bytes-delays: 11, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 27, interval: 40 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 575222 Average: 20.8119 StdDev: 185.85 Min: 0 Median: 1.7050 Max: 56552 Percentiles: P50: 1.71 P75: 2.59 P99: 395.66 P99.9: 2283.55 P99.99: 3901.88 ------------------------------------------------------ [ 0, 1 ] 73676 12.808% 12.808% ### ( 1, 2 ] 303438 52.751% 65.560% ########### ( 2, 3 ] 92706 16.117% 81.676% ### ( 3, 4 ] 35761 6.217% 87.893% # ( 4, 6 ] 33486 5.821% 93.715% # ( 6, 10 ] 8328 1.448% 95.162% ( 10, 15 ] 1345 0.234% 95.396% ( 15, 22 ] 184 0.032% 95.428% ( 22, 34 ] 21 0.004% 95.432% ( 34, 51 ] 11 0.002% 95.434% ( 51, 76 ] 16 0.003% 95.437% ( 76, 110 ] 2858 0.497% 95.933% ( 110, 170 ] 10712 1.862% 97.796% ( 170, 250 ] 4814 0.837% 98.633% ( 250, 380 ] 1960 0.341% 98.973% ( 380, 580 ] 1990 0.346% 99.319% ( 580, 870 ] 1490 0.259% 99.578% ( 870, 1300 ] 1072 0.186% 99.765% ( 1300, 1900 ] 617 0.107% 99.872% ( 1900, 2900 ] 427 0.074% 99.946% ( 2900, 4400 ] 381 0.066% 100.012% ( 4400, 6600 ] 121 0.021% 100.033% ( 6600, 9900 ] 8 0.001% 100.035% ( 9900, 14000 ] 1 0.000% 100.035% ( 14000, 22000 ] 1 0.000% 100.035% ( 22000, 33000 ] 1 0.000% 100.035% ( 50000, 75000 ] 1 0.000% 100.035% ** Level 1 read latency histogram (micros): Count: 663 Average: 487.4419 StdDev: 829.17 Min: 76 Median: 168.9157 Max: 6437 Percentiles: P50: 168.92 P75: 412.35 P99: 4344.41 P99.9: 6356.90 P99.99: 6437.00 ------------------------------------------------------ ( 51, 76 ] 1 0.151% 0.151% ( 76, 110 ] 86 12.971% 13.122% ### ( 110, 170 ] 249 37.557% 50.679% ######## ( 170, 250 ] 101 15.234% 65.913% ### ( 250, 380 ] 52 7.843% 73.756% ## ( 380, 580 ] 51 7.692% 81.448% ## ( 580, 870 ] 34 5.128% 86.576% # ( 870, 1300 ] 28 4.223% 90.799% # ( 1300, 1900 ] 21 3.167% 93.967% # ( 1900, 2900 ] 17 2.564% 96.531% # ( 2900, 4400 ] 17 2.564% 99.095% # ( 4400, 6600 ] 6 0.905% 100.000% ** Level 2 read latency histogram (micros): Count: 91270 Average: 12.0961 StdDev: 157.50 Min: 1 Median: 1.9858 Max: 9485 Percentiles: P50: 1.99 P75: 2.98 P99: 9.98 P99.9: 3023.11 P99.99: 5345.04 ------------------------------------------------------ [ 0, 1 ] 1501 1.645% 1.645% ( 1, 2 ] 44771 49.053% 50.698% ########## ( 2, 3 ] 22613 24.776% 75.474% ##### ( 3, 4 ] 9305 10.195% 85.669% ## ( 4, 6 ] 9659 10.583% 96.252% ## ( 6, 10 ] 2524 2.765% 99.017% # ( 10, 15 ] 355 0.389% 99.406% ( 15, 22 ] 39 0.043% 99.449% ( 22, 34 ] 7 0.008% 99.457% ( 34, 51 ] 2 0.002% 99.459% ( 51, 76 ] 7 0.008% 99.466% ( 76, 110 ] 1 0.001% 99.468% ( 110, 170 ] 3 0.003% 99.471% ( 170, 250 ] 9 0.010% 99.481% ( 250, 380 ] 14 0.015% 99.496% ( 380, 580 ] 58 0.064% 99.560% ( 580, 870 ] 82 0.090% 99.649% ( 870, 1300 ] 103 0.113% 99.762% ( 1300, 1900 ] 60 0.066% 99.828% ( 1900, 2900 ] 59 0.065% 99.893% ( 2900, 4400 ] 82 0.090% 99.982% ( 4400, 6600 ] 16 0.018% 100.000% ( 6600, 9900 ] 2 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 891590 Average: 9.6026 StdDev: 153.18 Min: 1 Median: 1.9857 Max: 38754 Percentiles: P50: 1.99 P75: 2.96 P99: 9.66 P99.9: 757.82 P99.99: 1189.93 ------------------------------------------------------ [ 0, 1 ] 15967 1.791% 1.791% ( 1, 2 ] 436046 48.907% 50.697% ########## ( 2, 3 ] 225078 25.245% 75.942% ##### ( 3, 4 ] 90219 10.119% 86.061% ## ( 4, 6 ] 94900 10.644% 96.705% ## ( 6, 10 ] 22373 2.509% 99.214% # ( 10, 15 ] 3614 0.405% 99.619% ( 15, 22 ] 509 0.057% 99.677% ( 22, 34 ] 69 0.008% 99.684% ( 34, 51 ] 49 0.005% 99.690% ( 51, 76 ] 60 0.007% 99.696% ( 76, 110 ] 91 0.010% 99.707% ( 110, 170 ] 388 0.044% 99.750% ( 170, 250 ] 225 0.025% 99.775% ( 250, 380 ] 271 0.030% 99.806% ( 380, 580 ] 439 0.049% 99.855% ( 580, 870 ] 653 0.073% 99.928% ( 870, 1300 ] 739 0.083% 100.011% ( 1300, 1900 ] 500 0.056% 100.067% ( 1900, 2900 ] 441 0.049% 100.117% ( 2900, 4400 ] 419 0.047% 100.164% ( 4400, 6600 ] 102 0.011% 100.175% ( 6600, 9900 ] 26 0.003% 100.178% ( 9900, 14000 ] 1 0.000% 100.178% ( 14000, 22000 ] 6 0.001% 100.179% ( 22000, 33000 ] 2 0.000% 100.179% ( 33000, 50000 ] 2 0.000% 100.179% ** Level 4 read latency histogram (micros): Count: 741962 Average: 19.2153 StdDev: 234.15 Min: 1 Median: 2.0007 Max: 56432 Percentiles: P50: 2.00 P75: 3.02 P99: 242.60 P99.9: 1733.23 P99.99: 2466.03 ------------------------------------------------------ [ 0, 1 ] 13969 1.883% 1.883% ( 1, 2 ] 356877 48.099% 49.982% ########## ( 2, 3 ] 184100 24.813% 74.794% ##### ( 3, 4 ] 73811 9.948% 84.742% ## ( 4, 6 ] 76287 10.282% 95.024% ## ( 6, 10 ] 19766 2.664% 97.688% # ( 10, 15 ] 3302 0.445% 98.133% ( 15, 22 ] 414 0.056% 98.189% ( 22, 34 ] 58 0.008% 98.197% ( 34, 51 ] 61 0.008% 98.205% ( 51, 76 ] 71 0.010% 98.215% ( 76, 110 ] 1004 0.135% 98.350% ( 110, 170 ] 3404 0.459% 98.809% ( 170, 250 ] 1563 0.211% 99.019% ( 250, 380 ] 1017 0.137% 99.157% ( 380, 580 ] 1475 0.199% 99.355% ( 580, 870 ] 1648 0.222% 99.577% ( 870, 1300 ] 1645 0.222% 99.799% ( 1300, 1900 ] 1036 0.140% 99.939% ( 1900, 2900 ] 671 0.090% 100.029% ( 2900, 4400 ] 685 0.092% 100.122% ( 4400, 6600 ] 197 0.027% 100.148% ( 6600, 9900 ] 25 0.003% 100.151% ( 9900, 14000 ] 3 0.000% 100.152% ( 14000, 22000 ] 8 0.001% 100.153% ( 33000, 50000 ] 4 0.001% 100.154% ( 50000, 75000 ] 2 0.000% 100.154% ** Level 5 read latency histogram (micros): Count: 61820 Average: 541.4529 StdDev: 1036.57 Min: 76 Median: 179.1056 Max: 61948 Percentiles: P50: 179.11 P75: 485.59 P99: 5067.08 P99.9: 8428.51 P99.99: 16908.80 ------------------------------------------------------ ( 51, 76 ] 4 0.006% 0.006% ( 76, 110 ] 6882 11.132% 11.139% ## ( 110, 170 ] 22955 37.132% 48.271% ####### ( 170, 250 ] 9392 15.192% 63.463% ### ( 250, 380 ] 4574 7.399% 70.862% # ( 380, 580 ] 4845 7.837% 78.699% ## ( 580, 870 ] 3907 6.320% 85.019% # ( 870, 1300 ] 3189 5.159% 90.178% # ( 1300, 1900 ] 1963 3.175% 93.353% # ( 1900, 2900 ] 1542 2.494% 95.848% ( 2900, 4400 ] 1732 2.802% 98.649% # ( 4400, 6600 ] 715 1.157% 99.806% ( 6600, 9900 ] 105 0.170% 99.976% ( 9900, 14000 ] 7 0.011% 99.987% ( 14000, 22000 ] 5 0.008% 99.995% ( 22000, 33000 ] 3 0.005% 100.000% ( 33000, 50000 ] 2 0.003% 100.003% ( 50000, 75000 ] 2 0.003% 100.006% ** DB Stats ** Uptime(secs): 12.5 total, 3.9 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 157.43 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.741 H:M:S, 37.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 254.66 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.394 H:M:S, 10.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/19-19:24:57 ... thread 0: (1000000,3000000) ops and (94383.5,131084.6) ops/second in (10.595071,22.885978) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/12 4.02 GB 257.0 3.7 0.0 3.7 6.4 2.8 0.0 2.3 129.0 225.4 29.28 22.01 50 0.586 3836K 12K 0.0 0.0 L1 8/8 837.83 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 14/0 924.68 MB 0.9 1.3 0.8 0.5 1.3 0.8 0.0 1.6 110.3 108.7 11.89 6.03 5 2.378 415K 0 0.0 0.0 L4 185/0 9.67 GB 1.0 6.7 3.7 3.0 6.6 3.6 0.0 1.8 114.1 111.9 60.34 32.25 39 1.547 1494K 0 0.0 0.0 L5 1140/0 93.87 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1366/20 109.28 GB 0.0 11.7 4.5 7.2 14.3 7.1 3.6 5.2 118.0 144.3 101.52 60.29 94 1.080 5746K 12K 0.0 0.0 Int 0/0 0.00 KB 0.0 11.7 4.5 7.2 14.3 7.1 3.6 5.2 118.0 144.3 101.52 60.29 94 1.080 5746K 12K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.7 4.5 7.2 11.5 4.4 0.0 0.0 136.4 134.5 87.79 52.33 53 1.656 5746K 12K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 206.5 13.73 7.96 41 0.335 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 23.1 total, 22.9 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.30 GB write, 634.08 MB/s write, 11.70 GB read, 518.52 MB/s read, 101.5 seconds Interval compaction: 14.30 GB write, 639.88 MB/s write, 11.70 GB read, 523.26 MB/s read, 101.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 11, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 11, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 27, pending-compaction-bytes-delays: 11, pending-compaction-bytes-stops: 0, total-delays: 22, total-stops: 27, interval: 49 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1304760 Average: 20.5748 StdDev: 132.86 Min: 0 Median: 1.5388 Max: 56552 Percentiles: P50: 1.54 P75: 2.11 P99: 289.93 P99.9: 1167.64 P99.99: 2316.18 ------------------------------------------------------ [ 0, 1 ] 287750 22.054% 22.054% #### ( 1, 2 ] 676692 51.863% 73.917% ########## ( 2, 3 ] 131390 10.070% 83.987% ## ( 3, 4 ] 49760 3.814% 87.801% # ( 4, 6 ] 40756 3.124% 90.925% # ( 6, 10 ] 9731 0.746% 91.670% ( 10, 15 ] 1641 0.126% 91.796% ( 15, 22 ] 222 0.017% 91.813% ( 22, 34 ] 27 0.002% 91.815% ( 34, 51 ] 15 0.001% 91.816% ( 51, 76 ] 22 0.002% 91.818% ( 76, 110 ] 15757 1.208% 93.026% ( 110, 170 ] 57510 4.408% 97.433% # ( 170, 250 ] 18665 1.431% 98.864% ( 250, 380 ] 5780 0.443% 99.307% ( 380, 580 ] 4773 0.366% 99.673% ( 580, 870 ] 2172 0.166% 99.839% ( 870, 1300 ] 1146 0.088% 99.927% ( 1300, 1900 ] 628 0.048% 99.975% ( 1900, 2900 ] 465 0.036% 100.011% ( 2900, 4400 ] 398 0.031% 100.041% ( 4400, 6600 ] 122 0.009% 100.051% ( 6600, 9900 ] 9 0.001% 100.051% ( 9900, 14000 ] 1 0.000% 100.052% ( 14000, 22000 ] 1 0.000% 100.052% ( 22000, 33000 ] 1 0.000% 100.052% ( 50000, 75000 ] 1 0.000% 100.052% ** Level 1 read latency histogram (micros): Count: 2367 Average: 257.1132 StdDev: 469.81 Min: 75 Median: 147.3710 Max: 6437 Percentiles: P50: 147.37 P75: 203.12 P99: 2860.59 P99.9: 5732.10 P99.99: 6437.00 ------------------------------------------------------ ( 51, 76 ] 3 0.127% 0.127% ( 76, 110 ] 420 17.744% 17.871% #### ( 110, 170 ] 1221 51.584% 69.455% ########## ( 170, 250 ] 317 13.392% 82.847% ### ( 250, 380 ] 139 5.872% 88.720% # ( 380, 580 ] 125 5.281% 94.001% # ( 580, 870 ] 50 2.112% 96.113% ( 870, 1300 ] 31 1.310% 97.423% ( 1300, 1900 ] 21 0.887% 98.310% ( 1900, 2900 ] 17 0.718% 99.028% ( 2900, 4400 ] 17 0.718% 99.747% ( 4400, 6600 ] 6 0.253% 100.000% ** Level 2 read latency histogram (micros): Count: 91270 Average: 12.0961 StdDev: 157.50 Min: 1 Median: 1.9858 Max: 9485 Percentiles: P50: 1.99 P75: 2.98 P99: 9.98 P99.9: 3023.11 P99.99: 5345.04 ------------------------------------------------------ [ 0, 1 ] 1501 1.645% 1.645% ( 1, 2 ] 44771 49.053% 50.698% ########## ( 2, 3 ] 22613 24.776% 75.474% ##### ( 3, 4 ] 9305 10.195% 85.669% ## ( 4, 6 ] 9659 10.583% 96.252% ## ( 6, 10 ] 2524 2.765% 99.017% # ( 10, 15 ] 355 0.389% 99.406% ( 15, 22 ] 39 0.043% 99.449% ( 22, 34 ] 7 0.008% 99.457% ( 34, 51 ] 2 0.002% 99.459% ( 51, 76 ] 7 0.008% 99.466% ( 76, 110 ] 1 0.001% 99.468% ( 110, 170 ] 3 0.003% 99.471% ( 170, 250 ] 9 0.010% 99.481% ( 250, 380 ] 14 0.015% 99.496% ( 380, 580 ] 58 0.064% 99.560% ( 580, 870 ] 82 0.090% 99.649% ( 870, 1300 ] 103 0.113% 99.762% ( 1300, 1900 ] 60 0.066% 99.828% ( 1900, 2900 ] 59 0.065% 99.893% ( 2900, 4400 ] 82 0.090% 99.982% ( 4400, 6600 ] 16 0.018% 100.000% ( 6600, 9900 ] 2 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 893758 Average: 9.9821 StdDev: 153.26 Min: 1 Median: 1.9882 Max: 38754 Percentiles: P50: 1.99 P75: 2.97 P99: 10.32 P99.9: 761.35 P99.99: 1190.69 ------------------------------------------------------ [ 0, 1 ] 15967 1.787% 1.787% ( 1, 2 ] 436046 48.788% 50.574% ########## ( 2, 3 ] 225081 25.184% 75.758% ##### ( 3, 4 ] 90219 10.094% 85.852% ## ( 4, 6 ] 94902 10.618% 96.471% ## ( 6, 10 ] 22373 2.503% 98.974% # ( 10, 15 ] 3614 0.404% 99.378% ( 15, 22 ] 509 0.057% 99.435% ( 22, 34 ] 69 0.008% 99.443% ( 34, 51 ] 49 0.005% 99.449% ( 51, 76 ] 60 0.007% 99.455% ( 76, 110 ] 440 0.049% 99.504% ( 110, 170 ] 1675 0.187% 99.692% ( 170, 250 ] 557 0.062% 99.754% ( 250, 380 ] 369 0.041% 99.795% ( 380, 580 ] 514 0.058% 99.853% ( 580, 870 ] 672 0.075% 99.928% ( 870, 1300 ] 741 0.083% 100.011% ( 1300, 1900 ] 501 0.056% 100.067% ( 1900, 2900 ] 441 0.049% 100.116% ( 2900, 4400 ] 419 0.047% 100.163% ( 4400, 6600 ] 102 0.011% 100.175% ( 6600, 9900 ] 26 0.003% 100.178% ( 9900, 14000 ] 1 0.000% 100.178% ( 14000, 22000 ] 6 0.001% 100.178% ( 22000, 33000 ] 2 0.000% 100.179% ( 33000, 50000 ] 2 0.000% 100.179% ** Level 4 read latency histogram (micros): Count: 767189 Average: 23.5106 StdDev: 232.24 Min: 1 Median: 2.0678 Max: 56432 Percentiles: P50: 2.07 P75: 3.26 P99: 351.28 P99.9: 1721.79 P99.99: 2464.86 ------------------------------------------------------ [ 0, 1 ] 13969 1.821% 1.821% ( 1, 2 ] 357104 46.547% 48.368% ######### ( 2, 3 ] 184692 24.074% 72.442% ##### ( 3, 4 ] 74409 9.699% 82.141% ## ( 4, 6 ] 77193 10.062% 92.202% ## ( 6, 10 ] 19844 2.587% 94.789% # ( 10, 15 ] 3328 0.434% 95.223% ( 15, 22 ] 415 0.054% 95.277% ( 22, 34 ] 58 0.008% 95.284% ( 34, 51 ] 61 0.008% 95.292% ( 51, 76 ] 76 0.010% 95.302% ( 76, 110 ] 5078 0.662% 95.964% ( 110, 170 ] 16419 2.140% 98.104% ( 170, 250 ] 5295 0.690% 98.795% ( 250, 380 ] 2023 0.264% 99.058% ( 380, 580 ] 2238 0.292% 99.350% ( 580, 870 ] 1816 0.237% 99.587% ( 870, 1300 ] 1672 0.218% 99.805% ( 1300, 1900 ] 1041 0.136% 99.940% ( 1900, 2900 ] 675 0.088% 100.028% ( 2900, 4400 ] 686 0.089% 100.118% ( 4400, 6600 ] 197 0.026% 100.143% ( 6600, 9900 ] 25 0.003% 100.147% ( 9900, 14000 ] 3 0.000% 100.147% ( 14000, 22000 ] 8 0.001% 100.148% ( 33000, 50000 ] 4 0.001% 100.149% ( 50000, 75000 ] 2 0.000% 100.149% ** Level 5 read latency histogram (micros): Count: 215555 Average: 273.9020 StdDev: 586.82 Min: 74 Median: 149.8220 Max: 61948 Percentiles: P50: 149.82 P75: 209.13 P99: 3255.11 P99.9: 6287.52 P99.99: 9505.40 ------------------------------------------------------ ( 51, 76 ] 20 0.009% 0.009% ( 76, 110 ] 34481 15.996% 16.006% ### ( 110, 170 ] 110406 51.219% 67.225% ########## ( 170, 250 ] 34266 15.897% 83.122% ### ( 250, 380 ] 11372 5.276% 88.397% # ( 380, 580 ] 10341 4.797% 93.195% # ( 580, 870 ] 5198 2.411% 95.606% ( 870, 1300 ] 3340 1.549% 97.156% ( 1300, 1900 ] 1993 0.925% 98.080% ( 1900, 2900 ] 1571 0.729% 98.809% ( 2900, 4400 ] 1738 0.806% 99.615% ( 4400, 6600 ] 715 0.332% 99.947% ( 6600, 9900 ] 105 0.049% 99.996% ( 9900, 14000 ] 7 0.003% 99.999% ( 14000, 22000 ] 5 0.002% 100.001% ( 22000, 33000 ] 3 0.001% 100.003% ( 33000, 50000 ] 2 0.001% 100.004% ( 50000, 75000 ] 2 0.001% 100.005% ** DB Stats ** Uptime(secs): 23.1 total, 10.6 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 127.83 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:11.618 H:M:S, 50.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 92.89 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:6.876 H:M:S, 64.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 232.440 micros/op 17185 ops/sec 60.168 seconds 1033996 operations; 16.7 MB/s (255999 of 255999 found) Microseconds per read: Count: 1033996 Average: 232.4421 StdDev: 671.34 Min: 2 Median: 173.7703 Max: 85886 Percentiles: P50: 173.77 P75: 231.98 P99: 888.03 P99.9: 10643.05 P99.99: 29236.51 ------------------------------------------------------ ( 1, 2 ] 8 0.001% 0.001% ( 2, 3 ] 33 0.003% 0.004% ( 3, 4 ] 43 0.004% 0.008% ( 4, 6 ] 91 0.009% 0.017% ( 6, 10 ] 302 0.029% 0.046% ( 10, 15 ] 2748 0.266% 0.312% ( 15, 22 ] 3720 0.360% 0.672% ( 22, 34 ] 11720 1.133% 1.805% ( 34, 51 ] 3087 0.299% 2.104% ( 51, 76 ] 326 0.032% 2.135% ( 76, 110 ] 18897 1.828% 3.963% ( 110, 170 ] 459281 44.418% 48.381% ######### ( 170, 250 ] 355244 34.356% 82.737% ####### ( 250, 380 ] 109595 10.599% 93.336% ## ( 380, 580 ] 44103 4.265% 97.602% # ( 580, 870 ] 14259 1.379% 98.981% ( 870, 1300 ] 4748 0.459% 99.440% ( 1300, 1900 ] 1893 0.183% 99.623% ( 1900, 2900 ] 1180 0.114% 99.737% ( 2900, 4400 ] 999 0.097% 99.834% ( 4400, 6600 ] 388 0.038% 99.871% ( 6600, 9900 ] 214 0.021% 99.892% ( 9900, 14000 ] 458 0.044% 99.936% ( 14000, 22000 ] 451 0.044% 99.980% ( 22000, 33000 ] 159 0.015% 99.995% ( 33000, 50000 ] 38 0.004% 99.999% ( 50000, 75000 ] 9 0.001% 100.000% ( 75000, 110000 ] 2 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 15143469 rocksdb.bloom.filter.full.positive COUNT : 1180956 rocksdb.bloom.filter.full.true.positive COUNT : 1033580 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 416 rocksdb.memtable.miss COUNT : 1033580 rocksdb.l0.hit COUNT : 43144 rocksdb.l1.hit COUNT : 12430 rocksdb.l2andup.hit COUNT : 978006 rocksdb.compaction.key.drop.new COUNT : 210693 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3954617 rocksdb.number.keys.read COUNT : 1033996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 4081164744 rocksdb.bytes.read COUNT : 1033996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1587 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 44373274 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 18372409568 rocksdb.compact.write.bytes COUNT : 18149056873 rocksdb.flush.write.bytes COUNT : 4029591864 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 33 rocksdb.number.superversion_acquires COUNT : 779 rocksdb.number.superversion_releases COUNT : 738 rocksdb.number.superversion_cleanups COUNT : 190 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 220 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3997034496 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1501184 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 22452595100 rocksdb.non.last.level.read.count COUNT : 5279336 rocksdb.block.checksum.compute.count COUNT : 5639781 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 171.846989 P95 : 456.453346 P99 : 886.007021 P100 : 85882.000000 COUNT : 1033996 SUM : 239332599 rocksdb.db.write.micros P50 : 1.549413 P95 : 5.103316 P99 : 14.424461 P100 : 733106.000000 COUNT : 3954617 SUM : 54691622 rocksdb.compaction.times.micros P50 : 1413043.478261 P95 : 4300000.000000 P99 : 39400000.000000 P100 : 45600381.000000 COUNT : 60 SUM : 143056285 rocksdb.compaction.times.cpu_micros P50 : 850000.000000 P95 : 1900000.000000 P99 : 15419633.000000 P100 : 15419633.000000 COUNT : 60 SUM : 77261130 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 756.718750 P95 : 113000.000000 P99 : 158600.000000 P100 : 160275.000000 COUNT : 57 SUM : 681120 rocksdb.compaction.outfile.sync.micros P50 : 43289.473684 P95 : 268055.555556 P99 : 384750.000000 P100 : 557742.000000 COUNT : 195 SUM : 15171713 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 214.444444 P95 : 5087.500000 P99 : 6517.500000 P100 : 6964.000000 COUNT : 130 SUM : 134691 rocksdb.table.open.io.micros P50 : 9053.762376 P95 : 19923.981900 P99 : 26214.375000 P100 : 67841.000000 COUNT : 1587 SUM : 14778527 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.907495 P95 : 99.447479 P99 : 217.008020 P100 : 56557.000000 COUNT : 4448014 SUM : 79003154 rocksdb.read.block.get.micros P50 : 141.200740 P95 : 329.398736 P99 : 839.257660 P100 : 62009.000000 COUNT : 1190191 SUM : 202925594 rocksdb.write.raw.block.micros P50 : 0.528465 P95 : 1.112079 P99 : 3.427478 P100 : 52283.000000 COUNT : 5371863 SUM : 15041119 rocksdb.numfiles.in.singlecompaction P50 : 3.136364 P95 : 5.740000 P99 : 25.240000 P100 : 27.000000 COUNT : 73 SUM : 292 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.567247 P95 : 1228.118845 P99 : 2628.663817 P100 : 732908.000000 COUNT : 253640 SUM : 44396506 rocksdb.sst.read.micros P50 : 2.105967 P95 : 177.125832 P99 : 462.684599 P100 : 61948.000000 COUNT : 5279486 SUM : 265329215 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1033996 SUM : 1033996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3954617 SUM : 4081164744 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 219565.217391 P95 : 877000.000000 P99 : 1036304.000000 P100 : 1036304.000000 COUNT : 57 SUM : 16912744 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 223 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0