[2024-11-18 16:00:17.885164] Starting SPDK v24.01.1-pre git sha1 c13c99a5e / DPDK 23.11.0 initialization... [2024-11-18 16:00:17.885268] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid970838 ] Set seed to 1731942017882714 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-18 16:00:17.986106] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-18 16:00:18.025492] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-18 16:00:18.025541] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 970838' to capture a snapshot of events at runtime. [2024-11-18 16:00:18.025553] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid970838 for offline analysis/debug. [2024-11-18 16:00:18.025576] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Nov 18 16:00:22 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/18-16:00:26 ... thread 0: (1000000,1000000) ops and (236639.8,236639.8) ops/second in (4.225831,4.225831) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/12 2.63 GB 88.1 0.7 0.0 0.7 1.6 0.9 0.0 1.9 126.2 273.3 5.99 5.27 17 0.352 767K 758 0.0 0.0 L1 5/5 487.85 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 25.34 MB 0.3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 13/0 921.09 MB 0.9 0.6 0.3 0.4 0.6 0.2 0.0 2.5 182.9 181.8 3.52 2.99 2 1.761 261K 0 0.0 0.0 L4 153/0 9.61 GB 1.0 0.4 0.3 0.1 0.4 0.3 0.2 1.2 202.3 201.2 1.89 1.69 3 0.630 330K 0 0.0 0.0 L5 1116/0 91.84 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1305/17 105.49 GB 0.0 1.7 0.6 1.2 2.6 1.4 0.5 3.0 156.3 233.1 11.40 9.94 22 0.518 1359K 758 0.0 0.0 Int 0/0 0.00 KB 0.0 1.7 0.6 1.2 2.6 1.4 0.5 3.0 156.3 233.1 11.40 9.94 22 0.518 1359K 758 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.7 0.6 1.2 1.7 0.6 0.0 0.0 212.8 212.0 8.38 7.55 8 1.047 1359K 758 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 291.5 3.03 2.39 14 0.216 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 4.4 total, 4.2 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 2.60 GB write, 602.02 MB/s write, 1.74 GB read, 403.75 MB/s read, 11.4 seconds Interval compaction: 2.60 GB write, 628.57 MB/s write, 1.74 GB read, 421.56 MB/s read, 11.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 4, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 4, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 0, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 4, total-stops: 0, interval: 4 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 341456 Average: 9.7039 StdDev: 73.39 Min: 1 Median: 1.5852 Max: 7217 Percentiles: P50: 1.59 P75: 2.15 P99: 196.05 P99.9: 684.30 P99.99: 1030.17 ------------------------------------------------------ [ 0, 1 ] 59391 17.393% 17.393% ### ( 1, 2 ] 190240 55.714% 73.108% ########### ( 2, 3 ] 42485 12.442% 85.550% ## ( 3, 4 ] 18444 5.402% 90.952% # ( 4, 6 ] 15305 4.482% 95.434% # ( 6, 10 ] 4882 1.430% 96.864% ( 10, 15 ] 806 0.236% 97.100% ( 15, 22 ] 181 0.053% 97.153% ( 22, 34 ] 65 0.019% 97.172% ( 34, 51 ] 25 0.007% 97.179% ( 51, 76 ] 22 0.006% 97.186% ( 76, 110 ] 1260 0.369% 97.555% ( 110, 170 ] 4336 1.270% 98.824% ( 170, 250 ] 1841 0.539% 99.364% ( 250, 380 ] 914 0.268% 99.631% ( 380, 580 ] 778 0.228% 99.859% ( 580, 870 ] 388 0.114% 99.973% ( 870, 1300 ] 158 0.046% 100.019% ( 1300, 1900 ] 64 0.019% 100.038% ( 1900, 2900 ] 49 0.014% 100.052% ( 2900, 4400 ] 24 0.007% 100.059% ( 4400, 6600 ] 12 0.004% 100.063% ( 6600, 9900 ] 1 0.000% 100.063% ** Level 1 read latency histogram (micros): Count: 207 Average: 459.2705 StdDev: 932.01 Min: 83 Median: 164.1379 Max: 5991 Percentiles: P50: 164.14 P75: 295.50 P99: 5689.20 P99.9: 5991.00 P99.99: 5991.00 ------------------------------------------------------ ( 76, 110 ] 25 12.077% 12.077% ## ( 110, 170 ] 87 42.029% 54.106% ######## ( 170, 250 ] 38 18.357% 72.464% #### ( 250, 380 ] 15 7.246% 79.710% # ( 380, 580 ] 11 5.314% 85.024% # ( 580, 870 ] 11 5.314% 90.338% # ( 870, 1300 ] 4 1.932% 92.271% ( 1300, 1900 ] 7 3.382% 95.652% # ( 1900, 2900 ] 1 0.483% 96.135% ( 2900, 4400 ] 3 1.449% 97.585% ( 4400, 6600 ] 5 2.415% 100.000% ** Level 2 read latency histogram (micros): Count: 14 Average: 518.8571 StdDev: 387.22 Min: 109 Median: 446.6667 Max: 1381 Percentiles: P50: 446.67 P75: 725.00 P99: 1381.00 P99.9: 1381.00 P99.99: 1381.00 ------------------------------------------------------ ( 76, 110 ] 1 7.143% 7.143% # ( 110, 170 ] 3 21.429% 28.571% #### ( 250, 380 ] 2 14.286% 42.857% ### ( 380, 580 ] 3 21.429% 64.286% #### ( 580, 870 ] 3 21.429% 85.714% #### ( 870, 1300 ] 1 7.143% 92.857% # ( 1300, 1900 ] 1 7.143% 100.000% # ** Level 3 read latency histogram (micros): Count: 115629 Average: 4.3877 StdDev: 41.64 Min: 1 Median: 1.7076 Max: 5731 Percentiles: P50: 1.71 P75: 2.52 P99: 9.11 P99.9: 353.61 P99.99: 712.79 ------------------------------------------------------ [ 0, 1 ] 10348 8.949% 8.949% ## ( 1, 2 ] 67083 58.016% 66.965% ############ ( 2, 3 ] 18016 15.581% 82.546% ### ( 3, 4 ] 9251 8.001% 90.546% ## ( 4, 6 ] 8447 7.305% 97.852% # ( 6, 10 ] 1708 1.477% 99.329% ( 10, 15 ] 218 0.189% 99.517% ( 15, 22 ] 55 0.048% 99.565% ( 22, 34 ] 23 0.020% 99.585% ( 34, 51 ] 6 0.005% 99.590% ( 51, 76 ] 5 0.004% 99.594% ( 76, 110 ] 35 0.030% 99.625% ( 110, 170 ] 167 0.144% 99.769% ( 170, 250 ] 90 0.078% 99.847% ( 250, 380 ] 77 0.067% 99.914% ( 380, 580 ] 66 0.057% 99.971% ( 580, 870 ] 49 0.042% 100.013% ( 870, 1300 ] 30 0.026% 100.039% ( 1300, 1900 ] 11 0.010% 100.048% ( 1900, 2900 ] 7 0.006% 100.054% ( 2900, 4400 ] 1 0.001% 100.055% ( 4400, 6600 ] 1 0.001% 100.056% ** Level 4 read latency histogram (micros): Count: 17846 Average: 79.5482 StdDev: 291.84 Min: 1 Median: 2.3852 Max: 5180 Percentiles: P50: 2.39 P75: 5.51 P99: 1405.11 P99.9: 3857.70 P99.99: 5180.00 ------------------------------------------------------ [ 0, 1 ] 1228 6.881% 6.881% # ( 1, 2 ] 6640 37.207% 44.088% ####### ( 2, 3 ] 2739 15.348% 59.436% ### ( 3, 4 ] 1767 9.901% 69.338% ## ( 4, 6 ] 1340 7.509% 76.846% ## ( 6, 10 ] 493 2.763% 79.609% # ( 10, 15 ] 111 0.622% 80.231% ( 15, 22 ] 25 0.140% 80.371% ( 22, 34 ] 9 0.050% 80.421% ( 34, 51 ] 2 0.011% 80.433% ( 51, 76 ] 1 0.006% 80.438% ( 76, 110 ] 376 2.107% 82.545% ( 110, 170 ] 1206 6.758% 89.303% # ( 170, 250 ] 585 3.278% 92.581% # ( 250, 380 ] 404 2.264% 94.845% ( 380, 580 ] 363 2.034% 96.879% ( 580, 870 ] 214 1.199% 98.078% ( 870, 1300 ] 150 0.841% 98.919% ( 1300, 1900 ] 83 0.465% 99.384% ( 1900, 2900 ] 73 0.409% 99.793% ( 2900, 4400 ] 30 0.168% 99.961% ( 4400, 6600 ] 8 0.045% 100.006% ** Level 5 read latency histogram (micros): Count: 26668 Average: 432.6927 StdDev: 702.59 Min: 75 Median: 179.1720 Max: 7621 Percentiles: P50: 179.17 P75: 372.52 P99: 3947.92 P99.9: 6081.85 P99.99: 7621.00 ------------------------------------------------------ ( 51, 76 ] 1 0.004% 0.004% ( 76, 110 ] 2752 10.319% 10.323% ## ( 110, 170 ] 10059 37.719% 48.043% ######## ( 170, 250 ] 4553 17.073% 65.115% ### ( 250, 380 ] 2797 10.488% 75.604% ## ( 380, 580 ] 2322 8.707% 84.311% ## ( 580, 870 ] 1396 5.235% 89.546% # ( 870, 1300 ] 812 3.045% 92.590% # ( 1300, 1900 ] 572 2.145% 94.735% ( 1900, 2900 ] 744 2.790% 97.525% # ( 2900, 4400 ] 563 2.111% 99.636% ( 4400, 6600 ] 92 0.345% 99.981% ( 6600, 9900 ] 5 0.019% 100.000% ** DB Stats ** Uptime(secs): 4.4 total, 4.2 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 222.93 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.609 H:M:S, 13.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 232.74 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.609 H:M:S, 14.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/18-16:00:34 ... thread 0: (1000000,2000000) ops and (125263.3,163813.4) ops/second in (7.983185,12.209016) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/16 3.61 GB 106.5 2.8 0.0 2.8 4.6 1.8 0.0 2.5 169.5 282.1 16.72 15.48 38 0.440 2877K 6375 0.0 0.0 L1 5/5 487.85 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 25.34 MB 0.3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 13/0 921.09 MB 0.9 0.6 0.3 0.4 0.6 0.2 0.0 2.5 182.9 181.8 3.52 2.99 2 1.761 261K 0 0.0 0.0 L4 153/0 9.61 GB 1.0 0.4 0.3 0.1 0.4 0.3 0.2 1.2 202.3 201.2 1.89 1.69 3 0.630 330K 0 0.0 0.0 L5 1116/0 91.84 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1305/21 106.47 GB 0.0 3.8 0.6 3.2 5.6 2.4 0.5 3.0 174.4 259.2 22.14 20.15 43 0.515 3469K 6375 0.0 0.0 Int 0/0 0.00 KB 0.0 3.8 0.6 3.2 5.6 2.4 0.5 3.0 174.4 259.2 22.14 20.15 43 0.515 3469K 6375 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.8 0.6 3.2 3.8 0.6 0.0 0.0 240.5 239.7 16.05 15.12 13 1.235 3469K 6375 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 310.7 6.08 5.04 30 0.203 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.4 total, 12.2 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.60 GB write, 462.82 MB/s write, 3.77 GB read, 311.38 MB/s read, 22.1 seconds Interval compaction: 5.60 GB write, 469.89 MB/s write, 3.77 GB read, 316.13 MB/s read, 22.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 12, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 12, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 0, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 12, total-stops: 0, interval: 12 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 939729 Average: 15.7635 StdDev: 69.13 Min: 0 Median: 1.4575 Max: 7217 Percentiles: P50: 1.46 P75: 1.93 P99: 242.07 P99.9: 570.95 P99.99: 777.47 ------------------------------------------------------ [ 0, 1 ] 244085 25.974% 25.974% ##### ( 1, 2 ] 493483 52.513% 78.487% ########### ( 2, 3 ] 72887 7.756% 86.243% ## ( 3, 4 ] 30605 3.257% 89.500% # ( 4, 6 ] 21616 2.300% 91.801% ( 6, 10 ] 6205 0.660% 92.461% ( 10, 15 ] 1121 0.119% 92.580% ( 15, 22 ] 230 0.024% 92.605% ( 22, 34 ] 68 0.007% 92.612% ( 34, 51 ] 26 0.003% 92.615% ( 51, 76 ] 26 0.003% 92.617% ( 76, 110 ] 10838 1.153% 93.771% ( 110, 170 ] 37740 4.016% 97.787% # ( 170, 250 ] 12657 1.347% 99.134% ( 250, 380 ] 4168 0.444% 99.577% ( 380, 580 ] 3178 0.338% 99.915% ( 580, 870 ] 1031 0.110% 100.025% ( 870, 1300 ] 234 0.025% 100.050% ( 1300, 1900 ] 86 0.009% 100.059% ( 1900, 2900 ] 91 0.010% 100.069% ( 2900, 4400 ] 40 0.004% 100.073% ( 4400, 6600 ] 12 0.001% 100.074% ( 6600, 9900 ] 1 0.000% 100.074% ** Level 1 read latency histogram (micros): Count: 1176 Average: 225.3121 StdDev: 418.49 Min: 83 Median: 150.5380 Max: 5991 Percentiles: P50: 150.54 P75: 203.12 P99: 1749.14 P99.9: 5991.00 P99.99: 5991.00 ------------------------------------------------------ ( 76, 110 ] 161 13.690% 13.690% ### ( 110, 170 ] 632 53.741% 67.432% ########### ( 170, 250 ] 215 18.282% 85.714% #### ( 250, 380 ] 72 6.122% 91.837% # ( 380, 580 ] 54 4.592% 96.429% # ( 580, 870 ] 21 1.786% 98.214% ( 870, 1300 ] 4 0.340% 98.554% ( 1300, 1900 ] 7 0.595% 99.150% ( 1900, 2900 ] 2 0.170% 99.320% ( 2900, 4400 ] 3 0.255% 99.575% ( 4400, 6600 ] 5 0.425% 100.000% ** Level 2 read latency histogram (micros): Count: 47 Average: 269.9574 StdDev: 272.58 Min: 97 Median: 163.4783 Max: 1381 Percentiles: P50: 163.48 P75: 244.00 P99: 1381.00 P99.9: 1381.00 P99.99: 1381.00 ------------------------------------------------------ ( 76, 110 ] 3 6.383% 6.383% # ( 110, 170 ] 23 48.936% 55.319% ########## ( 170, 250 ] 10 21.277% 76.596% #### ( 250, 380 ] 2 4.255% 80.851% # ( 380, 580 ] 4 8.511% 89.362% ## ( 580, 870 ] 3 6.383% 95.745% # ( 870, 1300 ] 1 2.128% 97.872% ( 1300, 1900 ] 1 2.128% 100.000% ** Level 3 read latency histogram (micros): Count: 117784 Average: 6.6457 StdDev: 46.89 Min: 1 Median: 1.7230 Max: 5731 Percentiles: P50: 1.72 P75: 2.60 P99: 144.53 P99.9: 466.28 P99.99: 749.02 ------------------------------------------------------ [ 0, 1 ] 10350 8.787% 8.787% ## ( 1, 2 ] 67136 56.999% 65.787% ########### ( 2, 3 ] 18185 15.439% 81.226% ### ( 3, 4 ] 9366 7.952% 89.178% ## ( 4, 6 ] 8626 7.324% 96.501% # ( 6, 10 ] 1730 1.469% 97.970% ( 10, 15 ] 226 0.192% 98.162% ( 15, 22 ] 57 0.048% 98.210% ( 22, 34 ] 23 0.020% 98.230% ( 34, 51 ] 6 0.005% 98.235% ( 51, 76 ] 5 0.004% 98.239% ( 76, 110 ] 259 0.220% 98.459% ( 110, 170 ] 1107 0.940% 99.399% ( 170, 250 ] 372 0.316% 99.715% ( 250, 380 ] 163 0.138% 99.853% ( 380, 580 ] 128 0.109% 99.962% ( 580, 870 ] 57 0.048% 100.010% ( 870, 1300 ] 32 0.027% 100.037% ( 1300, 1900 ] 12 0.010% 100.048% ( 1900, 2900 ] 7 0.006% 100.053% ( 2900, 4400 ] 1 0.001% 100.054% ( 4400, 6600 ] 1 0.001% 100.055% ** Level 4 read latency histogram (micros): Count: 31772 Average: 116.8322 StdDev: 236.32 Min: 1 Median: 87.5579 Max: 5180 Percentiles: P50: 87.56 P75: 153.59 P99: 993.97 P99.9: 3104.58 P99.99: 4901.27 ------------------------------------------------------ [ 0, 1 ] 1228 3.865% 3.865% # ( 1, 2 ] 6697 21.078% 24.943% #### ( 2, 3 ] 2933 9.231% 34.175% ## ( 3, 4 ] 1899 5.977% 40.152% # ( 4, 6 ] 1562 4.916% 45.068% # ( 6, 10 ] 531 1.671% 46.739% ( 10, 15 ] 122 0.384% 47.123% ( 15, 22 ] 27 0.085% 47.208% ( 22, 34 ] 9 0.028% 47.237% ( 34, 51 ] 2 0.006% 47.243% ( 51, 76 ] 1 0.003% 47.246% ( 76, 110 ] 2574 8.101% 55.347% ## ( 110, 170 ] 8594 27.049% 82.396% ##### ( 170, 250 ] 2890 9.096% 91.493% ## ( 250, 380 ] 1088 3.424% 94.917% # ( 380, 580 ] 892 2.808% 97.724% # ( 580, 870 ] 358 1.127% 98.851% ( 870, 1300 ] 164 0.516% 99.367% ( 1300, 1900 ] 84 0.264% 99.632% ( 1900, 2900 ] 81 0.255% 99.887% ( 2900, 4400 ] 31 0.098% 99.984% ( 4400, 6600 ] 8 0.025% 100.009% ** Level 5 read latency histogram (micros): Count: 142128 Average: 221.0266 StdDev: 335.96 Min: 73 Median: 149.7325 Max: 7621 Percentiles: P50: 149.73 P75: 203.88 P99: 1915.20 P99.9: 4265.69 P99.99: 6236.22 ------------------------------------------------------ ( 51, 76 ] 12 0.008% 0.008% ( 76, 110 ] 21781 15.325% 15.333% ### ( 110, 170 ] 74404 52.350% 67.683% ########## ( 170, 250 ] 24555 17.277% 84.960% ### ( 250, 380 ] 8890 6.255% 91.215% # ( 380, 580 ] 6955 4.893% 96.108% # ( 580, 870 ] 2544 1.790% 97.898% ( 870, 1300 ] 949 0.668% 98.566% ( 1300, 1900 ] 605 0.426% 98.992% ( 1900, 2900 ] 771 0.542% 99.534% ( 2900, 4400 ] 571 0.402% 99.936% ( 4400, 6600 ] 92 0.065% 100.001% ( 6600, 9900 ] 5 0.004% 100.004% ** DB Stats ** Uptime(secs): 12.4 total, 8.0 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 158.77 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.971 H:M:S, 40.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 123.28 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.362 H:M:S, 54.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/18-16:00:43 ... thread 0: (1000000,3000000) ops and (108646.0,140100.3) ops/second in (9.204207,21.413223) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/12 4.59 GB 288.6 5.1 0.0 5.1 7.9 2.8 0.0 2.8 182.1 282.6 28.70 26.90 59 0.486 5306K 12K 0.0 0.0 L1 5/5 487.85 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 25.34 MB 0.3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 13/0 921.09 MB 0.9 0.6 0.3 0.4 0.6 0.2 0.0 2.5 182.9 181.8 3.52 2.99 2 1.761 261K 0 0.0 0.0 L4 153/0 9.61 GB 1.0 0.4 0.3 0.1 0.4 0.3 0.2 1.2 202.3 201.2 1.89 1.69 3 0.630 330K 0 0.0 0.0 L5 1116/0 91.84 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1306/17 107.45 GB 0.0 6.1 0.6 5.5 8.9 3.4 0.5 3.2 183.3 267.7 34.12 31.58 64 0.533 5898K 12K 0.0 0.0 Int 0/0 0.00 KB 0.0 6.1 0.6 5.5 8.9 3.4 0.5 3.2 183.3 267.7 34.12 31.58 64 0.533 5898K 12K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.1 0.6 5.5 6.1 0.6 0.0 0.0 250.8 250.0 24.94 23.87 18 1.385 5898K 12K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 315.7 9.18 7.71 46 0.200 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 21.6 total, 21.4 interval Flush(GB): cumulative 2.830, interval 2.830 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.92 GB write, 422.77 MB/s write, 6.11 GB read, 289.45 MB/s read, 34.1 seconds Interval compaction: 8.92 GB write, 426.45 MB/s write, 6.11 GB read, 291.97 MB/s read, 34.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 24, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 24, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 0, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 24, total-stops: 0, interval: 24 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1627153 Average: 17.1605 StdDev: 67.21 Min: 0 Median: 1.4305 Max: 7217 Percentiles: P50: 1.43 P75: 1.91 P99: 244.72 P99.9: 561.94 P99.99: 739.33 ------------------------------------------------------ [ 0, 1 ] 451547 27.751% 27.751% ###### ( 1, 2 ] 840946 51.682% 79.433% ########## ( 2, 3 ] 109504 6.730% 86.163% # ( 3, 4 ] 45974 2.825% 88.988% # ( 4, 6 ] 31051 1.908% 90.896% ( 6, 10 ] 8049 0.495% 91.391% ( 10, 15 ] 1408 0.087% 91.478% ( 15, 22 ] 278 0.017% 91.495% ( 22, 34 ] 73 0.004% 91.499% ( 34, 51 ] 27 0.002% 91.501% ( 51, 76 ] 40 0.002% 91.503% ( 76, 110 ] 21887 1.345% 92.848% ( 110, 170 ] 77192 4.744% 97.592% # ( 170, 250 ] 24525 1.507% 99.100% ( 250, 380 ] 7714 0.474% 99.574% ( 380, 580 ] 5838 0.359% 99.932% ( 580, 870 ] 1706 0.105% 100.037% ( 870, 1300 ] 328 0.020% 100.057% ( 1300, 1900 ] 123 0.008% 100.065% ( 1900, 2900 ] 126 0.008% 100.073% ( 2900, 4400 ] 50 0.003% 100.076% ( 4400, 6600 ] 13 0.001% 100.077% ( 6600, 9900 ] 1 0.000% 100.077% ** Level 1 read latency histogram (micros): Count: 2310 Average: 201.0411 StdDev: 307.89 Min: 79 Median: 149.8568 Max: 5991 Percentiles: P50: 149.86 P75: 199.48 P99: 861.14 P99.9: 5583.60 P99.99: 5991.00 ------------------------------------------------------ ( 76, 110 ] 320 13.853% 13.853% ### ( 110, 170 ] 1257 54.416% 68.268% ########### ( 170, 250 ] 422 18.268% 86.537% #### ( 250, 380 ] 151 6.537% 93.074% # ( 380, 580 ] 102 4.416% 97.489% # ( 580, 870 ] 36 1.558% 99.048% ( 870, 1300 ] 5 0.216% 99.264% ( 1300, 1900 ] 7 0.303% 99.567% ( 1900, 2900 ] 2 0.087% 99.654% ( 2900, 4400 ] 3 0.130% 99.784% ( 4400, 6600 ] 5 0.216% 100.000% ** Level 2 read latency histogram (micros): Count: 83 Average: 238.2048 StdDev: 223.15 Min: 97 Median: 156.7442 Max: 1381 Percentiles: P50: 156.74 P75: 230.00 P99: 1381.00 P99.9: 1381.00 P99.99: 1381.00 ------------------------------------------------------ ( 76, 110 ] 8 9.639% 9.639% ## ( 110, 170 ] 43 51.807% 61.446% ########## ( 170, 250 ] 15 18.072% 79.518% #### ( 250, 380 ] 3 3.614% 83.133% # ( 380, 580 ] 9 10.843% 93.976% ## ( 580, 870 ] 3 3.614% 97.590% # ( 870, 1300 ] 1 1.205% 98.795% ( 1300, 1900 ] 1 1.205% 100.000% ** Level 3 read latency histogram (micros): Count: 120225 Average: 9.9677 StdDev: 54.16 Min: 1 Median: 1.7412 Max: 5731 Percentiles: P50: 1.74 P75: 2.70 P99: 187.49 P99.9: 539.63 P99.99: 810.43 ------------------------------------------------------ [ 0, 1 ] 10351 8.610% 8.610% ## ( 1, 2 ] 67139 55.844% 64.454% ########### ( 2, 3 ] 18197 15.136% 79.590% ### ( 3, 4 ] 9374 7.797% 87.387% ## ( 4, 6 ] 8640 7.187% 94.574% # ( 6, 10 ] 1731 1.440% 96.013% ( 10, 15 ] 227 0.189% 96.202% ( 15, 22 ] 57 0.047% 96.250% ( 22, 34 ] 23 0.019% 96.269% ( 34, 51 ] 6 0.005% 96.274% ( 51, 76 ] 5 0.004% 96.278% ( 76, 110 ] 644 0.536% 96.813% ( 110, 170 ] 2458 2.044% 98.858% ( 170, 250 ] 781 0.650% 99.508% ( 250, 380 ] 289 0.240% 99.748% ( 380, 580 ] 229 0.190% 99.938% ( 580, 870 ] 78 0.065% 100.003% ( 870, 1300 ] 39 0.032% 100.036% ( 1300, 1900 ] 12 0.010% 100.046% ( 1900, 2900 ] 8 0.007% 100.052% ( 2900, 4400 ] 1 0.001% 100.053% ( 4400, 6600 ] 1 0.001% 100.054% ** Level 4 read latency histogram (micros): Count: 47990 Average: 133.8535 StdDev: 204.58 Min: 1 Median: 121.7875 Max: 5180 Percentiles: P50: 121.79 P75: 162.51 P99: 825.14 P99.9: 2773.68 P99.99: 4455.27 ------------------------------------------------------ [ 0, 1 ] 1228 2.559% 2.559% # ( 1, 2 ] 6722 14.007% 16.566% ### ( 2, 3 ] 3025 6.303% 22.869% # ( 3, 4 ] 1975 4.115% 26.985% # ( 4, 6 ] 1674 3.488% 30.473% # ( 6, 10 ] 548 1.142% 31.615% ( 10, 15 ] 132 0.275% 31.890% ( 15, 22 ] 28 0.058% 31.948% ( 22, 34 ] 9 0.019% 31.967% ( 34, 51 ] 2 0.004% 31.971% ( 51, 76 ] 2 0.004% 31.975% ( 76, 110 ] 5177 10.788% 42.763% ## ( 110, 170 ] 17678 36.837% 79.600% ####### ( 170, 250 ] 5523 11.509% 91.109% ## ( 250, 380 ] 1874 3.905% 95.014% # ( 380, 580 ] 1482 3.088% 98.102% # ( 580, 870 ] 510 1.063% 99.164% ( 870, 1300 ] 189 0.394% 99.558% ( 1300, 1900 ] 88 0.183% 99.742% ( 1900, 2900 ] 87 0.181% 99.923% ( 2900, 4400 ] 32 0.067% 99.990% ( 4400, 6600 ] 8 0.017% 100.006% ** Level 5 read latency histogram (micros): Count: 275043 Average: 196.4825 StdDev: 254.16 Min: 73 Median: 147.4836 Max: 7621 Percentiles: P50: 147.48 P75: 191.42 P99: 1056.10 P99.9: 3899.02 P99.99: 5727.07 ------------------------------------------------------ ( 51, 76 ] 21 0.008% 0.008% ( 76, 110 ] 43889 15.957% 15.965% ### ( 110, 170 ] 149844 54.480% 70.445% ########### ( 170, 250 ] 46799 17.015% 87.460% ### ( 250, 380 ] 15376 5.590% 93.051% # ( 380, 580 ] 12076 4.391% 97.441% # ( 580, 870 ] 3805 1.383% 98.825% ( 870, 1300 ] 1115 0.405% 99.230% ( 1300, 1900 ] 657 0.239% 99.469% ( 1900, 2900 ] 803 0.292% 99.761% ( 2900, 4400 ] 575 0.209% 99.970% ( 4400, 6600 ] 92 0.033% 100.003% ( 6600, 9900 ] 5 0.002% 100.005% ** DB Stats ** Uptime(secs): 21.6 total, 9.2 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 136.68 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:10.499 H:M:S, 48.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 106.93 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.528 H:M:S, 60.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 212.215 micros/op 18844 ops/sec 60.176 seconds 1133996 operations; 18.3 MB/s (279999 of 279999 found) Microseconds per read: Count: 1133996 Average: 212.2162 StdDev: 343.52 Min: 2 Median: 175.6329 Max: 52513 Percentiles: P50: 175.63 P75: 233.66 P99: 707.97 P99.9: 5001.15 P99.99: 13730.18 ------------------------------------------------------ ( 1, 2 ] 4 0.000% 0.000% ( 2, 3 ] 32 0.003% 0.003% ( 3, 4 ] 56 0.005% 0.008% ( 4, 6 ] 92 0.008% 0.016% ( 6, 10 ] 429 0.038% 0.054% ( 10, 15 ] 4473 0.394% 0.449% ( 15, 22 ] 3441 0.303% 0.752% ( 22, 34 ] 10915 0.963% 1.714% ( 34, 51 ] 1761 0.155% 1.870% ( 51, 76 ] 169 0.015% 1.885% ( 76, 110 ] 18490 1.631% 3.515% ( 110, 170 ] 499617 44.058% 47.573% ######### ( 170, 250 ] 390831 34.465% 82.038% ####### ( 250, 380 ] 141504 12.478% 94.517% ## ( 380, 580 ] 45613 4.022% 98.539% # ( 580, 870 ] 11850 1.045% 99.584% ( 870, 1300 ] 2425 0.214% 99.798% ( 1300, 1900 ] 415 0.037% 99.834% ( 1900, 2900 ] 379 0.033% 99.868% ( 2900, 4400 ] 269 0.024% 99.891% ( 4400, 6600 ] 355 0.031% 99.923% ( 6600, 9900 ] 544 0.048% 99.971% ( 9900, 14000 ] 234 0.021% 99.991% ( 14000, 22000 ] 67 0.006% 99.997% ( 22000, 33000 ] 28 0.002% 100.000% ( 33000, 50000 ] 2 0.000% 100.000% ( 50000, 75000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 21998179 rocksdb.bloom.filter.full.positive COUNT : 1348257 rocksdb.bloom.filter.full.true.positive COUNT : 1133577 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 419 rocksdb.memtable.miss COUNT : 1133577 rocksdb.l0.hit COUNT : 51200 rocksdb.l1.hit COUNT : 5413 rocksdb.l2andup.hit COUNT : 1076964 rocksdb.compaction.key.drop.new COUNT : 41303 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3211541 rocksdb.number.keys.read COUNT : 1133996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3314310312 rocksdb.bytes.read COUNT : 1133996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1376 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 47675439 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 9167951106 rocksdb.compact.write.bytes COUNT : 9133211060 rocksdb.flush.write.bytes COUNT : 3303053921 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 10 rocksdb.number.superversion_acquires COUNT : 487 rocksdb.number.superversion_releases COUNT : 466 rocksdb.number.superversion_cleanups COUNT : 120 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3276207104 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1091584 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 14747204592 rocksdb.non.last.level.read.count COUNT : 3419496 rocksdb.block.checksum.compute.count COUNT : 3583131 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 174.096481 P95 : 401.949190 P99 : 706.519990 P100 : 52511.000000 COUNT : 1133996 SUM : 239685427 rocksdb.db.write.micros P50 : 1.537770 P95 : 5.223487 P99 : 289.975000 P100 : 5037.000000 COUNT : 3211541 SUM : 55914994 rocksdb.compaction.times.micros P50 : 1060909.090909 P95 : 3856666.666667 P99 : 3913210.000000 P100 : 3913210.000000 COUNT : 19 SUM : 25865321 rocksdb.compaction.times.cpu_micros P50 : 1060909.090909 P95 : 3856666.666667 P99 : 3868020.000000 P100 : 3868020.000000 COUNT : 19 SUM : 24774508 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 761.250000 P95 : 1840.000000 P99 : 3012.000000 P100 : 3012.000000 COUNT : 50 SUM : 43816 rocksdb.compaction.outfile.sync.micros P50 : 1038.928571 P95 : 3225.000000 P99 : 3939.000000 P100 : 3939.000000 COUNT : 47 SUM : 65152 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 150.000000 P95 : 609.000000 P99 : 794.000000 P100 : 794.000000 COUNT : 72 SUM : 15522 rocksdb.table.open.io.micros P50 : 5661.872910 P95 : 9848.270270 P99 : 13321.762712 P100 : 16407.000000 COUNT : 1376 SUM : 7995524 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.576567 P95 : 152.848428 P99 : 234.755335 P100 : 24003.000000 COUNT : 2221878 SUM : 55778531 rocksdb.read.block.get.micros P50 : 138.874730 P95 : 245.827649 P99 : 541.281539 P100 : 23418.000000 COUNT : 1359887 SUM : 203348070 rocksdb.write.raw.block.micros P50 : 0.541387 P95 : 1.475683 P99 : 3.128386 P100 : 3868.000000 COUNT : 3011537 SUM : 6186846 rocksdb.numfiles.in.singlecompaction P50 : 3.357143 P95 : 5.800000 P99 : 17.000000 P100 : 17.000000 COUNT : 22 SUM : 93 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.583764 P95 : 2036.537571 P99 : 2727.862984 P100 : 3998.000000 COUNT : 224024 SUM : 47697366 rocksdb.sst.read.micros P50 : 3.693278 P95 : 205.098410 P99 : 404.445674 P100 : 7621.000000 COUNT : 3419528 SUM : 247381706 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1133996 SUM : 1133996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3211541 SUM : 3314310312 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 210000.000000 P95 : 246000.000000 P99 : 248444.000000 P100 : 248444.000000 COUNT : 50 SUM : 10001910 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 72 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0