[2024-11-06 07:36:53.700529] Starting SPDK v25.01-pre git sha1 f220d590c / DPDK 24.03.0 initialization... [2024-11-06 07:36:53.700794] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2267240 ] Set seed to 1730875013693850 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-06 07:36:53.825143] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-06 07:36:53.926085] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-06 07:36:53.926126] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2267240' to capture a snapshot of events at runtime. [2024-11-06 07:36:53.926136] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-06 07:36:53.926146] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-06 07:36:53.926173] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2267240 for offline analysis/debug. [2024-11-06 07:36:53.927545] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Wed Nov 6 07:37:02 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/06-07:37:12 ... thread 0: (1000000,1000000) ops and (95835.3,95835.3) ops/second in (10.434567,10.434567) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/8 2.90 GB 56.6 0.4 0.0 0.4 1.2 0.9 0.0 1.4 57.9 192.9 6.53 3.96 12 0.544 383K 519 0.0 0.0 L1 4/4 509.90 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 22/0 966.64 MB 1.0 4.1 2.9 1.2 4.1 2.9 0.0 1.4 110.8 110.1 37.93 19.90 18 2.107 1414K 487 0.0 0.0 L4 173/0 9.63 GB 1.0 4.7 3.3 1.4 4.6 3.2 0.1 1.4 118.7 117.4 40.54 22.16 36 1.126 3501K 1403 0.0 0.0 L5 1132/0 92.95 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1343/12 106.93 GB 0.0 9.2 6.2 3.0 10.0 7.0 3.4 11.6 110.5 119.9 85.00 46.02 66 1.288 5300K 2409 0.0 0.0 Int 0/0 0.00 KB 0.0 9.2 6.2 3.0 10.0 7.0 3.4 11.6 110.5 119.9 85.00 46.02 66 1.288 5300K 2409 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.2 6.2 3.0 9.1 6.1 0.0 0.0 117.3 116.3 80.07 43.55 55 1.456 5300K 2409 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 179.0 4.93 2.46 11 0.448 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 10.7 total, 10.4 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 9.96 GB write, 953.70 MB/s write, 9.17 GB read, 878.66 MB/s read, 85.0 seconds Interval compaction: 9.96 GB write, 976.94 MB/s write, 9.17 GB read, 900.07 MB/s read, 85.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 25, pending-compaction-bytes-stops: 0, total-delays: 25, total-stops: 26, interval: 50 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 496050 Average: 12.2240 StdDev: 165.39 Min: 1 Median: 2.0634 Max: 31576 Percentiles: P50: 2.06 P75: 2.95 P99: 13.63 P99.9: 2199.10 P99.99: 3830.19 ------------------------------------------------------ [ 0, 1 ] 13316 2.684% 2.684% # ( 1, 2 ] 225873 45.534% 48.219% ######### ( 2, 3 ] 139447 28.111% 76.330% ###### ( 3, 4 ] 42727 8.613% 84.944% ## ( 4, 6 ] 53723 10.830% 95.774% ## ( 6, 10 ] 14363 2.895% 98.669% # ( 10, 15 ] 2261 0.456% 99.125% ( 15, 22 ] 321 0.065% 99.190% ( 22, 34 ] 28 0.006% 99.195% ( 34, 51 ] 25 0.005% 99.200% ( 51, 76 ] 20 0.004% 99.205% ( 76, 110 ] 59 0.012% 99.216% ( 110, 170 ] 412 0.083% 99.299% ( 170, 250 ] 489 0.099% 99.398% ( 250, 380 ] 452 0.091% 99.489% ( 380, 580 ] 507 0.102% 99.591% ( 580, 870 ] 505 0.102% 99.693% ( 870, 1300 ] 495 0.100% 99.793% ( 1300, 1900 ] 415 0.084% 99.877% ( 1900, 2900 ] 391 0.079% 99.955% ( 2900, 4400 ] 278 0.056% 100.011% ( 4400, 6600 ] 74 0.015% 100.026% ( 6600, 9900 ] 12 0.002% 100.029% ( 9900, 14000 ] 2 0.000% 100.029% ( 14000, 22000 ] 1 0.000% 100.029% ( 22000, 33000 ] 3 0.001% 100.030% ** Level 1 read latency histogram (micros): Count: 146 Average: 1557.7671 StdDev: 3893.65 Min: 93 Median: 521.6667 Max: 31479 Percentiles: P50: 521.67 P75: 1262.06 P99: 24970.00 P99.9: 31479.00 P99.99: 31479.00 ------------------------------------------------------ ( 76, 110 ] 2 1.370% 1.370% ( 110, 170 ] 17 11.644% 13.014% ## ( 170, 250 ] 26 17.808% 30.822% #### ( 250, 380 ] 11 7.534% 38.356% ## ( 380, 580 ] 24 16.438% 54.795% ### ( 580, 870 ] 14 9.589% 64.384% ## ( 870, 1300 ] 17 11.644% 76.027% ## ( 1300, 1900 ] 14 9.589% 85.616% ## ( 1900, 2900 ] 7 4.795% 90.411% # ( 2900, 4400 ] 6 4.110% 94.521% # ( 4400, 6600 ] 1 0.685% 95.205% ( 6600, 9900 ] 2 1.370% 96.575% ( 9900, 14000 ] 3 2.055% 98.630% ( 22000, 33000 ] 2 1.370% 100.000% ** Level 2 read latency histogram (micros): Count: 601986 Average: 16.4423 StdDev: 212.81 Min: 1 Median: 2.2347 Max: 31928 Percentiles: P50: 2.23 P75: 3.27 P99: 12.73 P99.9: 2170.58 P99.99: 2816.33 ------------------------------------------------------ [ 0, 1 ] 6026 1.001% 1.001% ( 1, 2 ] 254309 42.245% 43.246% ######## ( 2, 3 ] 173269 28.783% 72.029% ###### ( 3, 4 ] 65377 10.860% 82.889% ## ( 4, 6 ] 75724 12.579% 95.468% ### ( 6, 10 ] 19232 3.195% 98.663% # ( 10, 15 ] 3721 0.618% 99.281% ( 15, 22 ] 564 0.094% 99.375% ( 22, 34 ] 56 0.009% 99.384% ( 34, 51 ] 21 0.003% 99.388% ( 51, 76 ] 26 0.004% 99.392% ( 76, 110 ] 24 0.004% 99.396% ( 110, 170 ] 26 0.004% 99.400% ( 170, 250 ] 59 0.010% 99.410% ( 250, 380 ] 155 0.026% 99.436% ( 380, 580 ] 243 0.040% 99.476% ( 580, 870 ] 525 0.087% 99.563% ( 870, 1300 ] 838 0.139% 99.702% ( 1300, 1900 ] 962 0.160% 99.862% ( 1900, 2900 ] 839 0.139% 100.002% ( 2900, 4400 ] 496 0.082% 100.084% ( 4400, 6600 ] 137 0.023% 100.107% ( 6600, 9900 ] 46 0.008% 100.114% ( 9900, 14000 ] 6 0.001% 100.115% ( 14000, 22000 ] 2 0.000% 100.116% ( 22000, 33000 ] 6 0.001% 100.117% ** Level 3 read latency histogram (micros): Count: 1051805 Average: 9.1789 StdDev: 132.90 Min: 1 Median: 2.1405 Max: 25145 Percentiles: P50: 2.14 P75: 3.00 P99: 9.39 P99.9: 821.11 P99.99: 1471.52 ------------------------------------------------------ [ 0, 1 ] 10061 0.957% 0.957% ( 1, 2 ] 472961 44.967% 45.923% ######### ( 2, 3 ] 305307 29.027% 74.950% ###### ( 3, 4 ] 109040 10.367% 85.317% ## ( 4, 6 ] 123459 11.738% 97.055% ## ( 6, 10 ] 24148 2.296% 99.351% ( 10, 15 ] 4002 0.380% 99.731% ( 15, 22 ] 500 0.048% 99.779% ( 22, 34 ] 45 0.004% 99.783% ( 34, 51 ] 37 0.004% 99.787% ( 51, 76 ] 55 0.005% 99.792% ( 76, 110 ] 29 0.003% 99.795% ( 110, 170 ] 66 0.006% 99.801% ( 170, 250 ] 144 0.014% 99.815% ( 250, 380 ] 177 0.017% 99.831% ( 380, 580 ] 304 0.029% 99.860% ( 580, 870 ] 503 0.048% 99.908% ( 870, 1300 ] 666 0.063% 99.971% ( 1300, 1900 ] 685 0.065% 100.037% ( 1900, 2900 ] 589 0.056% 100.093% ( 2900, 4400 ] 476 0.045% 100.138% ( 4400, 6600 ] 136 0.013% 100.151% ( 6600, 9900 ] 35 0.003% 100.154% ( 14000, 22000 ] 1 0.000% 100.154% ( 22000, 33000 ] 1 0.000% 100.154% ** Level 4 read latency histogram (micros): Count: 368210 Average: 30.3185 StdDev: 265.63 Min: 1 Median: 2.1515 Max: 23152 Percentiles: P50: 2.15 P75: 3.14 P99: 883.22 P99.9: 3385.35 P99.99: 4166.93 ------------------------------------------------------ [ 0, 1 ] 4238 1.151% 1.151% ( 1, 2 ] 164385 44.644% 45.795% ######### ( 2, 3 ] 102219 27.761% 73.556% ###### ( 3, 4 ] 37722 10.245% 83.801% ## ( 4, 6 ] 40679 11.048% 94.849% ## ( 6, 10 ] 10180 2.765% 97.614% # ( 10, 15 ] 1548 0.420% 98.034% ( 15, 22 ] 204 0.055% 98.089% ( 22, 34 ] 26 0.007% 98.096% ( 34, 51 ] 17 0.005% 98.101% ( 51, 76 ] 25 0.007% 98.108% ( 76, 110 ] 55 0.015% 98.123% ( 110, 170 ] 419 0.114% 98.237% ( 170, 250 ] 558 0.152% 98.388% ( 250, 380 ] 525 0.143% 98.531% ( 380, 580 ] 749 0.203% 98.734% ( 580, 870 ] 944 0.256% 98.991% ( 870, 1300 ] 1135 0.308% 99.299% ( 1300, 1900 ] 1043 0.283% 99.582% ( 1900, 2900 ] 965 0.262% 99.844% ( 2900, 4400 ] 636 0.173% 100.017% ( 4400, 6600 ] 196 0.053% 100.070% ( 6600, 9900 ] 33 0.009% 100.079% ( 9900, 14000 ] 1 0.000% 100.079% ( 14000, 22000 ] 1 0.000% 100.080% ( 22000, 33000 ] 1 0.000% 100.080% ** Level 5 read latency histogram (micros): Count: 25149 Average: 1196.6489 StdDev: 1449.75 Min: 82 Median: 630.5201 Max: 45914 Percentiles: P50: 630.52 P75: 1572.04 P99: 6422.28 P99.9: 9726.51 P99.99: 13859.26 ------------------------------------------------------ ( 76, 110 ] 340 1.352% 1.352% ( 110, 170 ] 2706 10.760% 12.112% ## ( 170, 250 ] 3156 12.549% 24.661% ### ( 250, 380 ] 2786 11.078% 35.739% ## ( 380, 580 ] 3060 12.167% 47.906% ## ( 580, 870 ] 3028 12.040% 59.947% ## ( 870, 1300 ] 2748 10.927% 70.874% ## ( 1300, 1900 ] 2291 9.110% 79.983% ## ( 1900, 2900 ] 1987 7.901% 87.884% ## ( 2900, 4400 ] 1892 7.523% 95.407% ## ( 4400, 6600 ] 984 3.913% 99.320% # ( 6600, 9900 ] 155 0.616% 99.936% ( 9900, 14000 ] 15 0.060% 99.996% ( 33000, 50000 ] 2 0.008% 100.004% ** DB Stats ** Uptime(secs): 10.7 total, 10.4 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 92.06 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.544 H:M:S, 61.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 94.30 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:6.544 H:M:S, 62.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/06-07:37:16 ... thread 0: (1000000,2000000) ops and (289911.4,144051.8) ops/second in (3.449330,13.883897) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/8 3.88 GB 163.6 1.4 0.0 1.4 3.3 1.8 0.0 1.8 103.5 238.3 14.00 10.87 32 0.438 1471K 1606 0.0 0.0 L1 4/4 509.90 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 22/0 966.64 MB 1.0 4.1 2.9 1.2 4.1 2.9 0.0 1.4 110.8 110.1 37.93 19.90 18 2.107 1414K 487 0.0 0.0 L4 173/0 9.63 GB 1.0 4.7 3.3 1.4 4.6 3.2 0.1 1.4 118.7 117.4 40.54 22.16 36 1.126 3501K 1403 0.0 0.0 L5 1132/0 92.95 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1347/12 107.91 GB 0.0 10.2 6.2 4.0 12.0 8.0 3.4 6.5 113.2 132.7 92.47 52.93 86 1.075 6387K 3496 0.0 0.0 Int 0/0 0.00 KB 0.0 10.2 6.2 4.0 12.0 8.0 3.4 6.5 113.2 132.7 92.47 52.93 86 1.075 6387K 3496 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.2 6.2 4.0 10.1 6.1 0.0 0.0 124.2 123.2 84.28 47.67 59 1.428 6387K 3496 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 230.7 8.19 5.26 27 0.303 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 14.1 total, 13.9 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.99 GB write, 868.00 MB/s write, 10.22 GB read, 740.06 MB/s read, 92.5 seconds Interval compaction: 11.99 GB write, 883.90 MB/s write, 10.22 GB read, 753.61 MB/s read, 92.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 25, pending-compaction-bytes-stops: 0, total-delays: 25, total-stops: 26, interval: 50 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 916979 Average: 9.3317 StdDev: 124.35 Min: 1 Median: 1.8232 Max: 31576 Percentiles: P50: 1.82 P75: 2.67 P99: 103.70 P99.9: 1078.48 P99.99: 2181.78 ------------------------------------------------------ [ 0, 1 ] 62032 6.765% 6.765% # ( 1, 2 ] 481583 52.518% 59.283% ########### ( 2, 3 ] 214415 23.383% 82.666% ##### ( 3, 4 ] 60514 6.599% 89.265% # ( 4, 6 ] 66641 7.267% 96.533% # ( 6, 10 ] 18727 2.042% 98.575% ( 10, 15 ] 3084 0.336% 98.911% ( 15, 22 ] 472 0.051% 98.963% ( 22, 34 ] 46 0.005% 98.968% ( 34, 51 ] 28 0.003% 98.971% ( 51, 76 ] 23 0.003% 98.973% ( 76, 110 ] 301 0.033% 99.006% ( 110, 170 ] 2370 0.258% 99.265% ( 170, 250 ] 2261 0.247% 99.511% ( 250, 380 ] 1352 0.147% 99.659% ( 380, 580 ] 1170 0.128% 99.786% ( 580, 870 ] 773 0.084% 99.871% ( 870, 1300 ] 559 0.061% 99.932% ( 1300, 1900 ] 426 0.046% 99.978% ( 1900, 2900 ] 395 0.043% 100.021% ( 2900, 4400 ] 279 0.030% 100.051% ( 4400, 6600 ] 75 0.008% 100.060% ( 6600, 9900 ] 12 0.001% 100.061% ( 9900, 14000 ] 2 0.000% 100.061% ( 14000, 22000 ] 1 0.000% 100.061% ( 22000, 33000 ] 3 0.000% 100.062% ** Level 1 read latency histogram (micros): Count: 380 Average: 740.6553 StdDev: 2500.44 Min: 93 Median: 230.0000 Max: 31479 Percentiles: P50: 230.00 P75: 488.70 P99: 11540.00 P99.9: 30910.00 P99.99: 31479.00 ------------------------------------------------------ ( 76, 110 ] 14 3.684% 3.684% # ( 110, 170 ] 104 27.368% 31.053% ##### ( 170, 250 ] 96 25.263% 56.316% ##### ( 250, 380 ] 46 12.105% 68.421% ## ( 380, 580 ] 46 12.105% 80.526% ## ( 580, 870 ] 22 5.789% 86.316% # ( 870, 1300 ] 17 4.474% 90.789% # ( 1300, 1900 ] 14 3.684% 94.474% # ( 1900, 2900 ] 7 1.842% 96.316% ( 2900, 4400 ] 6 1.579% 97.895% ( 4400, 6600 ] 1 0.263% 98.158% ( 6600, 9900 ] 2 0.526% 98.684% ( 9900, 14000 ] 3 0.789% 99.474% ( 22000, 33000 ] 2 0.526% 100.000% ** Level 2 read latency histogram (micros): Count: 601986 Average: 16.4423 StdDev: 212.81 Min: 1 Median: 2.2347 Max: 31928 Percentiles: P50: 2.23 P75: 3.27 P99: 12.73 P99.9: 2170.58 P99.99: 2816.33 ------------------------------------------------------ [ 0, 1 ] 6026 1.001% 1.001% ( 1, 2 ] 254309 42.245% 43.246% ######## ( 2, 3 ] 173269 28.783% 72.029% ###### ( 3, 4 ] 65377 10.860% 82.889% ## ( 4, 6 ] 75724 12.579% 95.468% ### ( 6, 10 ] 19232 3.195% 98.663% # ( 10, 15 ] 3721 0.618% 99.281% ( 15, 22 ] 564 0.094% 99.375% ( 22, 34 ] 56 0.009% 99.384% ( 34, 51 ] 21 0.003% 99.388% ( 51, 76 ] 26 0.004% 99.392% ( 76, 110 ] 24 0.004% 99.396% ( 110, 170 ] 26 0.004% 99.400% ( 170, 250 ] 59 0.010% 99.410% ( 250, 380 ] 155 0.026% 99.436% ( 380, 580 ] 243 0.040% 99.476% ( 580, 870 ] 525 0.087% 99.563% ( 870, 1300 ] 838 0.139% 99.702% ( 1300, 1900 ] 962 0.160% 99.862% ( 1900, 2900 ] 839 0.139% 100.002% ( 2900, 4400 ] 496 0.082% 100.084% ( 4400, 6600 ] 137 0.023% 100.107% ( 6600, 9900 ] 46 0.008% 100.114% ( 9900, 14000 ] 6 0.001% 100.115% ( 14000, 22000 ] 2 0.000% 100.116% ( 22000, 33000 ] 6 0.001% 100.117% ** Level 3 read latency histogram (micros): Count: 1052474 Average: 9.3174 StdDev: 133.09 Min: 1 Median: 2.1415 Max: 25145 Percentiles: P50: 2.14 P75: 3.01 P99: 9.49 P99.9: 825.93 P99.99: 1473.84 ------------------------------------------------------ [ 0, 1 ] 10061 0.956% 0.956% ( 1, 2 ] 472964 44.938% 45.894% ######### ( 2, 3 ] 305320 29.010% 74.904% ###### ( 3, 4 ] 109050 10.361% 85.265% ## ( 4, 6 ] 123466 11.731% 96.996% ## ( 6, 10 ] 24150 2.295% 99.291% ( 10, 15 ] 4002 0.380% 99.671% ( 15, 22 ] 500 0.048% 99.719% ( 22, 34 ] 45 0.004% 99.723% ( 34, 51 ] 37 0.004% 99.726% ( 51, 76 ] 55 0.005% 99.732% ( 76, 110 ] 59 0.006% 99.737% ( 110, 170 ] 310 0.029% 99.767% ( 170, 250 ] 330 0.031% 99.798% ( 250, 380 ] 276 0.026% 99.824% ( 380, 580 ] 353 0.034% 99.858% ( 580, 870 ] 523 0.050% 99.908% ( 870, 1300 ] 669 0.064% 99.971% ( 1300, 1900 ] 686 0.065% 100.036% ( 1900, 2900 ] 590 0.056% 100.092% ( 2900, 4400 ] 477 0.045% 100.138% ( 4400, 6600 ] 136 0.013% 100.151% ( 6600, 9900 ] 35 0.003% 100.154% ( 14000, 22000 ] 1 0.000% 100.154% ( 22000, 33000 ] 1 0.000% 100.154% ** Level 4 read latency histogram (micros): Count: 374080 Average: 33.3006 StdDev: 265.55 Min: 1 Median: 2.1794 Max: 23152 Percentiles: P50: 2.18 P75: 3.25 P99: 879.22 P99.9: 3374.73 P99.99: 4166.28 ------------------------------------------------------ [ 0, 1 ] 4238 1.133% 1.133% ( 1, 2 ] 164423 43.954% 45.087% ######### ( 2, 3 ] 102428 27.381% 72.468% ##### ( 3, 4 ] 37857 10.120% 82.588% ## ( 4, 6 ] 40841 10.918% 93.506% ## ( 6, 10 ] 10204 2.728% 96.234% # ( 10, 15 ] 1552 0.415% 96.649% ( 15, 22 ] 206 0.055% 96.704% ( 22, 34 ] 26 0.007% 96.711% ( 34, 51 ] 17 0.005% 96.715% ( 51, 76 ] 25 0.007% 96.722% ( 76, 110 ] 299 0.080% 96.802% ( 110, 170 ] 2361 0.631% 97.433% ( 170, 250 ] 2177 0.582% 98.015% ( 250, 380 ] 1247 0.333% 98.348% ( 380, 580 ] 1261 0.337% 98.685% ( 580, 870 ] 1152 0.308% 98.993% ( 870, 1300 ] 1175 0.314% 99.307% ( 1300, 1900 ] 1048 0.280% 99.588% ( 1900, 2900 ] 967 0.259% 99.846% ( 2900, 4400 ] 638 0.171% 100.017% ( 4400, 6600 ] 196 0.052% 100.069% ( 6600, 9900 ] 33 0.009% 100.078% ( 9900, 14000 ] 1 0.000% 100.078% ( 14000, 22000 ] 1 0.000% 100.078% ( 22000, 33000 ] 1 0.000% 100.079% ** Level 5 read latency histogram (micros): Count: 62895 Average: 626.6115 StdDev: 1037.83 Min: 81 Median: 241.8767 Max: 45914 Percentiles: P50: 241.88 P75: 556.77 P99: 5581.47 P99.9: 8922.88 P99.99: 12827.54 ------------------------------------------------------ ( 76, 110 ] 2153 3.423% 3.423% # ( 110, 170 ] 16416 26.101% 29.524% ##### ( 170, 250 ] 14334 22.790% 52.314% ##### ( 250, 380 ] 8069 12.829% 65.143% ### ( 380, 580 ] 7014 11.152% 76.295% ## ( 580, 870 ] 4443 7.064% 83.360% # ( 870, 1300 ] 3063 4.870% 88.230% # ( 1300, 1900 ] 2328 3.701% 91.931% # ( 1900, 2900 ] 2016 3.205% 95.136% # ( 2900, 4400 ] 1900 3.021% 98.157% # ( 4400, 6600 ] 987 1.569% 99.727% ( 6600, 9900 ] 155 0.246% 99.973% ( 9900, 14000 ] 15 0.024% 99.997% ( 33000, 50000 ] 2 0.003% 100.000% ** DB Stats ** Uptime(secs): 14.1 total, 3.4 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 139.21 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.544 H:M:S, 46.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 285.34 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/06-07:37:19 ... thread 0: (1000000,3000000) ops and (276311.7,171399.2) ops/second in (3.619101,17.502998) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/8 4.80 GB 258.0 2.4 0.0 2.4 5.2 2.8 0.0 1.9 116.7 251.2 21.06 17.35 51 0.413 2494K 2611 0.0 0.0 L1 4/4 509.90 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 22/0 966.64 MB 1.0 4.1 2.9 1.2 4.1 2.9 0.0 1.4 110.8 110.1 37.93 19.90 18 2.107 1414K 487 0.0 0.0 L4 173/0 9.63 GB 1.0 4.7 3.3 1.4 4.6 3.2 0.1 1.4 118.7 117.4 40.54 22.16 36 1.126 3501K 1403 0.0 0.0 L5 1132/0 92.95 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1350/12 108.83 GB 0.0 11.2 6.2 5.0 13.9 8.9 3.4 5.0 115.3 142.9 99.53 59.40 105 0.948 7411K 4501 0.0 0.0 Int 0/0 0.00 KB 0.0 11.2 6.2 5.0 13.9 8.9 3.4 5.0 115.3 142.9 99.53 59.40 105 0.948 7411K 4501 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.2 6.2 5.0 11.1 6.1 0.0 0.0 130.0 129.1 88.22 51.51 63 1.400 7411K 4501 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 250.7 11.31 7.90 42 0.269 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 17.8 total, 17.5 interval Flush(GB): cumulative 2.769, interval 2.769 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.89 GB write, 801.02 MB/s write, 11.20 GB read, 646.00 MB/s read, 99.5 seconds Interval compaction: 13.89 GB write, 812.66 MB/s write, 11.20 GB read, 655.39 MB/s read, 99.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 1, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 1, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 25, pending-compaction-bytes-stops: 0, total-delays: 26, total-stops: 26, interval: 51 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1204136 Average: 10.1820 StdDev: 112.66 Min: 0 Median: 1.7415 Max: 31576 Percentiles: P50: 1.74 P75: 2.54 P99: 197.98 P99.9: 855.39 P99.99: 1840.66 ------------------------------------------------------ [ 0, 1 ] 116897 9.708% 9.708% ## ( 1, 2 ] 654338 54.341% 64.049% ########### ( 2, 3 ] 245835 20.416% 84.465% #### ( 3, 4 ] 68541 5.692% 90.157% # ( 4, 6 ] 72276 6.002% 96.159% # ( 6, 10 ] 19735 1.639% 97.798% ( 10, 15 ] 3425 0.284% 98.083% ( 15, 22 ] 501 0.042% 98.124% ( 22, 34 ] 48 0.004% 98.128% ( 34, 51 ] 29 0.002% 98.131% ( 51, 76 ] 23 0.002% 98.132% ( 76, 110 ] 847 0.070% 98.203% ( 110, 170 ] 7299 0.606% 98.809% ( 170, 250 ] 6580 0.546% 99.355% ( 250, 380 ] 3051 0.253% 99.609% ( 380, 580 ] 2374 0.197% 99.806% ( 580, 870 ] 1194 0.099% 99.905% ( 870, 1300 ] 628 0.052% 99.957% ( 1300, 1900 ] 439 0.036% 99.994% ( 1900, 2900 ] 407 0.034% 100.027% ( 2900, 4400 ] 285 0.024% 100.051% ( 4400, 6600 ] 75 0.006% 100.057% ( 6600, 9900 ] 12 0.001% 100.058% ( 9900, 14000 ] 2 0.000% 100.059% ( 14000, 22000 ] 1 0.000% 100.059% ( 22000, 33000 ] 3 0.000% 100.059% ** Level 1 read latency histogram (micros): Count: 50372 Average: 10.9987 StdDev: 228.28 Min: 1 Median: 2.5354 Max: 31479 Percentiles: P50: 2.54 P75: 3.75 P99: 167.29 P99.9: 1026.19 P99.99: 9838.62 ------------------------------------------------------ [ 0, 1 ] 752 1.493% 1.493% ( 1, 2 ] 15507 30.785% 32.278% ###### ( 2, 3 ] 16675 33.104% 65.382% ####### ( 3, 4 ] 6492 12.888% 78.270% ### ( 4, 6 ] 7379 14.649% 92.919% ### ( 6, 10 ] 2137 4.242% 97.161% # ( 10, 15 ] 425 0.844% 98.005% ( 15, 22 ] 163 0.324% 98.328% ( 22, 34 ] 28 0.056% 98.384% ( 34, 51 ] 1 0.002% 98.386% ( 76, 110 ] 40 0.079% 98.465% ( 110, 170 ] 282 0.560% 99.025% ( 170, 250 ] 227 0.451% 99.476% ( 250, 380 ] 96 0.191% 99.666% ( 380, 580 ] 78 0.155% 99.821% ( 580, 870 ] 32 0.064% 99.885% ( 870, 1300 ] 21 0.042% 99.927% ( 1300, 1900 ] 14 0.028% 99.954% ( 1900, 2900 ] 8 0.016% 99.970% ( 2900, 4400 ] 7 0.014% 99.984% ( 4400, 6600 ] 1 0.002% 99.986% ( 6600, 9900 ] 2 0.004% 99.990% ( 9900, 14000 ] 3 0.006% 99.996% ( 22000, 33000 ] 2 0.004% 100.000% ** Level 2 read latency histogram (micros): Count: 601986 Average: 16.4423 StdDev: 212.81 Min: 1 Median: 2.2347 Max: 31928 Percentiles: P50: 2.23 P75: 3.27 P99: 12.73 P99.9: 2170.58 P99.99: 2816.33 ------------------------------------------------------ [ 0, 1 ] 6026 1.001% 1.001% ( 1, 2 ] 254309 42.245% 43.246% ######## ( 2, 3 ] 173269 28.783% 72.029% ###### ( 3, 4 ] 65377 10.860% 82.889% ## ( 4, 6 ] 75724 12.579% 95.468% ### ( 6, 10 ] 19232 3.195% 98.663% # ( 10, 15 ] 3721 0.618% 99.281% ( 15, 22 ] 564 0.094% 99.375% ( 22, 34 ] 56 0.009% 99.384% ( 34, 51 ] 21 0.003% 99.388% ( 51, 76 ] 26 0.004% 99.392% ( 76, 110 ] 24 0.004% 99.396% ( 110, 170 ] 26 0.004% 99.400% ( 170, 250 ] 59 0.010% 99.410% ( 250, 380 ] 155 0.026% 99.436% ( 380, 580 ] 243 0.040% 99.476% ( 580, 870 ] 525 0.087% 99.563% ( 870, 1300 ] 838 0.139% 99.702% ( 1300, 1900 ] 962 0.160% 99.862% ( 1900, 2900 ] 839 0.139% 100.002% ( 2900, 4400 ] 496 0.082% 100.084% ( 4400, 6600 ] 137 0.023% 100.107% ( 6600, 9900 ] 46 0.008% 100.114% ( 9900, 14000 ] 6 0.001% 100.115% ( 14000, 22000 ] 2 0.000% 100.116% ( 22000, 33000 ] 6 0.001% 100.117% ** Level 3 read latency histogram (micros): Count: 1053199 Average: 9.4632 StdDev: 133.21 Min: 1 Median: 2.1427 Max: 25145 Percentiles: P50: 2.14 P75: 3.01 P99: 9.61 P99.9: 828.08 P99.99: 1474.39 ------------------------------------------------------ [ 0, 1 ] 10061 0.955% 0.955% ( 1, 2 ] 472964 44.907% 45.863% ######### ( 2, 3 ] 305320 28.990% 74.852% ###### ( 3, 4 ] 109050 10.354% 85.207% ## ( 4, 6 ] 123466 11.723% 96.930% ## ( 6, 10 ] 24150 2.293% 99.223% ( 10, 15 ] 4002 0.380% 99.603% ( 15, 22 ] 500 0.047% 99.650% ( 22, 34 ] 45 0.004% 99.654% ( 34, 51 ] 37 0.004% 99.658% ( 51, 76 ] 55 0.005% 99.663% ( 76, 110 ] 95 0.009% 99.672% ( 110, 170 ] 613 0.058% 99.730% ( 170, 250 ] 549 0.052% 99.782% ( 250, 380 ] 366 0.035% 99.817% ( 380, 580 ] 410 0.039% 99.856% ( 580, 870 ] 541 0.051% 99.907% ( 870, 1300 ] 670 0.064% 99.971% ( 1300, 1900 ] 687 0.065% 100.036% ( 1900, 2900 ] 590 0.056% 100.092% ( 2900, 4400 ] 477 0.045% 100.138% ( 4400, 6600 ] 136 0.013% 100.150% ( 6600, 9900 ] 35 0.003% 100.154% ( 14000, 22000 ] 1 0.000% 100.154% ( 22000, 33000 ] 1 0.000% 100.154% ** Level 4 read latency histogram (micros): Count: 380676 Average: 36.4663 StdDev: 265.07 Min: 1 Median: 2.2113 Max: 23152 Percentiles: P50: 2.21 P75: 3.38 P99: 867.89 P99.9: 3359.23 P99.99: 4164.73 ------------------------------------------------------ [ 0, 1 ] 4238 1.113% 1.113% ( 1, 2 ] 164443 43.198% 44.311% ######### ( 2, 3 ] 102513 26.929% 71.240% ##### ( 3, 4 ] 37902 9.956% 81.197% ## ( 4, 6 ] 40892 10.742% 91.939% ## ( 6, 10 ] 10210 2.682% 94.621% # ( 10, 15 ] 1556 0.409% 95.029% ( 15, 22 ] 206 0.054% 95.083% ( 22, 34 ] 26 0.007% 95.090% ( 34, 51 ] 17 0.004% 95.095% ( 51, 76 ] 25 0.007% 95.101% ( 76, 110 ] 601 0.158% 95.259% ( 110, 170 ] 5009 1.316% 96.575% ( 170, 250 ] 4170 1.095% 97.670% ( 250, 380 ] 1995 0.524% 98.195% ( 380, 580 ] 1733 0.455% 98.650% ( 580, 870 ] 1343 0.353% 99.003% ( 870, 1300 ] 1199 0.315% 99.318% ( 1300, 1900 ] 1051 0.276% 99.594% ( 1900, 2900 ] 971 0.255% 99.849% ( 2900, 4400 ] 638 0.168% 100.016% ( 4400, 6600 ] 196 0.051% 100.068% ( 6600, 9900 ] 33 0.009% 100.076% ( 9900, 14000 ] 1 0.000% 100.077% ( 14000, 22000 ] 1 0.000% 100.077% ( 22000, 33000 ] 1 0.000% 100.077% ** Level 5 read latency histogram (micros): Count: 104861 Average: 464.9799 StdDev: 832.69 Min: 79 Median: 212.3072 Max: 45914 Percentiles: P50: 212.31 P75: 388.35 P99: 4641.60 P99.9: 8008.12 P99.99: 11407.13 ------------------------------------------------------ ( 76, 110 ] 4153 3.960% 3.960% # ( 110, 170 ] 33765 32.200% 36.160% ###### ( 170, 250 ] 27444 26.172% 62.332% ##### ( 250, 380 ] 12849 12.253% 74.585% ## ( 380, 580 ] 10425 9.942% 84.527% ## ( 580, 870 ] 5552 5.295% 89.822% # ( 870, 1300 ] 3245 3.095% 92.916% # ( 1300, 1900 ] 2340 2.232% 95.148% ( 1900, 2900 ] 2029 1.935% 97.083% ( 2900, 4400 ] 1902 1.814% 98.897% ( 4400, 6600 ] 987 0.941% 99.838% ( 6600, 9900 ] 155 0.148% 99.986% ( 9900, 14000 ] 15 0.014% 100.000% ( 33000, 50000 ] 2 0.002% 100.002% ** DB Stats ** Uptime(secs): 17.8 total, 3.6 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 166.26 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.644 H:M:S, 37.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 271.94 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.100 H:M:S, 2.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 263.953 micros/op 15136 ops/sec 60.186 seconds 910996 operations; 14.7 MB/s (232999 of 232999 found) Microseconds per read: Count: 910996 Average: 263.9544 StdDev: 802.43 Min: 2 Median: 196.8727 Max: 82189 Percentiles: P50: 196.87 P75: 240.11 P99: 1256.87 P99.9: 14596.54 P99.99: 31607.03 ------------------------------------------------------ ( 1, 2 ] 9 0.001% 0.001% ( 2, 3 ] 41 0.005% 0.005% ( 3, 4 ] 66 0.007% 0.013% ( 4, 6 ] 82 0.009% 0.022% ( 6, 10 ] 419 0.046% 0.068% ( 10, 15 ] 3880 0.426% 0.494% ( 15, 22 ] 5028 0.552% 1.046% ( 22, 34 ] 9102 0.999% 2.045% ( 34, 51 ] 2756 0.303% 2.347% ( 51, 76 ] 302 0.033% 2.380% ( 76, 110 ] 4129 0.453% 2.834% ( 110, 170 ] 288149 31.630% 34.464% ###### ( 170, 250 ] 421350 46.252% 80.715% ######### ( 250, 380 ] 96741 10.619% 91.335% ## ( 380, 580 ] 46865 5.144% 96.479% # ( 580, 870 ] 16689 1.832% 98.311% ( 870, 1300 ] 6978 0.766% 99.077% ( 1300, 1900 ] 3139 0.345% 99.421% ( 1900, 2900 ] 2011 0.221% 99.642% ( 2900, 4400 ] 1360 0.149% 99.791% ( 4400, 6600 ] 537 0.059% 99.850% ( 6600, 9900 ] 126 0.014% 99.864% ( 9900, 14000 ] 275 0.030% 99.894% ( 14000, 22000 ] 684 0.075% 99.969% ( 22000, 33000 ] 214 0.023% 99.993% ( 33000, 50000 ] 51 0.006% 99.999% ( 50000, 75000 ] 10 0.001% 100.000% ( 75000, 110000 ] 3 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 12044787 rocksdb.bloom.filter.full.positive COUNT : 1027909 rocksdb.bloom.filter.full.true.positive COUNT : 910592 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 404 rocksdb.memtable.miss COUNT : 910592 rocksdb.l0.hit COUNT : 35464 rocksdb.l1.hit COUNT : 11719 rocksdb.l2andup.hit COUNT : 863409 rocksdb.compaction.key.drop.new COUNT : 174114 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3855802 rocksdb.number.keys.read COUNT : 910996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3979187664 rocksdb.bytes.read COUNT : 910996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1640 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 44931425 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 21828805241 rocksdb.compact.write.bytes COUNT : 21645320274 rocksdb.flush.write.bytes COUNT : 3963566213 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 42 rocksdb.number.superversion_acquires COUNT : 897 rocksdb.number.superversion_releases COUNT : 856 rocksdb.number.superversion_cleanups COUNT : 225 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 227 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3931487232 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1428480 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 25674811483 rocksdb.non.last.level.read.count COUNT : 6059886 rocksdb.block.checksum.compute.count COUNT : 6323846 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 195.912963 P95 : 521.552704 P99 : 1255.920518 P100 : 82187.000000 COUNT : 910996 SUM : 239685073 rocksdb.db.write.micros P50 : 1.590214 P95 : 5.277689 P99 : 14.746080 P100 : 664201.000000 COUNT : 3855802 SUM : 55115976 rocksdb.compaction.times.micros P50 : 1125625.000000 P95 : 2716666.666667 P99 : 22990000.000000 P100 : 25064369.000000 COUNT : 91 SUM : 145550043 rocksdb.compaction.times.cpu_micros P50 : 767051.282051 P95 : 1823000.000000 P99 : 14458551.000000 P100 : 14458551.000000 COUNT : 91 SUM : 91672747 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 833.750000 P95 : 17066.666667 P99 : 26730.000000 P100 : 31245.000000 COUNT : 57 SUM : 137246 rocksdb.compaction.outfile.sync.micros P50 : 13754.000000 P95 : 168142.857143 P99 : 245760.000000 P100 : 257470.000000 COUNT : 253 SUM : 10657578 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 227.931034 P95 : 4360.000000 P99 : 6386.285714 P100 : 9857.000000 COUNT : 168 SUM : 182532 rocksdb.table.open.io.micros P50 : 11133.565217 P95 : 21148.936170 P99 : 32057.142857 P100 : 57382.000000 COUNT : 1640 SUM : 17817578 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.183820 P95 : 13.686473 P99 : 221.950339 P100 : 34630.000000 COUNT : 5285632 SUM : 92731879 rocksdb.read.block.get.micros P50 : 156.523045 P95 : 393.072053 P99 : 1258.729482 P100 : 46063.000000 COUNT : 1036563 SUM : 208145099 rocksdb.write.raw.block.micros P50 : 0.530325 P95 : 1.192979 P99 : 3.667681 P100 : 29565.000000 COUNT : 6202836 SUM : 15441297 rocksdb.numfiles.in.singlecompaction P50 : 2.625000 P95 : 5.340000 P99 : 9.826667 P100 : 26.000000 COUNT : 113 SUM : 377 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.566452 P95 : 1236.633866 P99 : 2656.759337 P100 : 664020.000000 COUNT : 259657 SUM : 44956436 rocksdb.sst.read.micros P50 : 2.324531 P95 : 204.386862 P99 : 489.549788 P100 : 45914.000000 COUNT : 6059936 SUM : 284061852 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 910996 SUM : 910996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3855802 SUM : 3979187664 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 215600.000000 P95 : 681166.666667 P99 : 885358.000000 P100 : 885358.000000 COUNT : 57 SUM : 14320987 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 280 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0