[2024-11-20 02:42:00.253997] Starting SPDK v24.09.1-pre git sha1 b18e1bd62 / DPDK 22.11.4 initialization... [2024-11-20 02:42:00.254160] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3738435 ] Set seed to 1732066920250478 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-20 02:42:00.345197] app.c: 917:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-20 02:42:00.384165] app.c: 610:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-20 02:42:00.384211] app.c: 611:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3738435' to capture a snapshot of events at runtime. [2024-11-20 02:42:00.384221] app.c: 616:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-20 02:42:00.384232] app.c: 617:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-20 02:42:00.384240] app.c: 618:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3738435 for offline analysis/debug. [2024-11-20 02:42:00.384325] reactor.c: 990:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Wed Nov 20 02:42:04 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/20-02:42:09 ... thread 0: (1000000,1000000) ops and (202982.1,202982.1) ops/second in (4.926542,4.926542) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/11 3.25 GB 63.0 0.3 0.0 0.3 1.2 0.9 0.0 1.4 55.5 210.7 5.68 3.66 13 0.437 319K 376 0.0 0.0 L1 3/3 279.48 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 20/0 996.64 MB 1.0 0.2 0.1 0.1 0.2 0.1 0.1 1.5 100.6 100.5 1.88 0.92 1 1.881 130K 0 0.0 0.0 L4 169/0 9.46 GB 1.0 2.3 1.4 0.9 2.2 1.4 0.1 1.6 106.9 106.1 21.63 10.95 14 1.545 499K 0 0.0 0.0 L5 1124/0 92.40 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1332/14 106.35 GB 0.0 2.8 1.5 1.2 3.6 2.4 3.9 4.2 96.5 126.1 29.19 15.54 28 1.043 950K 376 0.0 0.0 Int 0/0 0.00 KB 0.0 2.8 1.5 1.2 3.6 2.4 3.9 4.2 96.5 126.1 29.19 15.54 28 1.043 950K 376 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.8 1.5 1.2 2.7 1.5 0.0 0.0 113.5 112.8 24.81 13.10 16 1.550 950K 376 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 201.3 4.38 2.44 12 0.365 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 5.1 total, 4.9 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.59 GB write, 722.24 MB/s write, 2.75 GB read, 552.75 MB/s read, 29.2 seconds Interval compaction: 3.59 GB write, 746.60 MB/s write, 2.75 GB read, 571.39 MB/s read, 29.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 26, interval: 26 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 299699 Average: 10.0315 StdDev: 118.82 Min: 1 Median: 1.6725 Max: 10002 Percentiles: P50: 1.67 P75: 2.40 P99: 155.77 P99.9: 1173.82 P99.99: 2680.73 ------------------------------------------------------ [ 0, 1 ] 38284 12.774% 12.774% ### ( 1, 2 ] 165897 55.355% 68.129% ########### ( 2, 3 ] 51563 17.205% 85.334% ### ( 3, 4 ] 17662 5.893% 91.227% # ( 4, 6 ] 16050 5.355% 96.582% # ( 6, 10 ] 4762 1.589% 98.171% ( 10, 15 ] 746 0.249% 98.420% ( 15, 22 ] 114 0.038% 98.458% ( 22, 34 ] 27 0.009% 98.467% ( 34, 51 ] 10 0.003% 98.470% ( 51, 76 ] 4 0.001% 98.472% ( 76, 110 ] 354 0.118% 98.590% ( 110, 170 ] 1611 0.538% 99.127% ( 170, 250 ] 899 0.300% 99.427% ( 250, 380 ] 521 0.174% 99.601% ( 380, 580 ] 481 0.160% 99.762% ( 580, 870 ] 285 0.095% 99.857% ( 870, 1300 ] 183 0.061% 99.918% ( 1300, 1900 ] 120 0.040% 99.958% ( 1900, 2900 ] 123 0.041% 99.999% ( 2900, 4400 ] 103 0.034% 100.033% ( 4400, 6600 ] 28 0.009% 100.043% ( 6600, 9900 ] 8 0.003% 100.045% ( 9900, 14000 ] 1 0.000% 100.046% ** Level 1 read latency histogram (micros): Count: 54 Average: 965.2778 StdDev: 1748.58 Min: 87 Median: 232.2222 Max: 8881 Percentiles: P50: 232.22 P75: 749.17 P99: 8118.00 P99.9: 8881.00 P99.99: 8881.00 ------------------------------------------------------ ( 76, 110 ] 6 11.111% 11.111% ## ( 110, 170 ] 14 25.926% 37.037% ##### ( 170, 250 ] 9 16.667% 53.704% ### ( 250, 380 ] 6 11.111% 64.815% ## ( 380, 580 ] 2 3.704% 68.519% # ( 580, 870 ] 6 11.111% 79.630% ## ( 870, 1300 ] 1 1.852% 81.481% ( 1300, 1900 ] 2 3.704% 85.185% # ( 1900, 2900 ] 3 5.556% 90.741% # ( 2900, 4400 ] 1 1.852% 92.593% ( 4400, 6600 ] 3 5.556% 98.148% # ( 6600, 9900 ] 1 1.852% 100.000% ** Level 2 read latency histogram (micros): Count: 65416 Average: 17.1972 StdDev: 210.91 Min: 1 Median: 1.9665 Max: 9516 Percentiles: P50: 1.97 P75: 2.96 P99: 13.64 P99.9: 3773.45 P99.99: 6331.33 ------------------------------------------------------ [ 0, 1 ] 1820 2.782% 2.782% # ( 1, 2 ] 31959 48.855% 51.637% ########## ( 2, 3 ] 15935 24.359% 75.997% ##### ( 3, 4 ] 6517 9.962% 85.959% ## ( 4, 6 ] 6487 9.917% 95.876% ## ( 6, 10 ] 1761 2.692% 98.568% # ( 10, 15 ] 388 0.593% 99.161% ( 15, 22 ] 62 0.095% 99.256% ( 22, 34 ] 11 0.017% 99.272% ( 34, 51 ] 2 0.003% 99.275% ( 51, 76 ] 6 0.009% 99.285% ( 76, 110 ] 2 0.003% 99.288% ( 110, 170 ] 6 0.009% 99.297% ( 170, 250 ] 7 0.011% 99.308% ( 250, 380 ] 18 0.028% 99.335% ( 380, 580 ] 35 0.054% 99.389% ( 580, 870 ] 56 0.086% 99.474% ( 870, 1300 ] 83 0.127% 99.601% ( 1300, 1900 ] 67 0.102% 99.703% ( 1900, 2900 ] 82 0.125% 99.829% ( 2900, 4400 ] 80 0.122% 99.951% ( 4400, 6600 ] 29 0.044% 99.995% ( 6600, 9900 ] 6 0.009% 100.005% ** Level 3 read latency histogram (micros): Count: 200688 Average: 13.8331 StdDev: 171.87 Min: 1 Median: 1.9718 Max: 8997 Percentiles: P50: 1.97 P75: 2.98 P99: 12.49 P99.9: 1197.02 P99.99: 1736.13 ------------------------------------------------------ [ 0, 1 ] 5465 2.723% 2.723% # ( 1, 2 ] 97631 48.648% 51.371% ########## ( 2, 3 ] 48498 24.166% 75.537% ##### ( 3, 4 ] 20587 10.258% 85.795% ## ( 4, 6 ] 20447 10.188% 95.984% ## ( 6, 10 ] 5442 2.712% 98.695% # ( 10, 15 ] 1228 0.612% 99.307% ( 15, 22 ] 197 0.098% 99.406% ( 22, 34 ] 32 0.016% 99.421% ( 34, 51 ] 19 0.009% 99.431% ( 51, 76 ] 21 0.010% 99.441% ( 76, 110 ] 20 0.010% 99.451% ( 110, 170 ] 115 0.057% 99.509% ( 170, 250 ] 117 0.058% 99.567% ( 250, 380 ] 113 0.056% 99.623% ( 380, 580 ] 179 0.089% 99.712% ( 580, 870 ] 209 0.104% 99.817% ( 870, 1300 ] 220 0.110% 99.926% ( 1300, 1900 ] 176 0.088% 100.014% ( 1900, 2900 ] 178 0.089% 100.103% ( 2900, 4400 ] 176 0.088% 100.190% ( 4400, 6600 ] 45 0.022% 100.213% ( 6600, 9900 ] 12 0.006% 100.219% ** Level 4 read latency histogram (micros): Count: 172488 Average: 30.9450 StdDev: 266.45 Min: 0 Median: 1.9585 Max: 9366 Percentiles: P50: 1.96 P75: 2.98 P99: 751.23 P99.9: 2937.55 P99.99: 3622.43 ------------------------------------------------------ [ 0, 1 ] 4767 2.764% 2.764% # ( 1, 2 ] 85006 49.282% 52.046% ########## ( 2, 3 ] 40341 23.388% 75.434% ##### ( 3, 4 ] 17472 10.129% 85.563% ## ( 4, 6 ] 15725 9.117% 94.680% ## ( 6, 10 ] 4179 2.423% 97.102% ( 10, 15 ] 867 0.503% 97.605% ( 15, 22 ] 132 0.077% 97.682% ( 22, 34 ] 18 0.010% 97.692% ( 34, 51 ] 12 0.007% 97.699% ( 51, 76 ] 15 0.009% 97.708% ( 76, 110 ] 153 0.089% 97.796% ( 110, 170 ] 593 0.344% 98.140% ( 170, 250 ] 390 0.226% 98.366% ( 250, 380 ] 343 0.199% 98.565% ( 380, 580 ] 436 0.253% 98.818% ( 580, 870 ] 532 0.308% 99.126% ( 870, 1300 ] 552 0.320% 99.446% ( 1300, 1900 ] 429 0.249% 99.695% ( 1900, 2900 ] 345 0.200% 99.895% ( 2900, 4400 ] 340 0.197% 100.092% ( 4400, 6600 ] 109 0.063% 100.155% ( 6600, 9900 ] 17 0.010% 100.165% ** Level 5 read latency histogram (micros): Count: 17610 Average: 757.6529 StdDev: 1208.46 Min: 75 Median: 241.6723 Max: 11846 Percentiles: P50: 241.67 P75: 695.67 P99: 6052.16 P99.9: 9213.12 P99.99: 10095.90 ------------------------------------------------------ ( 51, 76 ] 2 0.011% 0.011% ( 76, 110 ] 1301 7.388% 7.399% # ( 110, 170 ] 4856 27.575% 34.974% ###### ( 170, 250 ] 2954 16.775% 51.749% ### ( 250, 380 ] 1888 10.721% 62.470% ## ( 380, 580 ] 1733 9.841% 72.311% ## ( 580, 870 ] 1189 6.752% 79.063% # ( 870, 1300 ] 861 4.889% 83.952% # ( 1300, 1900 ] 711 4.037% 87.990% # ( 1900, 2900 ] 767 4.355% 92.345% # ( 2900, 4400 ] 874 4.963% 97.308% # ( 4400, 6600 ] 398 2.260% 99.568% ( 6600, 9900 ] 75 0.426% 99.994% ( 9900, 14000 ] 5 0.028% 100.023% ** DB Stats ** Uptime(secs): 5.1 total, 4.9 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 193.17 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.242 H:M:S, 24.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 199.65 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.242 H:M:S, 25.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/20-02:42:13 ... thread 0: (1000000,2000000) ops and (271515.1,232299.4) ops/second in (3.683036,8.609578) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/7 4.23 GB 188.8 1.5 0.0 1.5 3.4 1.8 0.0 1.8 116.1 255.3 13.56 10.98 34 0.399 1599K 1639 0.0 0.0 L1 3/3 279.48 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 20/0 996.64 MB 1.0 0.2 0.1 0.1 0.2 0.1 0.1 1.5 100.6 100.5 1.88 0.92 1 1.881 130K 0 0.0 0.0 L4 169/0 9.46 GB 1.0 2.3 1.4 0.9 2.2 1.4 0.1 1.6 106.9 106.1 21.63 10.95 14 1.545 499K 0 0.0 0.0 L5 1124/0 92.40 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1333/10 107.34 GB 0.0 4.0 1.5 2.5 5.8 3.4 3.9 3.1 109.9 160.4 37.08 22.86 49 0.757 2229K 1639 0.0 0.0 Int 0/0 0.00 KB 0.0 4.0 1.5 2.5 5.8 3.4 3.9 3.1 109.9 160.4 37.08 22.86 49 0.757 2229K 1639 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.0 1.5 2.5 4.0 1.5 0.0 0.0 137.8 137.1 29.59 17.77 21 1.409 2229K 1639 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 252.3 7.49 5.08 28 0.267 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.8 total, 8.6 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.81 GB write, 677.43 MB/s write, 3.98 GB read, 464.37 MB/s read, 37.1 seconds Interval compaction: 5.81 GB write, 690.51 MB/s write, 3.98 GB read, 473.33 MB/s read, 37.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 2, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 2, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 2, total-stops: 26, interval: 28 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 604447 Average: 12.8393 StdDev: 96.08 Min: 0 Median: 1.5533 Max: 10002 Percentiles: P50: 1.55 P75: 2.04 P99: 235.89 P99.9: 749.02 P99.99: 1250.47 ------------------------------------------------------ [ 0, 1 ] 119023 19.691% 19.691% #### ( 1, 2 ] 331132 54.783% 74.474% ########### ( 2, 3 ] 74360 12.302% 86.776% ## ( 3, 4 ] 26562 4.394% 91.170% # ( 4, 6 ] 20730 3.430% 94.600% # ( 6, 10 ] 6016 0.995% 95.595% ( 10, 15 ] 1206 0.200% 95.795% ( 15, 22 ] 212 0.035% 95.830% ( 22, 34 ] 48 0.008% 95.838% ( 34, 51 ] 15 0.002% 95.840% ( 51, 76 ] 7 0.001% 95.841% ( 76, 110 ] 3100 0.513% 96.354% ( 110, 170 ] 11558 1.912% 98.267% ( 170, 250 ] 5384 0.891% 99.157% ( 250, 380 ] 2264 0.375% 99.532% ( 380, 580 ] 1804 0.298% 99.830% ( 580, 870 ] 725 0.120% 99.950% ( 870, 1300 ] 273 0.045% 99.995% ( 1300, 1900 ] 141 0.023% 100.019% ( 1900, 2900 ] 136 0.022% 100.041% ( 2900, 4400 ] 113 0.019% 100.060% ( 4400, 6600 ] 28 0.005% 100.065% ( 6600, 9900 ] 8 0.001% 100.066% ( 9900, 14000 ] 1 0.000% 100.066% ** Level 1 read latency histogram (micros): Count: 220 Average: 384.9773 StdDev: 932.64 Min: 87 Median: 162.4211 Max: 8881 Percentiles: P50: 162.42 P75: 269.26 P99: 5720.00 P99.9: 8881.00 P99.99: 8881.00 ------------------------------------------------------ ( 76, 110 ] 27 12.273% 12.273% ## ( 110, 170 ] 95 43.182% 55.455% ######### ( 170, 250 ] 39 17.727% 73.182% #### ( 250, 380 ] 27 12.273% 85.455% ## ( 380, 580 ] 12 5.455% 90.909% # ( 580, 870 ] 9 4.091% 95.000% # ( 870, 1300 ] 1 0.455% 95.455% ( 1300, 1900 ] 2 0.909% 96.364% ( 1900, 2900 ] 3 1.364% 97.727% ( 2900, 4400 ] 1 0.455% 98.182% ( 4400, 6600 ] 3 1.364% 99.545% ( 6600, 9900 ] 1 0.455% 100.000% ** Level 2 read latency histogram (micros): Count: 65416 Average: 17.1972 StdDev: 210.91 Min: 1 Median: 1.9665 Max: 9516 Percentiles: P50: 1.97 P75: 2.96 P99: 13.64 P99.9: 3773.45 P99.99: 6331.33 ------------------------------------------------------ [ 0, 1 ] 1820 2.782% 2.782% # ( 1, 2 ] 31959 48.855% 51.637% ########## ( 2, 3 ] 15935 24.359% 75.997% ##### ( 3, 4 ] 6517 9.962% 85.959% ## ( 4, 6 ] 6487 9.917% 95.876% ## ( 6, 10 ] 1761 2.692% 98.568% # ( 10, 15 ] 388 0.593% 99.161% ( 15, 22 ] 62 0.095% 99.256% ( 22, 34 ] 11 0.017% 99.272% ( 34, 51 ] 2 0.003% 99.275% ( 51, 76 ] 6 0.009% 99.285% ( 76, 110 ] 2 0.003% 99.288% ( 110, 170 ] 6 0.009% 99.297% ( 170, 250 ] 7 0.011% 99.308% ( 250, 380 ] 18 0.028% 99.335% ( 380, 580 ] 35 0.054% 99.389% ( 580, 870 ] 56 0.086% 99.474% ( 870, 1300 ] 83 0.127% 99.601% ( 1300, 1900 ] 67 0.102% 99.703% ( 1900, 2900 ] 82 0.125% 99.829% ( 2900, 4400 ] 80 0.122% 99.951% ( 4400, 6600 ] 29 0.044% 99.995% ( 6600, 9900 ] 6 0.009% 100.005% ** Level 3 read latency histogram (micros): Count: 201558 Average: 14.6698 StdDev: 172.42 Min: 1 Median: 1.9763 Max: 8997 Percentiles: P50: 1.98 P75: 2.99 P99: 47.80 P99.9: 1201.03 P99.99: 1742.65 ------------------------------------------------------ [ 0, 1 ] 5465 2.711% 2.711% # ( 1, 2 ] 97631 48.438% 51.150% ########## ( 2, 3 ] 48498 24.062% 75.211% ##### ( 3, 4 ] 20587 10.214% 85.425% ## ( 4, 6 ] 20447 10.144% 95.570% ## ( 6, 10 ] 5442 2.700% 98.269% # ( 10, 15 ] 1228 0.609% 98.879% ( 15, 22 ] 197 0.098% 98.976% ( 22, 34 ] 32 0.016% 98.992% ( 34, 51 ] 19 0.009% 99.002% ( 51, 76 ] 21 0.010% 99.012% ( 76, 110 ] 136 0.067% 99.080% ( 110, 170 ] 532 0.264% 99.344% ( 170, 250 ] 274 0.136% 99.480% ( 250, 380 ] 196 0.097% 99.577% ( 380, 580 ] 255 0.127% 99.703% ( 580, 870 ] 224 0.111% 99.814% ( 870, 1300 ] 224 0.111% 99.926% ( 1300, 1900 ] 176 0.087% 100.013% ( 1900, 2900 ] 179 0.089% 100.102% ( 2900, 4400 ] 177 0.088% 100.190% ( 4400, 6600 ] 45 0.022% 100.212% ( 6600, 9900 ] 12 0.006% 100.218% ** Level 4 read latency histogram (micros): Count: 178996 Average: 36.3353 StdDev: 264.77 Min: 0 Median: 1.9962 Max: 9366 Percentiles: P50: 2.00 P75: 3.22 P99: 752.83 P99.9: 2913.14 P99.99: 3617.64 ------------------------------------------------------ [ 0, 1 ] 4767 2.663% 2.663% # ( 1, 2 ] 85054 47.517% 50.180% ########## ( 2, 3 ] 40564 22.662% 72.842% ##### ( 3, 4 ] 17593 9.829% 82.671% ## ( 4, 6 ] 15890 8.877% 91.548% ## ( 6, 10 ] 4192 2.342% 93.890% ( 10, 15 ] 872 0.487% 94.378% ( 15, 22 ] 133 0.074% 94.452% ( 22, 34 ] 18 0.010% 94.462% ( 34, 51 ] 12 0.007% 94.469% ( 51, 76 ] 15 0.008% 94.477% ( 76, 110 ] 1028 0.574% 95.051% ( 110, 170 ] 3446 1.925% 96.976% ( 170, 250 ] 1589 0.888% 97.864% ( 250, 380 ] 854 0.477% 98.341% ( 380, 580 ] 800 0.447% 98.788% ( 580, 870 ] 636 0.355% 99.144% ( 870, 1300 ] 571 0.319% 99.463% ( 1300, 1900 ] 432 0.241% 99.704% ( 1900, 2900 ] 348 0.194% 99.898% ( 2900, 4400 ] 343 0.192% 100.090% ( 4400, 6600 ] 109 0.061% 100.151% ( 6600, 9900 ] 17 0.009% 100.160% ** Level 5 read latency histogram (micros): Count: 65982 Average: 346.5207 StdDev: 684.09 Min: 75 Median: 162.7596 Max: 11846 Percentiles: P50: 162.76 P75: 251.60 P99: 4081.66 P99.9: 6952.79 P99.99: 9565.68 ------------------------------------------------------ ( 51, 76 ] 5 0.008% 0.008% ( 76, 110 ] 8014 12.146% 12.153% ## ( 110, 170 ] 28399 43.041% 55.194% ######### ( 170, 250 ] 12998 19.699% 74.893% #### ( 250, 380 ] 5713 8.658% 83.552% ## ( 380, 580 ] 4848 7.347% 90.899% # ( 580, 870 ] 2100 3.183% 94.082% # ( 870, 1300 ] 1002 1.519% 95.600% ( 1300, 1900 ] 739 1.120% 96.720% ( 1900, 2900 ] 807 1.223% 97.943% ( 2900, 4400 ] 885 1.341% 99.285% ( 4400, 6600 ] 398 0.603% 99.888% ( 6600, 9900 ] 75 0.114% 100.002% ( 9900, 14000 ] 5 0.008% 100.009% ** DB Stats ** Uptime(secs): 8.8 total, 3.7 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 224.24 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.534 H:M:S, 17.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 267.23 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.292 H:M:S, 7.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/20-02:42:24 ... thread 0: (1000000,3000000) ops and (93091.1,155024.8) ops/second in (10.742165,19.351743) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/11 5.20 GB 263.3 3.8 0.0 3.8 6.6 2.8 0.0 2.3 155.0 269.5 25.20 22.06 54 0.467 3964K 13K 0.0 0.0 L1 3/3 279.48 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 20/0 996.64 MB 1.0 0.2 0.1 0.1 0.2 0.1 0.1 1.5 100.6 100.5 1.88 0.92 1 1.881 130K 0 0.0 0.0 L4 169/0 9.46 GB 1.0 2.3 1.4 0.9 2.2 1.4 0.1 1.6 106.9 106.1 21.63 10.95 14 1.545 499K 0 0.0 0.0 L5 1124/0 92.40 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1334/14 108.31 GB 0.0 6.3 1.5 4.7 9.1 4.3 3.9 3.2 131.5 190.4 48.71 33.93 69 0.706 4594K 13K 0.0 0.0 Int 0/0 0.00 KB 0.0 6.3 1.5 4.7 9.1 4.3 3.9 3.2 131.5 190.4 48.71 33.93 69 0.706 4594K 13K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.3 1.5 4.7 6.2 1.5 0.0 0.0 167.8 167.0 38.18 26.25 25 1.527 4594K 13K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 275.3 10.53 7.68 44 0.239 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 19.5 total, 19.4 interval Flush(GB): cumulative 2.830, interval 2.830 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 9.05 GB write, 475.02 MB/s write, 6.26 GB read, 328.15 MB/s read, 48.7 seconds Interval compaction: 9.05 GB write, 479.10 MB/s write, 6.26 GB read, 330.97 MB/s read, 48.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 11, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 11, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 11, total-stops: 26, interval: 37 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1341408 Average: 16.2809 StdDev: 80.83 Min: 0 Median: 1.4798 Max: 10002 Percentiles: P50: 1.48 P75: 1.96 P99: 242.92 P99.9: 579.49 P99.99: 814.80 ------------------------------------------------------ [ 0, 1 ] 337655 25.172% 25.172% ##### ( 1, 2 ] 694163 51.749% 76.921% ########## ( 2, 3 ] 119053 8.875% 85.796% ## ( 3, 4 ] 48608 3.624% 89.419% # ( 4, 6 ] 32117 2.394% 91.814% ( 6, 10 ] 8325 0.621% 92.434% ( 10, 15 ] 1725 0.129% 92.563% ( 15, 22 ] 348 0.026% 92.589% ( 22, 34 ] 84 0.006% 92.595% ( 34, 51 ] 23 0.002% 92.597% ( 51, 76 ] 13 0.001% 92.598% ( 76, 110 ] 15035 1.121% 93.719% ( 110, 170 ] 54322 4.050% 97.768% # ( 170, 250 ] 18129 1.351% 99.120% ( 250, 380 ] 5973 0.445% 99.565% ( 380, 580 ] 4506 0.336% 99.901% ( 580, 870 ] 1477 0.110% 100.011% ( 870, 1300 ] 344 0.026% 100.037% ( 1300, 1900 ] 170 0.013% 100.049% ( 1900, 2900 ] 178 0.013% 100.063% ( 2900, 4400 ] 137 0.010% 100.073% ( 4400, 6600 ] 29 0.002% 100.075% ( 6600, 9900 ] 8 0.001% 100.076% ( 9900, 14000 ] 1 0.000% 100.076% ** Level 1 read latency histogram (micros): Count: 797 Average: 238.0753 StdDev: 519.19 Min: 83 Median: 149.1647 Max: 8881 Percentiles: P50: 149.16 P75: 202.33 P99: 2306.00 P99.9: 7269.90 P99.99: 8881.00 ------------------------------------------------------ ( 76, 110 ] 125 15.684% 15.684% ### ( 110, 170 ] 419 52.572% 68.256% ########### ( 170, 250 ] 133 16.688% 84.944% ### ( 250, 380 ] 54 6.775% 91.719% # ( 380, 580 ] 33 4.141% 95.859% # ( 580, 870 ] 17 2.133% 97.992% ( 870, 1300 ] 4 0.502% 98.494% ( 1300, 1900 ] 2 0.251% 98.745% ( 1900, 2900 ] 5 0.627% 99.373% ( 2900, 4400 ] 1 0.125% 99.498% ( 4400, 6600 ] 3 0.376% 99.875% ( 6600, 9900 ] 1 0.125% 100.000% ** Level 2 read latency histogram (micros): Count: 65416 Average: 17.1972 StdDev: 210.91 Min: 1 Median: 1.9665 Max: 9516 Percentiles: P50: 1.97 P75: 2.96 P99: 13.64 P99.9: 3773.45 P99.99: 6331.33 ------------------------------------------------------ [ 0, 1 ] 1820 2.782% 2.782% # ( 1, 2 ] 31959 48.855% 51.637% ########## ( 2, 3 ] 15935 24.359% 75.997% ##### ( 3, 4 ] 6517 9.962% 85.959% ## ( 4, 6 ] 6487 9.917% 95.876% ## ( 6, 10 ] 1761 2.692% 98.568% # ( 10, 15 ] 388 0.593% 99.161% ( 15, 22 ] 62 0.095% 99.256% ( 22, 34 ] 11 0.017% 99.272% ( 34, 51 ] 2 0.003% 99.275% ( 51, 76 ] 6 0.009% 99.285% ( 76, 110 ] 2 0.003% 99.288% ( 110, 170 ] 6 0.009% 99.297% ( 170, 250 ] 7 0.011% 99.308% ( 250, 380 ] 18 0.028% 99.335% ( 380, 580 ] 35 0.054% 99.389% ( 580, 870 ] 56 0.086% 99.474% ( 870, 1300 ] 83 0.127% 99.601% ( 1300, 1900 ] 67 0.102% 99.703% ( 1900, 2900 ] 82 0.125% 99.829% ( 2900, 4400 ] 80 0.122% 99.951% ( 4400, 6600 ] 29 0.044% 99.995% ( 6600, 9900 ] 6 0.009% 100.005% ** Level 3 read latency histogram (micros): Count: 204400 Average: 16.7912 StdDev: 172.55 Min: 1 Median: 1.9908 Max: 8997 Percentiles: P50: 1.99 P75: 3.08 P99: 171.29 P99.9: 1198.85 P99.99: 1742.58 ------------------------------------------------------ [ 0, 1 ] 5465 2.674% 2.674% # ( 1, 2 ] 97631 47.765% 50.438% ########## ( 2, 3 ] 48498 23.727% 74.165% ##### ( 3, 4 ] 20587 10.072% 84.237% ## ( 4, 6 ] 20447 10.003% 94.241% ## ( 6, 10 ] 5442 2.662% 96.903% # ( 10, 15 ] 1228 0.601% 97.504% ( 15, 22 ] 197 0.096% 97.600% ( 22, 34 ] 32 0.016% 97.616% ( 34, 51 ] 19 0.009% 97.625% ( 51, 76 ] 21 0.010% 97.636% ( 76, 110 ] 628 0.307% 97.943% ( 110, 170 ] 2149 1.051% 98.994% ( 170, 250 ] 745 0.364% 99.359% ( 250, 380 ] 328 0.160% 99.519% ( 380, 580 ] 351 0.172% 99.691% ( 580, 870 ] 254 0.124% 99.815% ( 870, 1300 ] 227 0.111% 99.926% ( 1300, 1900 ] 177 0.087% 100.013% ( 1900, 2900 ] 179 0.088% 100.100% ( 2900, 4400 ] 177 0.087% 100.187% ( 4400, 6600 ] 45 0.022% 100.209% ( 6600, 9900 ] 12 0.006% 100.215% ** Level 4 read latency histogram (micros): Count: 200617 Average: 50.0456 StdDev: 255.76 Min: 0 Median: 2.2569 Max: 9366 Percentiles: P50: 2.26 P75: 4.27 P99: 714.99 P99.9: 2855.76 P99.99: 3615.05 ------------------------------------------------------ [ 0, 1 ] 4767 2.376% 2.376% ( 1, 2 ] 85091 42.415% 44.791% ######## ( 2, 3 ] 40679 20.277% 65.068% #### ( 3, 4 ] 17722 8.834% 73.902% ## ( 4, 6 ] 16044 7.997% 81.899% ## ( 6, 10 ] 4206 2.097% 83.995% ( 10, 15 ] 878 0.438% 84.433% ( 15, 22 ] 134 0.067% 84.500% ( 22, 34 ] 18 0.009% 84.509% ( 34, 51 ] 12 0.006% 84.515% ( 51, 76 ] 15 0.007% 84.522% ( 76, 110 ] 4810 2.398% 86.920% ( 110, 170 ] 15489 7.721% 94.641% ## ( 170, 250 ] 5032 2.508% 97.149% # ( 250, 380 ] 1806 0.900% 98.049% ( 380, 580 ] 1528 0.762% 98.811% ( 580, 870 ] 816 0.407% 99.217% ( 870, 1300 ] 597 0.298% 99.515% ( 1300, 1900 ] 435 0.217% 99.732% ( 1900, 2900 ] 353 0.176% 99.908% ( 2900, 4400 ] 346 0.172% 100.080% ( 4400, 6600 ] 109 0.054% 100.135% ( 6600, 9900 ] 17 0.008% 100.143% ** Level 5 read latency histogram (micros): Count: 225187 Average: 219.6431 StdDev: 390.22 Min: 75 Median: 148.2696 Max: 11846 Percentiles: P50: 148.27 P75: 197.31 P99: 1887.57 P99.9: 5736.65 P99.99: 8645.18 ------------------------------------------------------ ( 51, 76 ] 16 0.007% 0.007% ( 76, 110 ] 36525 16.220% 16.227% ### ( 110, 170 ] 119237 52.950% 69.177% ########### ( 170, 250 ] 38411 17.057% 86.235% ### ( 250, 380 ] 12986 5.767% 92.001% # ( 380, 580 ] 10367 4.604% 96.605% # ( 580, 870 ] 3456 1.535% 98.140% ( 870, 1300 ] 1187 0.527% 98.667% ( 1300, 1900 ] 766 0.340% 99.007% ( 1900, 2900 ] 869 0.386% 99.393% ( 2900, 4400 ] 900 0.400% 99.793% ( 4400, 6600 ] 398 0.177% 99.969% ( 6600, 9900 ] 75 0.033% 100.003% ( 9900, 14000 ] 5 0.002% 100.005% ** DB Stats ** Uptime(secs): 19.5 total, 10.7 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 151.26 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:8.560 H:M:S, 43.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 91.62 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:7.025 H:M:S, 65.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 211.282 micros/op 18925 ops/sec 60.130 seconds 1137996 operations; 18.3 MB/s (274999 of 274999 found) Microseconds per read: Count: 1137996 Average: 211.2845 StdDev: 541.44 Min: 2 Median: 168.9748 Max: 99584 Percentiles: P50: 168.97 P75: 228.30 P99: 686.90 P99.9: 4894.41 P99.99: 27980.31 ------------------------------------------------------ ( 1, 2 ] 13 0.001% 0.001% ( 2, 3 ] 44 0.004% 0.005% ( 3, 4 ] 65 0.006% 0.011% ( 4, 6 ] 97 0.009% 0.019% ( 6, 10 ] 414 0.036% 0.056% ( 10, 15 ] 3960 0.348% 0.404% ( 15, 22 ] 3038 0.267% 0.671% ( 22, 34 ] 9049 0.795% 1.466% ( 34, 51 ] 1596 0.140% 1.606% ( 51, 76 ] 213 0.019% 1.625% ( 76, 110 ] 23918 2.102% 3.726% ( 110, 170 ] 535745 47.078% 50.804% ######### ( 170, 250 ] 377846 33.203% 84.007% ####### ( 250, 380 ] 125474 11.026% 95.033% ## ( 380, 580 ] 41119 3.613% 98.646% # ( 580, 870 ] 10919 0.959% 99.606% ( 870, 1300 ] 2188 0.192% 99.798% ( 1300, 1900 ] 361 0.032% 99.830% ( 1900, 2900 ] 389 0.034% 99.864% ( 2900, 4400 ] 330 0.029% 99.893% ( 4400, 6600 ] 356 0.031% 99.924% ( 6600, 9900 ] 225 0.020% 99.944% ( 9900, 14000 ] 176 0.015% 99.959% ( 14000, 22000 ] 270 0.024% 99.983% ( 22000, 33000 ] 142 0.012% 99.996% ( 33000, 50000 ] 42 0.004% 99.999% ( 50000, 75000 ] 6 0.001% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 18331712 rocksdb.bloom.filter.full.positive COUNT : 1316257 rocksdb.bloom.filter.full.true.positive COUNT : 1137567 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 429 rocksdb.memtable.miss COUNT : 1137567 rocksdb.l0.hit COUNT : 62480 rocksdb.l1.hit COUNT : 2984 rocksdb.l2andup.hit COUNT : 1072103 rocksdb.compaction.key.drop.new COUNT : 50989 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3534631 rocksdb.number.keys.read COUNT : 1137996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3647739192 rocksdb.bytes.read COUNT : 1137996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1438 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 46746276 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 10822929138 rocksdb.compact.write.bytes COUNT : 10777132194 rocksdb.flush.write.bytes COUNT : 3633337792 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 55 rocksdb.number.superversion_acquires COUNT : 573 rocksdb.number.superversion_releases COUNT : 545 rocksdb.number.superversion_cleanups COUNT : 148 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3603874816 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1257472 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 16141292691 rocksdb.non.last.level.read.count COUNT : 3755540 rocksdb.block.checksum.compute.count COUNT : 3951267 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 167.926587 P95 : 379.117898 P99 : 684.903231 P100 : 99581.000000 COUNT : 1137996 SUM : 239329405 rocksdb.db.write.micros P50 : 1.533258 P95 : 4.806481 P99 : 16.301929 P100 : 759080.000000 COUNT : 3534631 SUM : 55633238 rocksdb.compaction.times.micros P50 : 1146875.000000 P95 : 3670000.000000 P99 : 5796377.000000 P100 : 5796377.000000 COUNT : 29 SUM : 44163892 rocksdb.compaction.times.cpu_micros P50 : 994210.526316 P95 : 3670000.000000 P99 : 5726359.000000 P100 : 5726359.000000 COUNT : 29 SUM : 32102617 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 749.857143 P95 : 11335.000000 P99 : 61750.000000 P100 : 69619.000000 COUNT : 53 SUM : 163605 rocksdb.compaction.outfile.sync.micros P50 : 4025.000000 P95 : 69062.500000 P99 : 122894.000000 P100 : 122894.000000 COUNT : 78 SUM : 1201070 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 146.279070 P95 : 5115.000000 P99 : 8349.000000 P100 : 9182.000000 COUNT : 94 SUM : 76541 rocksdb.table.open.io.micros P50 : 6600.000000 P95 : 20013.930348 P99 : 25810.000000 P100 : 26927.000000 COUNT : 1438 SUM : 11861895 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.717261 P95 : 146.630620 P99 : 237.026271 P100 : 12753.000000 COUNT : 2622903 SUM : 64054315 rocksdb.read.block.get.micros P50 : 138.788869 P95 : 247.399336 P99 : 552.096787 P100 : 18624.000000 COUNT : 1326896 SUM : 202722634 rocksdb.write.raw.block.micros P50 : 0.547238 P95 : 1.570682 P99 : 3.461837 P100 : 16696.000000 COUNT : 3490232 SUM : 8728378 rocksdb.numfiles.in.singlecompaction P50 : 3.392857 P95 : 5.866667 P99 : 9.120000 P100 : 10.000000 COUNT : 44 SUM : 165 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.566191 P95 : 1945.621723 P99 : 2709.364045 P100 : 758909.000000 COUNT : 254899 SUM : 46764221 rocksdb.sst.read.micros P50 : 2.767288 P95 : 198.959317 P99 : 421.427746 P100 : 11846.000000 COUNT : 3755575 SUM : 253201892 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1137996 SUM : 1137996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3534631 SUM : 3647739192 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 214166.666667 P95 : 620750.000000 P99 : 998043.000000 P100 : 998043.000000 COUNT : 53 SUM : 12446815 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 107 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0