[2024-07-14 12:52:14.780697] Starting SPDK v24.05.1-pre git sha1 5fa2f5086 / DPDK 22.11.4 initialization... [2024-07-14 12:52:14.780834] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1342495 ] Set seed to 1720954334777368 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-14 12:52:14.868160] app.c: 909:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-14 12:52:14.907326] app.c: 604:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-14 12:52:14.907374] app.c: 605:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1342495' to capture a snapshot of events at runtime. [2024-07-14 12:52:14.907384] app.c: 610:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-14 12:52:14.907410] app.c: 611:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-14 12:52:14.907419] app.c: 612:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1342495 for offline analysis/debug. [2024-07-14 12:52:14.907516] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Sun Jul 14 12:52:19 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/07/14-12:52:27 ... thread 0: (1000000,1000000) ops and (127512.2,127512.2) ops/second in (7.842386,7.842386) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 6/3 566.87 MB 25.2 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 112.8 7.26 2.44 9 0.807 0 0 0.0 0.0 L1 3/3 144.59 MB 0.0 0.3 0.3 0.1 0.3 0.3 0.0 1.3 116.0 115.9 2.92 1.53 2 1.458 343K 251 0.0 0.0 L2 11/8 1.25 GB 2.5 0.1 0.1 0.0 0.1 0.1 0.1 1.0 133.0 133.0 0.89 0.54 1 0.894 120K 0 0.0 0.0 L3 42/9 3.54 GB 2.8 3.9 2.6 1.3 3.9 2.6 0.0 1.5 104.5 103.7 38.09 18.13 18 2.116 1779K 2496 0.0 0.0 L4 193/12 11.16 GB 1.1 2.1 1.3 0.7 2.0 1.3 0.3 1.5 113.0 111.3 18.58 9.79 14 1.327 1455K 714 0.0 0.0 L5 1100/0 90.43 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1355/35 107.07 GB 0.0 6.4 4.3 2.1 7.1 5.0 0.4 8.9 96.5 107.7 67.74 32.43 44 1.539 3699K 3461 0.0 0.0 Int 0/0 0.00 KB 0.0 6.4 4.3 2.1 7.1 5.0 0.4 8.9 96.5 107.7 67.74 32.43 44 1.539 3699K 3461 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.4 4.3 2.1 6.3 4.2 0.0 0.0 108.1 107.1 60.47 29.99 35 1.728 3699K 3461 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 112.8 7.26 2.44 9 0.807 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.1 total, 7.8 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.12 GB write, 905.81 MB/s write, 6.39 GB read, 812.02 MB/s read, 67.7 seconds Interval compaction: 7.12 GB write, 929.94 MB/s write, 6.39 GB read, 833.65 MB/s read, 67.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 35, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 35, interval: 35 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 105084 Average: 10.5082 StdDev: 247.08 Min: 1 Median: 1.9063 Max: 34810 Percentiles: P50: 1.91 P75: 2.91 P99: 9.87 P99.9: 1891.09 P99.99: 6525.43 ------------------------------------------------------ [ 0, 1 ] 3183 3.029% 3.029% # ( 1, 2 ] 54462 51.827% 54.856% ########## ( 2, 3 ] 23178 22.057% 76.913% #### ( 3, 4 ] 10567 10.056% 86.969% ## ( 4, 6 ] 10689 10.172% 97.140% ## ( 6, 10 ] 2019 1.921% 99.062% ( 10, 15 ] 505 0.481% 99.542% ( 15, 22 ] 58 0.055% 99.597% ( 22, 34 ] 3 0.003% 99.600% ( 34, 51 ] 4 0.004% 99.604% ( 51, 76 ] 7 0.007% 99.611% ( 76, 110 ] 1 0.001% 99.612% ( 110, 170 ] 1 0.001% 99.613% ( 170, 250 ] 9 0.009% 99.621% ( 250, 380 ] 26 0.025% 99.646% ( 380, 580 ] 41 0.039% 99.685% ( 580, 870 ] 74 0.070% 99.755% ( 870, 1300 ] 80 0.076% 99.832% ( 1300, 1900 ] 73 0.069% 99.901% ( 1900, 2900 ] 41 0.039% 99.940% ( 2900, 4400 ] 38 0.036% 99.976% ( 4400, 6600 ] 15 0.014% 99.990% ( 6600, 9900 ] 3 0.003% 99.993% ( 14000, 22000 ] 3 0.003% 99.996% ( 22000, 33000 ] 3 0.003% 99.999% ( 33000, 50000 ] 1 0.001% 100.000% ** Level 1 read latency histogram (micros): Count: 95989 Average: 6.5563 StdDev: 102.36 Min: 1 Median: 1.9325 Max: 15830 Percentiles: P50: 1.93 P75: 2.96 P99: 9.64 P99.9: 1241.47 P99.99: 4154.62 ------------------------------------------------------ [ 0, 1 ] 2247 2.341% 2.341% ( 1, 2 ] 49059 51.109% 53.450% ########## ( 2, 3 ] 21656 22.561% 76.011% ##### ( 3, 4 ] 10166 10.591% 86.602% ## ( 4, 6 ] 10147 10.571% 97.173% ## ( 6, 10 ] 1928 2.009% 99.181% ( 10, 15 ] 478 0.498% 99.679% ( 15, 22 ] 57 0.059% 99.739% ( 22, 34 ] 4 0.004% 99.743% ( 34, 51 ] 2 0.002% 99.745% ( 51, 76 ] 3 0.003% 99.748% ( 76, 110 ] 2 0.002% 99.750% ( 110, 170 ] 2 0.002% 99.752% ( 170, 250 ] 7 0.007% 99.759% ( 250, 380 ] 14 0.015% 99.774% ( 380, 580 ] 41 0.043% 99.817% ( 580, 870 ] 42 0.044% 99.860% ( 870, 1300 ] 44 0.046% 99.906% ( 1300, 1900 ] 39 0.041% 99.947% ( 1900, 2900 ] 23 0.024% 99.971% ( 2900, 4400 ] 22 0.023% 99.994% ( 4400, 6600 ] 7 0.007% 100.001% ( 14000, 22000 ] 1 0.001% 100.002% ** Level 2 read latency histogram (micros): Count: 751782 Average: 14.4866 StdDev: 193.53 Min: 1 Median: 2.0033 Max: 32412 Percentiles: P50: 2.00 P75: 3.10 P99: 12.66 P99.9: 1028.90 P99.99: 1318.31 ------------------------------------------------------ [ 0, 1 ] 16999 2.261% 2.261% ( 1, 2 ] 358299 47.660% 49.921% ########## ( 2, 3 ] 180070 23.952% 73.874% ##### ( 3, 4 ] 81795 10.880% 84.754% ## ( 4, 6 ] 83007 11.041% 95.795% ## ( 6, 10 ] 20781 2.764% 98.559% # ( 10, 15 ] 6236 0.829% 99.389% ( 15, 22 ] 897 0.119% 99.508% ( 22, 34 ] 102 0.014% 99.522% ( 34, 51 ] 48 0.006% 99.528% ( 51, 76 ] 47 0.006% 99.534% ( 76, 110 ] 29 0.004% 99.538% ( 110, 170 ] 82 0.011% 99.549% ( 170, 250 ] 188 0.025% 99.574% ( 250, 380 ] 387 0.051% 99.626% ( 380, 580 ] 719 0.096% 99.721% ( 580, 870 ] 961 0.128% 99.849% ( 870, 1300 ] 1037 0.138% 99.987% ( 1300, 1900 ] 748 0.099% 100.086% ( 1900, 2900 ] 642 0.085% 100.172% ( 2900, 4400 ] 609 0.081% 100.253% ( 4400, 6600 ] 171 0.023% 100.276% ( 6600, 9900 ] 32 0.004% 100.280% ( 9900, 14000 ] 10 0.001% 100.281% ( 14000, 22000 ] 6 0.001% 100.282% ( 22000, 33000 ] 6 0.001% 100.283% ** Level 3 read latency histogram (micros): Count: 713596 Average: 11.6839 StdDev: 182.61 Min: 1 Median: 1.9498 Max: 34907 Percentiles: P50: 1.95 P75: 2.97 P99: 9.77 P99.9: 767.28 P99.99: 1115.50 ------------------------------------------------------ [ 0, 1 ] 15618 2.189% 2.189% ( 1, 2 ] 359208 50.338% 52.526% ########## ( 2, 3 ] 165309 23.166% 75.692% ##### ( 3, 4 ] 75609 10.595% 86.287% ## ( 4, 6 ] 75909 10.638% 96.925% ## ( 6, 10 ] 15712 2.202% 99.127% ( 10, 15 ] 3819 0.535% 99.662% ( 15, 22 ] 442 0.062% 99.724% ( 22, 34 ] 37 0.005% 99.729% ( 34, 51 ] 37 0.005% 99.734% ( 51, 76 ] 43 0.006% 99.740% ( 76, 110 ] 12 0.002% 99.742% ( 110, 170 ] 42 0.006% 99.748% ( 170, 250 ] 78 0.011% 99.759% ( 250, 380 ] 193 0.027% 99.786% ( 380, 580 ] 396 0.055% 99.841% ( 580, 870 ] 651 0.091% 99.933% ( 870, 1300 ] 721 0.101% 100.034% ( 1300, 1900 ] 538 0.075% 100.109% ( 1900, 2900 ] 458 0.064% 100.173% ( 2900, 4400 ] 447 0.063% 100.236% ( 4400, 6600 ] 121 0.017% 100.253% ( 6600, 9900 ] 25 0.004% 100.256% ( 9900, 14000 ] 6 0.001% 100.257% ( 14000, 22000 ] 13 0.002% 100.259% ( 22000, 33000 ] 4 0.001% 100.260% ( 33000, 50000 ] 1 0.000% 100.260% ** Level 4 read latency histogram (micros): Count: 243862 Average: 31.7575 StdDev: 329.47 Min: 1 Median: 1.9863 Max: 34968 Percentiles: P50: 1.99 P75: 3.11 P99: 851.85 P99.9: 3669.34 P99.99: 4360.97 ------------------------------------------------------ [ 0, 1 ] 5025 2.061% 2.061% ( 1, 2 ] 118530 48.605% 50.666% ########## ( 2, 3 ] 56380 23.120% 73.786% ##### ( 3, 4 ] 26467 10.853% 84.639% ## ( 4, 6 ] 25540 10.473% 95.112% ## ( 6, 10 ] 6171 2.531% 97.643% # ( 10, 15 ] 1507 0.618% 98.260% ( 15, 22 ] 190 0.078% 98.338% ( 22, 34 ] 13 0.005% 98.344% ( 34, 51 ] 20 0.008% 98.352% ( 51, 76 ] 24 0.010% 98.362% ( 76, 110 ] 13 0.005% 98.367% ( 110, 170 ] 32 0.013% 98.380% ( 170, 250 ] 109 0.045% 98.425% ( 250, 380 ] 182 0.075% 98.500% ( 380, 580 ] 477 0.196% 98.695% ( 580, 870 ] 793 0.325% 99.020% ( 870, 1300 ] 840 0.344% 99.365% ( 1300, 1900 ] 578 0.237% 99.602% ( 1900, 2900 ] 483 0.198% 99.800% ( 2900, 4400 ] 476 0.195% 99.995% ( 4400, 6600 ] 178 0.073% 100.068% ( 6600, 9900 ] 20 0.008% 100.076% ( 9900, 14000 ] 6 0.002% 100.079% ( 14000, 22000 ] 12 0.005% 100.084% ( 22000, 33000 ] 6 0.002% 100.086% ( 33000, 50000 ] 1 0.000% 100.087% ** Level 5 read latency histogram (micros): Count: 12994 Average: 1765.8552 StdDev: 1779.63 Min: 82 Median: 1210.0042 Max: 39646 Percentiles: P50: 1210.00 P75: 2444.23 P99: 6792.29 P99.9: 25886.24 P99.99: 38955.10 ------------------------------------------------------ ( 76, 110 ] 9 0.069% 0.069% ( 110, 170 ] 60 0.462% 0.531% ( 170, 250 ] 212 1.632% 2.163% ( 250, 380 ] 642 4.941% 7.103% # ( 380, 580 ] 1416 10.897% 18.001% ## ( 580, 870 ] 2269 17.462% 35.463% ### ( 870, 1300 ] 2389 18.385% 53.848% #### ( 1300, 1900 ] 1838 14.145% 67.993% ### ( 1900, 2900 ] 1673 12.875% 80.868% ### ( 2900, 4400 ] 1681 12.937% 93.805% ### ( 4400, 6600 ] 669 5.149% 98.953% # ( 6600, 9900 ] 104 0.800% 99.754% ( 9900, 14000 ] 8 0.062% 99.815% ( 14000, 22000 ] 5 0.038% 99.854% ( 22000, 33000 ] 17 0.131% 99.985% ( 33000, 50000 ] 2 0.015% 100.000% ** DB Stats ** Uptime(secs): 8.1 total, 7.8 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 122.21 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.757 H:M:S, 46.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 125.46 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.757 H:M:S, 47.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/07/14-12:52:33 ... thread 0: (1000000,2000000) ops and (148051.2,137016.3) ops/second in (6.754418,14.596804) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/9 1.29 GB 31.5 0.4 0.0 0.4 2.2 1.8 0.0 1.2 23.6 141.7 16.00 6.94 23 0.696 383K 534 0.0 0.0 L1 2/2 146.76 MB 0.0 0.8 0.6 0.2 0.8 0.6 0.0 1.4 118.7 118.5 6.72 3.76 3 2.240 810K 1057 0.0 0.0 L2 4/3 241.80 MB 0.7 0.2 0.2 0.1 0.2 0.2 0.4 1.3 129.2 129.2 1.93 1.26 2 0.967 253K 177 0.0 0.0 L3 30/22 2.28 GB 0.6 5.6 3.8 1.8 5.6 3.8 0.1 1.5 107.1 106.4 53.60 26.42 22 2.436 3240K 3588 0.0 0.0 L4 180/0 9.73 GB 1.0 6.4 3.8 2.6 6.3 3.7 0.3 1.6 115.1 113.4 56.65 31.20 39 1.453 4094K 5725 0.0 0.0 L5 1148/0 94.37 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1377/36 108.05 GB 0.0 13.4 8.3 5.0 15.1 10.1 4.7 8.2 101.4 114.4 134.91 69.57 89 1.516 8783K 11K 0.0 0.0 Int 0/0 0.00 KB 0.0 13.4 8.3 5.0 15.1 10.1 4.7 8.2 101.4 114.4 134.91 69.57 89 1.516 8783K 11K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.4 8.3 5.0 13.2 8.2 0.0 0.0 113.2 112.1 120.91 64.11 67 1.805 8783K 11K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 135.0 14.00 5.46 22 0.636 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 14.8 total, 14.6 interval Flush(GB): cumulative 1.845, interval 1.845 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 15.08 GB write, 1042.62 MB/s write, 13.36 GB read, 924.22 MB/s read, 134.9 seconds Interval compaction: 15.08 GB write, 1057.55 MB/s write, 13.36 GB read, 937.45 MB/s read, 134.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 53, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 53, interval: 53 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 391349 Average: 10.5657 StdDev: 181.48 Min: 1 Median: 1.8668 Max: 34810 Percentiles: P50: 1.87 P75: 2.84 P99: 10.75 P99.9: 1801.08 P99.99: 4114.78 ------------------------------------------------------ [ 0, 1 ] 14615 3.735% 3.735% # ( 1, 2 ] 208874 53.373% 57.107% ########### ( 2, 3 ] 83457 21.325% 78.433% #### ( 3, 4 ] 36819 9.408% 87.841% ## ( 4, 6 ] 36132 9.233% 97.074% ## ( 6, 10 ] 7292 1.863% 98.937% ( 10, 15 ] 1652 0.422% 99.359% ( 15, 22 ] 170 0.043% 99.403% ( 22, 34 ] 5 0.001% 99.404% ( 34, 51 ] 13 0.003% 99.407% ( 51, 76 ] 39 0.010% 99.417% ( 76, 110 ] 10 0.003% 99.420% ( 110, 170 ] 48 0.012% 99.432% ( 170, 250 ] 127 0.032% 99.464% ( 250, 380 ] 221 0.056% 99.521% ( 380, 580 ] 348 0.089% 99.610% ( 580, 870 ] 473 0.121% 99.731% ( 870, 1300 ] 438 0.112% 99.843% ( 1300, 1900 ] 269 0.069% 99.911% ( 1900, 2900 ] 171 0.044% 99.955% ( 2900, 4400 ] 169 0.043% 99.998% ( 4400, 6600 ] 38 0.010% 100.008% ( 6600, 9900 ] 5 0.001% 100.009% ( 9900, 14000 ] 4 0.001% 100.010% ( 14000, 22000 ] 6 0.002% 100.012% ( 22000, 33000 ] 4 0.001% 100.013% ( 33000, 50000 ] 1 0.000% 100.013% ** Level 1 read latency histogram (micros): Count: 213712 Average: 9.8607 StdDev: 163.92 Min: 1 Median: 1.9366 Max: 31299 Percentiles: P50: 1.94 P75: 2.94 P99: 10.70 P99.9: 2038.98 P99.99: 4451.24 ------------------------------------------------------ [ 0, 1 ] 4471 2.092% 2.092% ( 1, 2 ] 109320 51.153% 53.245% ########## ( 2, 3 ] 49350 23.092% 76.337% ##### ( 3, 4 ] 21956 10.274% 86.610% ## ( 4, 6 ] 21729 10.167% 96.778% ## ( 6, 10 ] 4590 2.148% 98.926% ( 10, 15 ] 1136 0.532% 99.457% ( 15, 22 ] 128 0.060% 99.517% ( 22, 34 ] 24 0.011% 99.528% ( 34, 51 ] 11 0.005% 99.533% ( 51, 76 ] 10 0.005% 99.538% ( 76, 110 ] 5 0.002% 99.541% ( 110, 170 ] 16 0.007% 99.548% ( 170, 250 ] 35 0.016% 99.564% ( 250, 380 ] 70 0.033% 99.597% ( 380, 580 ] 155 0.073% 99.670% ( 580, 870 ] 190 0.089% 99.759% ( 870, 1300 ] 171 0.080% 99.839% ( 1300, 1900 ] 116 0.054% 99.893% ( 1900, 2900 ] 110 0.051% 99.944% ( 2900, 4400 ] 97 0.045% 99.990% ( 4400, 6600 ] 27 0.013% 100.002% ( 6600, 9900 ] 2 0.001% 100.003% ( 9900, 14000 ] 2 0.001% 100.004% ( 14000, 22000 ] 2 0.001% 100.005% ( 22000, 33000 ] 2 0.001% 100.006% ** Level 2 read latency histogram (micros): Count: 848664 Average: 14.8539 StdDev: 210.42 Min: 1 Median: 2.0090 Max: 36392 Percentiles: P50: 2.01 P75: 3.10 P99: 12.83 P99.9: 1134.40 P99.99: 1527.58 ------------------------------------------------------ [ 0, 1 ] 18355 2.163% 2.163% ( 1, 2 ] 404139 47.621% 49.783% ########## ( 2, 3 ] 204336 24.077% 73.861% ##### ( 3, 4 ] 92331 10.880% 84.740% ## ( 4, 6 ] 93732 11.045% 95.785% ## ( 6, 10 ] 23343 2.751% 98.536% # ( 10, 15 ] 6973 0.822% 99.357% ( 15, 22 ] 1006 0.119% 99.476% ( 22, 34 ] 114 0.013% 99.489% ( 34, 51 ] 55 0.006% 99.496% ( 51, 76 ] 60 0.007% 99.503% ( 76, 110 ] 35 0.004% 99.507% ( 110, 170 ] 99 0.012% 99.519% ( 170, 250 ] 219 0.026% 99.544% ( 250, 380 ] 436 0.051% 99.596% ( 380, 580 ] 787 0.093% 99.688% ( 580, 870 ] 1087 0.128% 99.817% ( 870, 1300 ] 1152 0.136% 99.952% ( 1300, 1900 ] 844 0.099% 100.052% ( 1900, 2900 ] 723 0.085% 100.137% ( 2900, 4400 ] 709 0.084% 100.220% ( 4400, 6600 ] 192 0.023% 100.243% ( 6600, 9900 ] 39 0.005% 100.248% ( 9900, 14000 ] 14 0.002% 100.249% ( 14000, 22000 ] 11 0.001% 100.251% ( 22000, 33000 ] 9 0.001% 100.252% ( 33000, 50000 ] 2 0.000% 100.252% ** Level 3 read latency histogram (micros): Count: 1634079 Average: 12.0771 StdDev: 208.52 Min: 1 Median: 1.9718 Max: 46665 Percentiles: P50: 1.97 P75: 2.98 P99: 9.92 P99.9: 658.05 P99.99: 945.56 ------------------------------------------------------ [ 0, 1 ] 27683 1.694% 1.694% ( 1, 2 ] 812261 49.708% 51.402% ########## ( 2, 3 ] 391957 23.986% 75.388% ##### ( 3, 4 ] 173565 10.622% 86.010% ## ( 4, 6 ] 174428 10.674% 96.684% ## ( 6, 10 ] 38637 2.364% 99.049% ( 10, 15 ] 9811 0.600% 99.649% ( 15, 22 ] 1153 0.071% 99.719% ( 22, 34 ] 104 0.006% 99.726% ( 34, 51 ] 117 0.007% 99.733% ( 51, 76 ] 159 0.010% 99.743% ( 76, 110 ] 55 0.003% 99.746% ( 110, 170 ] 148 0.009% 99.755% ( 170, 250 ] 291 0.018% 99.773% ( 250, 380 ] 590 0.036% 99.809% ( 380, 580 ] 1051 0.064% 99.873% ( 580, 870 ] 1616 0.099% 99.972% ( 870, 1300 ] 1648 0.101% 100.073% ( 1300, 1900 ] 1099 0.067% 100.140% ( 1900, 2900 ] 991 0.061% 100.201% ( 2900, 4400 ] 1048 0.064% 100.265% ( 4400, 6600 ] 266 0.016% 100.281% ( 6600, 9900 ] 59 0.004% 100.285% ( 9900, 14000 ] 22 0.001% 100.286% ( 14000, 22000 ] 41 0.003% 100.289% ( 22000, 33000 ] 17 0.001% 100.290% ( 33000, 50000 ] 6 0.000% 100.290% ** Level 4 read latency histogram (micros): Count: 671039 Average: 24.2393 StdDev: 276.51 Min: 1 Median: 1.9749 Max: 34968 Percentiles: P50: 1.97 P75: 3.03 P99: 559.70 P99.9: 3131.66 P99.99: 4064.62 ------------------------------------------------------ [ 0, 1 ] 12205 1.819% 1.819% ( 1, 2 ] 331622 49.419% 51.238% ########## ( 2, 3 ] 157334 23.446% 74.684% ##### ( 3, 4 ] 70522 10.509% 85.194% ## ( 4, 6 ] 69725 10.391% 95.584% ## ( 6, 10 ] 16071 2.395% 97.979% ( 10, 15 ] 4084 0.609% 98.588% ( 15, 22 ] 490 0.073% 98.661% ( 22, 34 ] 45 0.007% 98.668% ( 34, 51 ] 70 0.010% 98.678% ( 51, 76 ] 90 0.013% 98.691% ( 76, 110 ] 36 0.005% 98.697% ( 110, 170 ] 108 0.016% 98.713% ( 170, 250 ] 344 0.051% 98.764% ( 250, 380 ] 561 0.084% 98.848% ( 380, 580 ] 1137 0.169% 99.017% ( 580, 870 ] 1862 0.277% 99.295% ( 870, 1300 ] 1783 0.266% 99.560% ( 1300, 1900 ] 1150 0.171% 99.732% ( 1900, 2900 ] 979 0.146% 99.878% ( 2900, 4400 ] 971 0.145% 100.022% ( 4400, 6600 ] 279 0.042% 100.064% ( 6600, 9900 ] 51 0.008% 100.072% ( 9900, 14000 ] 16 0.002% 100.074% ( 14000, 22000 ] 22 0.003% 100.077% ( 22000, 33000 ] 12 0.002% 100.079% ( 33000, 50000 ] 1 0.000% 100.079% ** Level 5 read latency histogram (micros): Count: 29804 Average: 1461.9953 StdDev: 1707.51 Min: 82 Median: 966.1842 Max: 44023 Percentiles: P50: 966.18 P75: 1814.45 P99: 6393.94 P99.9: 24524.30 P99.99: 43233.32 ------------------------------------------------------ ( 76, 110 ] 61 0.205% 0.205% ( 110, 170 ] 400 1.342% 1.547% ( 170, 250 ] 990 3.322% 4.868% # ( 250, 380 ] 2121 7.116% 11.985% # ( 380, 580 ] 4138 13.884% 25.869% ### ( 580, 870 ] 5934 19.910% 45.779% #### ( 870, 1300 ] 5624 18.870% 64.649% #### ( 1300, 1900 ] 3598 12.072% 76.721% ## ( 1900, 2900 ] 2879 9.660% 86.381% ## ( 2900, 4400 ] 2880 9.663% 96.044% ## ( 4400, 6600 ] 972 3.261% 99.305% # ( 6600, 9900 ] 132 0.443% 99.748% ( 9900, 14000 ] 17 0.057% 99.805% ( 14000, 22000 ] 22 0.074% 99.879% ( 22000, 33000 ] 27 0.091% 99.970% ( 33000, 50000 ] 10 0.034% 100.003% ** DB Stats ** Uptime(secs): 14.8 total, 6.8 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 132.93 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.474 H:M:S, 43.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 145.71 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.717 H:M:S, 40.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 5 num-running-flushes: 1 2024/07/14-12:52:38 ... thread 0: (1000000,3000000) ops and (195193.7,152130.4) ops/second in (5.123116,19.719920) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/11 1.54 GB 6.3 1.2 0.0 1.2 4.0 2.8 0.0 1.4 51.2 166.3 24.61 13.00 39 0.631 1279K 1480 0.0 0.0 L1 6/5 424.92 MB 0.0 1.6 1.2 0.4 1.6 1.2 0.0 1.3 130.3 130.0 12.57 7.68 4 3.141 1662K 3667 0.0 0.0 L2 3/1 322.86 MB 1.9 0.2 0.2 0.1 0.2 0.2 0.8 1.3 129.2 129.2 1.93 1.26 2 0.967 253K 177 0.0 0.0 L3 17/5 1.15 GB 0.7 7.6 4.0 3.5 7.5 4.0 0.2 1.9 114.6 113.9 67.59 35.82 26 2.599 4494K 7914 0.0 0.0 L4 188/14 10.13 GB 1.0 9.1 4.9 4.2 8.9 4.7 0.7 1.8 120.6 118.3 77.17 43.85 54 1.429 5414K 5725 0.0 0.0 L5 1160/0 95.33 GB 1.0 0.0 0.0 0.0 0.0 0.0 4.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1386/36 108.88 GB 0.0 19.7 10.3 9.4 22.3 12.9 6.6 8.0 109.9 124.0 183.87 101.61 125 1.471 13M 18K 0.0 0.0 Int 0/0 0.00 KB 0.0 19.7 10.3 9.4 22.3 12.9 6.6 8.0 109.9 124.0 183.87 101.61 125 1.471 13M 18K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 19.7 10.3 9.4 19.5 10.1 0.0 0.0 122.3 121.0 165.10 93.59 90 1.834 13M 18K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 151.0 18.77 8.02 35 0.536 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 19.9 total, 19.7 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 22.27 GB write, 1144.39 MB/s write, 19.73 GB read, 1013.46 MB/s read, 183.9 seconds Interval compaction: 22.27 GB write, 1156.52 MB/s write, 19.73 GB read, 1024.20 MB/s read, 183.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 69, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 69, interval: 69 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 728365 Average: 9.9548 StdDev: 158.06 Min: 1 Median: 1.8445 Max: 34810 Percentiles: P50: 1.84 P75: 2.78 P99: 10.17 P99.9: 1638.70 P99.99: 3648.27 ------------------------------------------------------ [ 0, 1 ] 29215 4.011% 4.011% # ( 1, 2 ] 396666 54.460% 58.471% ########### ( 2, 3 ] 153819 21.118% 79.589% #### ( 3, 4 ] 65823 9.037% 88.626% ## ( 4, 6 ] 63298 8.690% 97.317% ## ( 6, 10 ] 12172 1.671% 98.988% ( 10, 15 ] 2732 0.375% 99.363% ( 15, 22 ] 256 0.035% 99.398% ( 22, 34 ] 14 0.002% 99.400% ( 34, 51 ] 40 0.005% 99.406% ( 51, 76 ] 65 0.009% 99.414% ( 76, 110 ] 21 0.003% 99.417% ( 110, 170 ] 134 0.018% 99.436% ( 170, 250 ] 280 0.038% 99.474% ( 250, 380 ] 509 0.070% 99.544% ( 380, 580 ] 706 0.097% 99.641% ( 580, 870 ] 888 0.122% 99.763% ( 870, 1300 ] 750 0.103% 99.866% ( 1300, 1900 ] 444 0.061% 99.927% ( 1900, 2900 ] 317 0.044% 99.970% ( 2900, 4400 ] 291 0.040% 100.010% ( 4400, 6600 ] 65 0.009% 100.019% ( 6600, 9900 ] 7 0.001% 100.020% ( 9900, 14000 ] 5 0.001% 100.021% ( 14000, 22000 ] 10 0.001% 100.022% ( 22000, 33000 ] 4 0.001% 100.023% ( 33000, 50000 ] 1 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 264088 Average: 10.8909 StdDev: 171.38 Min: 1 Median: 1.9377 Max: 31299 Percentiles: P50: 1.94 P75: 2.94 P99: 11.61 P99.9: 2304.81 P99.99: 5129.28 ------------------------------------------------------ [ 0, 1 ] 5385 2.039% 2.039% ( 1, 2 ] 135071 51.146% 53.185% ########## ( 2, 3 ] 61246 23.192% 76.377% ##### ( 3, 4 ] 26845 10.165% 86.542% ## ( 4, 6 ] 26595 10.071% 96.612% ## ( 6, 10 ] 5848 2.214% 98.827% ( 10, 15 ] 1422 0.538% 99.365% ( 15, 22 ] 167 0.063% 99.429% ( 22, 34 ] 24 0.009% 99.438% ( 34, 51 ] 14 0.005% 99.443% ( 51, 76 ] 13 0.005% 99.448% ( 76, 110 ] 9 0.003% 99.451% ( 110, 170 ] 33 0.012% 99.464% ( 170, 250 ] 67 0.025% 99.489% ( 250, 380 ] 117 0.044% 99.533% ( 380, 580 ] 230 0.087% 99.621% ( 580, 870 ] 274 0.104% 99.724% ( 870, 1300 ] 242 0.092% 99.816% ( 1300, 1900 ] 162 0.061% 99.877% ( 1900, 2900 ] 148 0.056% 99.933% ( 2900, 4400 ] 136 0.051% 99.985% ( 4400, 6600 ] 41 0.016% 100.000% ( 6600, 9900 ] 3 0.001% 100.002% ( 9900, 14000 ] 4 0.002% 100.003% ( 14000, 22000 ] 3 0.001% 100.004% ( 22000, 33000 ] 2 0.001% 100.005% ** Level 2 read latency histogram (micros): Count: 860386 Average: 14.7914 StdDev: 209.42 Min: 1 Median: 2.0074 Max: 36392 Percentiles: P50: 2.01 P75: 3.10 P99: 12.84 P99.9: 1137.96 P99.99: 1536.44 ------------------------------------------------------ [ 0, 1 ] 18518 2.152% 2.152% ( 1, 2 ] 410144 47.670% 49.822% ########## ( 2, 3 ] 207143 24.076% 73.898% ##### ( 3, 4 ] 93458 10.862% 84.760% ## ( 4, 6 ] 94895 11.029% 95.789% ## ( 6, 10 ] 23623 2.746% 98.535% # ( 10, 15 ] 7038 0.818% 99.353% ( 15, 22 ] 1011 0.118% 99.470% ( 22, 34 ] 114 0.013% 99.484% ( 34, 51 ] 56 0.007% 99.490% ( 51, 76 ] 60 0.007% 99.497% ( 76, 110 ] 36 0.004% 99.501% ( 110, 170 ] 106 0.012% 99.514% ( 170, 250 ] 240 0.028% 99.542% ( 250, 380 ] 450 0.052% 99.594% ( 380, 580 ] 805 0.094% 99.687% ( 580, 870 ] 1102 0.128% 99.816% ( 870, 1300 ] 1166 0.136% 99.951% ( 1300, 1900 ] 850 0.099% 100.050% ( 1900, 2900 ] 729 0.085% 100.135% ( 2900, 4400 ] 711 0.083% 100.217% ( 4400, 6600 ] 194 0.023% 100.240% ( 6600, 9900 ] 39 0.005% 100.244% ( 9900, 14000 ] 14 0.002% 100.246% ( 14000, 22000 ] 11 0.001% 100.247% ( 22000, 33000 ] 9 0.001% 100.248% ( 33000, 50000 ] 2 0.000% 100.248% ** Level 3 read latency histogram (micros): Count: 2127522 Average: 11.2098 StdDev: 192.72 Min: 1 Median: 1.9665 Max: 46665 Percentiles: P50: 1.97 P75: 2.97 P99: 9.84 P99.9: 610.74 P99.99: 907.76 ------------------------------------------------------ [ 0, 1 ] 35568 1.672% 1.672% ( 1, 2 ] 1063841 50.004% 51.676% ########## ( 2, 3 ] 510869 24.012% 75.688% ##### ( 3, 4 ] 222883 10.476% 86.164% ## ( 4, 6 ] 225257 10.588% 96.752% ## ( 6, 10 ] 49785 2.340% 99.092% ( 10, 15 ] 12348 0.580% 99.672% ( 15, 22 ] 1400 0.066% 99.738% ( 22, 34 ] 119 0.006% 99.744% ( 34, 51 ] 176 0.008% 99.752% ( 51, 76 ] 232 0.011% 99.763% ( 76, 110 ] 73 0.003% 99.766% ( 110, 170 ] 208 0.010% 99.776% ( 170, 250 ] 392 0.018% 99.795% ( 250, 380 ] 750 0.035% 99.830% ( 380, 580 ] 1287 0.060% 99.890% ( 580, 870 ] 1948 0.092% 99.982% ( 870, 1300 ] 1973 0.093% 100.075% ( 1300, 1900 ] 1303 0.061% 100.136% ( 1900, 2900 ] 1200 0.056% 100.192% ( 2900, 4400 ] 1232 0.058% 100.250% ( 4400, 6600 ] 317 0.015% 100.265% ( 6600, 9900 ] 66 0.003% 100.268% ( 9900, 14000 ] 24 0.001% 100.269% ( 14000, 22000 ] 47 0.002% 100.271% ( 22000, 33000 ] 17 0.001% 100.272% ( 33000, 50000 ] 6 0.000% 100.273% ** Level 4 read latency histogram (micros): Count: 1236432 Average: 20.2404 StdDev: 235.39 Min: 1 Median: 1.9642 Max: 34968 Percentiles: P50: 1.96 P75: 2.99 P99: 295.25 P99.9: 2364.43 P99.99: 3268.34 ------------------------------------------------------ [ 0, 1 ] 21226 1.717% 1.717% ( 1, 2 ] 619165 50.077% 51.793% ########## ( 2, 3 ] 291077 23.542% 75.335% ##### ( 3, 4 ] 126838 10.258% 85.594% ## ( 4, 6 ] 125394 10.142% 95.735% ## ( 6, 10 ] 30076 2.432% 98.168% ( 10, 15 ] 7541 0.610% 98.778% ( 15, 22 ] 816 0.066% 98.844% ( 22, 34 ] 68 0.005% 98.849% ( 34, 51 ] 119 0.010% 98.859% ( 51, 76 ] 166 0.013% 98.872% ( 76, 110 ] 82 0.007% 98.879% ( 110, 170 ] 315 0.025% 98.904% ( 170, 250 ] 761 0.062% 98.966% ( 250, 380 ] 1220 0.099% 99.064% ( 380, 580 ] 2112 0.171% 99.235% ( 580, 870 ] 3063 0.248% 99.483% ( 870, 1300 ] 2778 0.225% 99.708% ( 1300, 1900 ] 1721 0.139% 99.847% ( 1900, 2900 ] 1418 0.115% 99.962% ( 2900, 4400 ] 1439 0.116% 100.078% ( 4400, 6600 ] 400 0.032% 100.110% ( 6600, 9900 ] 69 0.006% 100.116% ( 9900, 14000 ] 18 0.001% 100.117% ( 14000, 22000 ] 25 0.002% 100.119% ( 22000, 33000 ] 12 0.001% 100.120% ( 33000, 50000 ] 1 0.000% 100.120% ** Level 5 read latency histogram (micros): Count: 45322 Average: 1297.0017 StdDev: 1531.24 Min: 80 Median: 840.4606 Max: 44023 Percentiles: P50: 840.46 P75: 1591.90 P99: 6168.09 P99.9: 19514.13 P99.99: 38895.26 ------------------------------------------------------ ( 76, 110 ] 131 0.289% 0.289% ( 110, 170 ] 933 2.059% 2.348% ( 170, 250 ] 2222 4.903% 7.250% # ( 250, 380 ] 4139 9.132% 16.383% ## ( 380, 580 ] 7027 15.505% 31.887% ### ( 580, 870 ] 9140 20.167% 52.054% #### ( 870, 1300 ] 8039 17.738% 69.792% #### ( 1300, 1900 ] 4852 10.706% 80.497% ## ( 1900, 2900 ] 3854 8.504% 89.001% ## ( 2900, 4400 ] 3618 7.983% 96.984% ## ( 4400, 6600 ] 1137 2.509% 99.493% # ( 6600, 9900 ] 145 0.320% 99.812% ( 9900, 14000 ] 19 0.042% 99.854% ( 14000, 22000 ] 30 0.066% 99.921% ( 22000, 33000 ] 28 0.062% 99.982% ( 33000, 50000 ] 10 0.022% 100.004% ** DB Stats ** Uptime(secs): 19.9 total, 5.1 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 148.14 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.633 H:M:S, 38.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 192.11 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.158 H:M:S, 22.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 5 num-running-flushes: 1 2024/07/14-12:52:42 ... thread 0: (1000000,4000000) ops and (275674.0,171325.3) ops/second in (3.627473,23.347393) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/11 2.52 GB 107.0 2.2 0.0 2.2 6.0 3.8 0.0 1.6 71.0 191.2 31.94 19.64 59 0.541 2302K 2432 0.0 0.0 L1 6/5 424.92 MB 0.0 1.6 1.2 0.4 1.6 1.2 0.0 1.3 130.3 130.0 12.57 7.68 4 3.141 1662K 3667 0.0 0.0 L2 1/0 65.32 MB 0.7 0.2 0.2 0.1 0.2 0.2 0.8 1.3 129.2 129.2 1.93 1.26 2 0.967 253K 177 0.0 0.0 L3 16/0 997.06 MB 1.0 8.1 4.3 3.9 8.1 4.2 0.2 1.9 118.0 117.3 70.51 38.42 28 2.518 5074K 9949 0.0 0.0 L4 181/1 9.82 GB 1.0 10.1 5.2 4.9 9.9 5.0 0.8 1.9 124.7 122.4 82.68 48.29 57 1.451 5992K 9290 0.0 0.0 L5 1172/5 96.06 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1392/22 109.85 GB 0.0 22.3 10.9 11.4 25.8 14.4 7.4 6.9 114.1 132.2 199.63 115.29 150 1.331 15M 25K 0.0 0.0 Int 0/0 0.00 KB 0.0 22.3 10.9 11.4 25.8 14.4 7.4 6.9 114.1 132.2 199.63 115.29 150 1.331 15M 25K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 22.3 10.9 11.4 22.0 10.6 0.0 0.0 128.4 127.0 177.45 104.42 99 1.792 15M 25K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 173.2 22.18 10.87 51 0.435 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 23.6 total, 23.3 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 25.76 GB write, 1119.89 MB/s write, 22.25 GB read, 967.24 MB/s read, 199.6 seconds Interval compaction: 25.76 GB write, 1129.91 MB/s write, 22.25 GB read, 975.90 MB/s read, 199.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 69, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 69, interval: 69 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1085403 Average: 10.0110 StdDev: 134.84 Min: 1 Median: 1.7721 Max: 34810 Percentiles: P50: 1.77 P75: 2.63 P99: 154.52 P99.9: 1185.58 P99.99: 2658.65 ------------------------------------------------------ [ 0, 1 ] 68501 6.311% 6.311% # ( 1, 2 ] 614136 56.581% 62.892% ########### ( 2, 3 ] 208959 19.252% 82.144% #### ( 3, 4 ] 84360 7.772% 89.916% ## ( 4, 6 ] 76648 7.062% 96.978% # ( 6, 10 ] 14990 1.381% 98.359% ( 10, 15 ] 3380 0.311% 98.671% ( 15, 22 ] 368 0.034% 98.705% ( 22, 34 ] 28 0.003% 98.707% ( 34, 51 ] 43 0.004% 98.711% ( 51, 76 ] 66 0.006% 98.717% ( 76, 110 ] 612 0.056% 98.774% ( 110, 170 ] 3314 0.305% 99.079% ( 170, 250 ] 2779 0.256% 99.335% ( 250, 380 ] 2061 0.190% 99.525% ( 380, 580 ] 1929 0.178% 99.703% ( 580, 870 ] 1476 0.136% 99.838% ( 870, 1300 ] 911 0.084% 99.922% ( 1300, 1900 ] 475 0.044% 99.966% ( 1900, 2900 ] 342 0.032% 99.998% ( 2900, 4400 ] 302 0.028% 100.026% ( 4400, 6600 ] 66 0.006% 100.032% ( 6600, 9900 ] 7 0.001% 100.032% ( 9900, 14000 ] 6 0.001% 100.033% ( 14000, 22000 ] 10 0.001% 100.034% ( 22000, 33000 ] 4 0.000% 100.034% ( 33000, 50000 ] 1 0.000% 100.034% ** Level 1 read latency histogram (micros): Count: 314402 Average: 10.3144 StdDev: 158.47 Min: 1 Median: 1.9222 Max: 31299 Percentiles: P50: 1.92 P75: 2.91 P99: 12.64 P99.9: 1976.30 P99.99: 4698.33 ------------------------------------------------------ [ 0, 1 ] 6791 2.160% 2.160% ( 1, 2 ] 163099 51.876% 54.036% ########## ( 2, 3 ] 72568 23.081% 77.117% ##### ( 3, 4 ] 30816 9.801% 86.919% ## ( 4, 6 ] 30046 9.557% 96.475% ## ( 6, 10 ] 7076 2.251% 98.726% ( 10, 15 ] 1633 0.519% 99.245% ( 15, 22 ] 206 0.066% 99.311% ( 22, 34 ] 29 0.009% 99.320% ( 34, 51 ] 15 0.005% 99.325% ( 51, 76 ] 15 0.005% 99.330% ( 76, 110 ] 38 0.012% 99.342% ( 110, 170 ] 195 0.062% 99.404% ( 170, 250 ] 219 0.070% 99.473% ( 250, 380 ] 216 0.069% 99.542% ( 380, 580 ] 345 0.110% 99.652% ( 580, 870 ] 339 0.108% 99.760% ( 870, 1300 ] 266 0.085% 99.844% ( 1300, 1900 ] 164 0.052% 99.896% ( 1900, 2900 ] 152 0.048% 99.945% ( 2900, 4400 ] 137 0.044% 99.988% ( 4400, 6600 ] 41 0.013% 100.001% ( 6600, 9900 ] 3 0.001% 100.002% ( 9900, 14000 ] 4 0.001% 100.003% ( 14000, 22000 ] 3 0.001% 100.004% ( 22000, 33000 ] 2 0.001% 100.005% ** Level 2 read latency histogram (micros): Count: 860468 Average: 14.8200 StdDev: 209.44 Min: 1 Median: 2.0076 Max: 36392 Percentiles: P50: 2.01 P75: 3.10 P99: 12.90 P99.9: 1138.07 P99.99: 1536.44 ------------------------------------------------------ [ 0, 1 ] 18518 2.152% 2.152% ( 1, 2 ] 410144 47.665% 49.817% ########## ( 2, 3 ] 207143 24.073% 73.891% ##### ( 3, 4 ] 93458 10.861% 84.752% ## ( 4, 6 ] 94895 11.028% 95.780% ## ( 6, 10 ] 23623 2.745% 98.526% # ( 10, 15 ] 7038 0.818% 99.343% ( 15, 22 ] 1011 0.117% 99.461% ( 22, 34 ] 114 0.013% 99.474% ( 34, 51 ] 56 0.007% 99.481% ( 51, 76 ] 60 0.007% 99.488% ( 76, 110 ] 44 0.005% 99.493% ( 110, 170 ] 123 0.014% 99.507% ( 170, 250 ] 259 0.030% 99.537% ( 250, 380 ] 459 0.053% 99.591% ( 380, 580 ] 825 0.096% 99.686% ( 580, 870 ] 1110 0.129% 99.815% ( 870, 1300 ] 1167 0.136% 99.951% ( 1300, 1900 ] 850 0.099% 100.050% ( 1900, 2900 ] 729 0.085% 100.135% ( 2900, 4400 ] 711 0.083% 100.217% ( 4400, 6600 ] 194 0.023% 100.240% ( 6600, 9900 ] 39 0.005% 100.244% ( 9900, 14000 ] 14 0.002% 100.246% ( 14000, 22000 ] 11 0.001% 100.247% ( 22000, 33000 ] 9 0.001% 100.248% ( 33000, 50000 ] 2 0.000% 100.248% ** Level 3 read latency histogram (micros): Count: 2223357 Average: 10.9804 StdDev: 188.90 Min: 1 Median: 1.9577 Max: 46665 Percentiles: P50: 1.96 P75: 2.96 P99: 9.86 P99.9: 608.02 P99.99: 906.87 ------------------------------------------------------ [ 0, 1 ] 38314 1.723% 1.723% ( 1, 2 ] 1120782 50.409% 52.133% ########## ( 2, 3 ] 531481 23.904% 76.037% ##### ( 3, 4 ] 229714 10.332% 86.369% ## ( 4, 6 ] 231341 10.405% 96.774% ## ( 6, 10 ] 51314 2.308% 99.082% ( 10, 15 ] 12620 0.568% 99.650% ( 15, 22 ] 1444 0.065% 99.715% ( 22, 34 ] 124 0.006% 99.720% ( 34, 51 ] 189 0.009% 99.729% ( 51, 76 ] 236 0.011% 99.739% ( 76, 110 ] 112 0.005% 99.744% ( 110, 170 ] 412 0.019% 99.763% ( 170, 250 ] 565 0.025% 99.788% ( 250, 380 ] 880 0.040% 99.828% ( 380, 580 ] 1410 0.063% 99.891% ( 580, 870 ] 2025 0.091% 99.982% ( 870, 1300 ] 2002 0.090% 100.072% ( 1300, 1900 ] 1312 0.059% 100.131% ( 1900, 2900 ] 1206 0.054% 100.186% ( 2900, 4400 ] 1237 0.056% 100.241% ( 4400, 6600 ] 318 0.014% 100.256% ( 6600, 9900 ] 67 0.003% 100.259% ( 9900, 14000 ] 24 0.001% 100.260% ( 14000, 22000 ] 47 0.002% 100.262% ( 22000, 33000 ] 17 0.001% 100.262% ( 33000, 50000 ] 6 0.000% 100.263% ** Level 4 read latency histogram (micros): Count: 1393195 Average: 19.4975 StdDev: 223.45 Min: 1 Median: 1.9244 Max: 34968 Percentiles: P50: 1.92 P75: 2.93 P99: 315.52 P99.9: 2252.64 P99.99: 3232.31 ------------------------------------------------------ [ 0, 1 ] 33902 2.433% 2.433% ( 1, 2 ] 716869 51.455% 53.888% ########## ( 2, 3 ] 315368 22.636% 76.525% ##### ( 3, 4 ] 134336 9.642% 86.167% ## ( 4, 6 ] 130949 9.399% 95.566% ## ( 6, 10 ] 32802 2.354% 97.921% ( 10, 15 ] 7941 0.570% 98.491% ( 15, 22 ] 859 0.062% 98.552% ( 22, 34 ] 72 0.005% 98.557% ( 34, 51 ] 122 0.009% 98.566% ( 51, 76 ] 168 0.012% 98.578% ( 76, 110 ] 416 0.030% 98.608% ( 110, 170 ] 2144 0.154% 98.762% ( 170, 250 ] 2247 0.161% 98.923% ( 250, 380 ] 2121 0.152% 99.076% ( 380, 580 ] 2852 0.205% 99.280% ( 580, 870 ] 3484 0.250% 99.530% ( 870, 1300 ] 2901 0.208% 99.739% ( 1300, 1900 ] 1742 0.125% 99.864% ( 1900, 2900 ] 1440 0.103% 99.967% ( 2900, 4400 ] 1452 0.104% 100.071% ( 4400, 6600 ] 402 0.029% 100.100% ( 6600, 9900 ] 69 0.005% 100.105% ( 9900, 14000 ] 18 0.001% 100.106% ( 14000, 22000 ] 25 0.002% 100.108% ( 22000, 33000 ] 12 0.001% 100.109% ( 33000, 50000 ] 1 0.000% 100.109% ** Level 5 read latency histogram (micros): Count: 76642 Average: 892.6203 StdDev: 1291.27 Min: 77 Median: 502.8117 Max: 44023 Percentiles: P50: 502.81 P75: 1048.45 P99: 5571.00 P99.9: 10793.39 P99.99: 31167.64 ------------------------------------------------------ ( 76, 110 ] 2036 2.657% 2.657% # ( 110, 170 ] 10463 13.652% 16.308% ### ( 170, 250 ] 9910 12.930% 29.239% ### ( 250, 380 ] 9037 11.791% 41.030% ## ( 380, 580 ] 11196 14.608% 55.638% ### ( 580, 870 ] 11229 14.651% 70.289% ### ( 870, 1300 ] 8700 11.351% 81.641% ## ( 1300, 1900 ] 5006 6.532% 88.172% # ( 1900, 2900 ] 3997 5.215% 93.387% # ( 2900, 4400 ] 3690 4.815% 98.202% # ( 4400, 6600 ] 1149 1.499% 99.701% ( 6600, 9900 ] 148 0.193% 99.894% ( 9900, 14000 ] 20 0.026% 99.920% ( 14000, 22000 ] 30 0.039% 99.960% ( 22000, 33000 ] 28 0.037% 99.996% ( 33000, 50000 ] 10 0.013% 100.009% ** DB Stats ** Uptime(secs): 23.6 total, 3.6 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 167.11 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.633 H:M:S, 32.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 271.32 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/07/14-12:52:47 ... thread 0: (1000000,5000000) ops and (199245.3,176265.3) ops/second in (5.018940,28.366333) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/14 3.50 GB 75.5 3.4 0.0 3.4 8.2 4.7 0.0 1.7 88.6 210.3 39.83 27.05 80 0.498 3582K 3696 0.0 0.0 L1 6/5 424.92 MB 0.0 1.6 1.2 0.4 1.6 1.2 0.0 1.3 130.3 130.0 12.57 7.68 4 3.141 1662K 3667 0.0 0.0 L2 1/0 65.32 MB 0.7 0.2 0.2 0.1 0.2 0.2 0.8 1.3 129.2 129.2 1.93 1.26 2 0.967 253K 177 0.0 0.0 L3 16/0 997.06 MB 1.0 8.1 4.3 3.9 8.1 4.2 0.2 1.9 118.0 117.3 70.51 38.42 28 2.518 5074K 9949 0.0 0.0 L4 180/0 9.69 GB 1.0 10.1 5.2 4.9 9.9 5.0 0.8 1.9 124.7 122.4 82.68 48.29 57 1.451 5992K 9290 0.0 0.0 L5 1176/0 96.14 GB 1.0 0.6 0.1 0.4 0.5 0.1 5.6 4.1 253.3 232.1 2.25 2.08 1 2.249 578K 48K 0.0 0.0 Sum 1396/19 110.78 GB 0.0 24.0 11.0 13.1 28.5 15.4 7.4 6.0 117.3 139.1 209.77 124.77 172 1.220 17M 75K 0.0 0.0 Int 0/0 0.00 KB 0.0 24.0 11.0 13.1 28.5 15.4 7.4 6.0 117.3 139.1 209.77 124.77 172 1.220 17M 75K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 24.0 11.0 13.1 23.7 10.7 0.0 0.0 133.4 131.8 184.47 111.19 105 1.757 17M 75K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 191.7 25.30 13.59 67 0.378 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 28.6 total, 28.4 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 28.49 GB write, 1020.78 MB/s write, 24.04 GB read, 861.39 MB/s read, 209.8 seconds Interval compaction: 28.49 GB write, 1028.30 MB/s write, 24.04 GB read, 867.74 MB/s read, 209.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 4, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 4, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 69, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 4, total-stops: 69, interval: 73 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1426450 Average: 12.0330 StdDev: 122.22 Min: 0 Median: 1.6971 Max: 34810 Percentiles: P50: 1.70 P75: 2.49 P99: 225.28 P99.9: 969.40 P99.99: 2039.10 ------------------------------------------------------ [ 0, 1 ] 155926 10.931% 10.931% ## ( 1, 2 ] 799502 56.048% 66.979% ########### ( 2, 3 ] 233205 16.349% 83.328% ### ( 3, 4 ] 92639 6.494% 89.822% # ( 4, 6 ] 81357 5.703% 95.526% # ( 6, 10 ] 15931 1.117% 96.643% ( 10, 15 ] 3741 0.262% 96.905% ( 15, 22 ] 422 0.030% 96.935% ( 22, 34 ] 34 0.002% 96.937% ( 34, 51 ] 43 0.003% 96.940% ( 51, 76 ] 69 0.005% 96.945% ( 76, 110 ] 4614 0.323% 97.268% ( 110, 170 ] 18475 1.295% 98.563% ( 170, 250 ] 9013 0.632% 99.195% ( 250, 380 ] 4217 0.296% 99.491% ( 380, 580 ] 3610 0.253% 99.744% ( 580, 870 ] 2000 0.140% 99.884% ( 870, 1300 ] 980 0.069% 99.953% ( 1300, 1900 ] 480 0.034% 99.987% ( 1900, 2900 ] 362 0.025% 100.012% ( 2900, 4400 ] 307 0.022% 100.033% ( 4400, 6600 ] 66 0.005% 100.038% ( 6600, 9900 ] 7 0.000% 100.039% ( 9900, 14000 ] 6 0.000% 100.039% ( 14000, 22000 ] 10 0.001% 100.040% ( 22000, 33000 ] 4 0.000% 100.040% ( 33000, 50000 ] 1 0.000% 100.040% ** Level 1 read latency histogram (micros): Count: 329734 Average: 10.5196 StdDev: 155.18 Min: 1 Median: 1.9414 Max: 31299 Percentiles: P50: 1.94 P75: 2.96 P99: 17.51 P99.9: 1890.00 P99.99: 4616.06 ------------------------------------------------------ [ 0, 1 ] 7273 2.206% 2.206% ( 1, 2 ] 167400 50.768% 52.974% ########## ( 2, 3 ] 75311 22.840% 75.814% ##### ( 3, 4 ] 33192 10.066% 85.880% ## ( 4, 6 ] 33522 10.166% 96.047% ## ( 6, 10 ] 7820 2.372% 98.418% ( 10, 15 ] 1808 0.548% 98.966% ( 15, 22 ] 309 0.094% 99.060% ( 22, 34 ] 44 0.013% 99.073% ( 34, 51 ] 16 0.005% 99.078% ( 51, 76 ] 16 0.005% 99.083% ( 76, 110 ] 184 0.056% 99.139% ( 110, 170 ] 658 0.200% 99.339% ( 170, 250 ] 396 0.120% 99.459% ( 250, 380 ] 279 0.085% 99.543% ( 380, 580 ] 394 0.119% 99.663% ( 580, 870 ] 353 0.107% 99.770% ( 870, 1300 ] 268 0.081% 99.851% ( 1300, 1900 ] 164 0.050% 99.901% ( 1900, 2900 ] 153 0.046% 99.947% ( 2900, 4400 ] 137 0.042% 99.989% ( 4400, 6600 ] 41 0.012% 100.001% ( 6600, 9900 ] 3 0.001% 100.002% ( 9900, 14000 ] 4 0.001% 100.003% ( 14000, 22000 ] 3 0.001% 100.004% ( 22000, 33000 ] 2 0.001% 100.005% ** Level 2 read latency histogram (micros): Count: 860624 Average: 14.8504 StdDev: 209.44 Min: 1 Median: 2.0080 Max: 36392 Percentiles: P50: 2.01 P75: 3.10 P99: 13.01 P99.9: 1138.01 P99.99: 1536.43 ------------------------------------------------------ [ 0, 1 ] 18518 2.152% 2.152% ( 1, 2 ] 410144 47.657% 49.808% ########## ( 2, 3 ] 207143 24.069% 73.877% ##### ( 3, 4 ] 93458 10.859% 84.737% ## ( 4, 6 ] 94895 11.026% 95.763% ## ( 6, 10 ] 23623 2.745% 98.508% # ( 10, 15 ] 7038 0.818% 99.325% ( 15, 22 ] 1011 0.117% 99.443% ( 22, 34 ] 114 0.013% 99.456% ( 34, 51 ] 56 0.007% 99.463% ( 51, 76 ] 60 0.007% 99.470% ( 76, 110 ] 64 0.007% 99.477% ( 110, 170 ] 214 0.025% 99.502% ( 170, 250 ] 282 0.033% 99.535% ( 250, 380 ] 471 0.055% 99.589% ( 380, 580 ] 832 0.097% 99.686% ( 580, 870 ] 1113 0.129% 99.815% ( 870, 1300 ] 1167 0.136% 99.951% ( 1300, 1900 ] 850 0.099% 100.050% ( 1900, 2900 ] 729 0.085% 100.135% ( 2900, 4400 ] 711 0.083% 100.217% ( 4400, 6600 ] 194 0.023% 100.240% ( 6600, 9900 ] 39 0.005% 100.244% ( 9900, 14000 ] 14 0.002% 100.246% ( 14000, 22000 ] 11 0.001% 100.247% ( 22000, 33000 ] 9 0.001% 100.248% ( 33000, 50000 ] 2 0.000% 100.248% ** Level 3 read latency histogram (micros): Count: 2225225 Average: 11.1007 StdDev: 188.90 Min: 1 Median: 1.9585 Max: 46665 Percentiles: P50: 1.96 P75: 2.96 P99: 9.98 P99.9: 610.60 P99.99: 907.24 ------------------------------------------------------ [ 0, 1 ] 38314 1.722% 1.722% ( 1, 2 ] 1120806 50.368% 52.090% ########## ( 2, 3 ] 531566 23.888% 75.978% ##### ( 3, 4 ] 229789 10.327% 86.305% ## ( 4, 6 ] 231438 10.401% 96.705% ## ( 6, 10 ] 51329 2.307% 99.012% ( 10, 15 ] 12623 0.567% 99.579% ( 15, 22 ] 1445 0.065% 99.644% ( 22, 34 ] 124 0.006% 99.650% ( 34, 51 ] 189 0.008% 99.658% ( 51, 76 ] 236 0.011% 99.669% ( 76, 110 ] 320 0.014% 99.683% ( 110, 170 ] 1251 0.056% 99.740% ( 170, 250 ] 869 0.039% 99.779% ( 250, 380 ] 985 0.044% 99.823% ( 380, 580 ] 1500 0.067% 99.890% ( 580, 870 ] 2045 0.092% 99.982% ( 870, 1300 ] 2003 0.090% 100.072% ( 1300, 1900 ] 1313 0.059% 100.131% ( 1900, 2900 ] 1206 0.054% 100.185% ( 2900, 4400 ] 1237 0.056% 100.241% ( 4400, 6600 ] 318 0.014% 100.255% ( 6600, 9900 ] 67 0.003% 100.258% ( 9900, 14000 ] 24 0.001% 100.259% ( 14000, 22000 ] 47 0.002% 100.262% ( 22000, 33000 ] 17 0.001% 100.262% ( 33000, 50000 ] 6 0.000% 100.263% ** Level 4 read latency histogram (micros): Count: 1443934 Average: 20.3384 StdDev: 220.30 Min: 1 Median: 1.9198 Max: 34968 Percentiles: P50: 1.92 P75: 2.94 P99: 346.38 P99.9: 2218.86 P99.99: 3225.01 ------------------------------------------------------ [ 0, 1 ] 38082 2.637% 2.637% # ( 1, 2 ] 743522 51.493% 54.130% ########## ( 2, 3 ] 321027 22.233% 76.363% #### ( 3, 4 ] 135638 9.394% 85.757% ## ( 4, 6 ] 131718 9.122% 94.879% ## ( 6, 10 ] 33489 2.319% 97.198% ( 10, 15 ] 8034 0.556% 97.754% ( 15, 22 ] 866 0.060% 97.814% ( 22, 34 ] 72 0.005% 97.819% ( 34, 51 ] 122 0.008% 97.828% ( 51, 76 ] 169 0.012% 97.840% ( 76, 110 ] 2136 0.148% 97.988% ( 110, 170 ] 8022 0.556% 98.543% ( 170, 250 ] 4432 0.307% 98.850% ( 250, 380 ] 2921 0.202% 99.052% ( 380, 580 ] 3444 0.239% 99.291% ( 580, 870 ] 3657 0.253% 99.544% ( 870, 1300 ] 2931 0.203% 99.747% ( 1300, 1900 ] 1747 0.121% 99.868% ( 1900, 2900 ] 1446 0.100% 99.968% ( 2900, 4400 ] 1452 0.101% 100.069% ( 4400, 6600 ] 402 0.028% 100.097% ( 6600, 9900 ] 69 0.005% 100.101% ( 9900, 14000 ] 18 0.001% 100.103% ( 14000, 22000 ] 25 0.002% 100.104% ( 22000, 33000 ] 12 0.001% 100.105% ( 33000, 50000 ] 1 0.000% 100.105% ** Level 5 read latency histogram (micros): Count: 142265 Average: 566.0152 StdDev: 1015.11 Min: 1 Median: 222.3769 Max: 44023 Percentiles: P50: 222.38 P75: 574.73 P99: 4380.22 P99.9: 8444.77 P99.99: 27018.16 ------------------------------------------------------ [ 0, 1 ] 12 0.008% 0.008% ( 1, 2 ] 47 0.033% 0.041% ( 2, 3 ] 124 0.087% 0.129% ( 3, 4 ] 82 0.058% 0.186% ( 4, 6 ] 113 0.079% 0.266% ( 6, 10 ] 20 0.014% 0.280% ( 10, 15 ] 5 0.004% 0.283% ( 51, 76 ] 1 0.001% 0.284% ( 76, 110 ] 11742 8.254% 8.538% ## ( 110, 170 ] 44264 31.114% 39.651% ###### ( 170, 250 ] 22487 15.806% 55.458% ### ( 250, 380 ] 13464 9.464% 64.922% ## ( 380, 580 ] 14726 10.351% 75.273% ## ( 580, 870 ] 12189 8.568% 83.841% ## ( 870, 1300 ] 8868 6.233% 90.074% # ( 1300, 1900 ] 5026 3.533% 93.607% # ( 1900, 2900 ] 4031 2.833% 96.440% # ( 2900, 4400 ] 3690 2.594% 99.034% # ( 4400, 6600 ] 1149 0.808% 99.842% ( 6600, 9900 ] 148 0.104% 99.946% ( 9900, 14000 ] 20 0.014% 99.960% ( 14000, 22000 ] 30 0.021% 99.981% ( 22000, 33000 ] 28 0.020% 100.001% ( 33000, 50000 ] 10 0.007% 100.008% ** DB Stats ** Uptime(secs): 28.6 total, 5.0 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 172.20 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.063 H:M:S, 31.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 196.10 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.430 H:M:S, 28.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/14-12:53:02 ... thread 0: (1000000,6000000) ops and (65728.5,137676.4) ops/second in (15.214106,43.580439) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/7 4.47 GB 332.1 6.9 0.0 6.9 12.7 5.7 0.0 2.2 126.8 230.8 56.14 42.65 102 0.550 7225K 19K 0.0 0.0 L1 6/5 424.92 MB 0.0 1.6 1.2 0.4 1.6 1.2 0.0 1.3 130.3 130.0 12.57 7.68 4 3.141 1662K 3667 0.0 0.0 L2 1/0 65.32 MB 0.7 0.2 0.2 0.1 0.2 0.2 0.8 1.3 129.2 129.2 1.93 1.26 2 0.967 253K 177 0.0 0.0 L3 16/0 997.06 MB 1.0 8.1 4.3 3.9 8.1 4.2 0.2 1.9 118.0 117.3 70.51 38.42 28 2.518 5074K 9949 0.0 0.0 L4 180/0 9.69 GB 1.0 10.1 5.2 4.9 9.9 5.0 0.8 1.9 124.7 122.4 82.68 48.29 57 1.451 5992K 9290 0.0 0.0 L5 1176/0 96.14 GB 1.0 0.6 0.1 0.4 0.5 0.1 5.6 4.1 253.3 232.1 2.25 2.08 1 2.249 578K 48K 0.0 0.0 Sum 1391/12 111.75 GB 0.0 27.5 11.0 16.6 33.0 16.4 7.4 5.8 124.8 149.3 226.08 140.38 194 1.165 20M 91K 0.0 0.0 Int 0/0 0.00 KB 0.0 27.5 11.0 16.6 33.0 16.4 7.4 5.8 124.8 149.3 226.08 140.38 194 1.165 20M 91K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 27.5 11.0 16.6 27.2 10.7 0.0 0.0 142.7 141.2 197.60 124.06 111 1.780 20M 91K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 205.8 28.47 16.32 83 0.343 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 43.8 total, 43.6 interval Flush(GB): cumulative 5.721, interval 5.721 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 32.96 GB write, 770.74 MB/s write, 27.54 GB read, 644.06 MB/s read, 226.1 seconds Interval compaction: 32.96 GB write, 774.44 MB/s write, 27.54 GB read, 647.15 MB/s read, 226.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 69, pending-compaction-bytes-delays: 5, pending-compaction-bytes-stops: 0, total-delays: 18, total-stops: 69, interval: 87 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2458193 Average: 15.1550 StdDev: 101.92 Min: 0 Median: 1.5848 Max: 34810 Percentiles: P50: 1.58 P75: 2.20 P99: 238.28 P99.9: 745.72 P99.99: 1241.83 ------------------------------------------------------ [ 0, 1 ] 445542 18.125% 18.125% #### ( 1, 2 ] 1339895 54.507% 72.632% ########### ( 2, 3 ] 293576 11.943% 84.575% ## ( 3, 4 ] 112530 4.578% 89.153% # ( 4, 6 ] 91921 3.739% 92.892% # ( 6, 10 ] 17840 0.726% 93.618% ( 10, 15 ] 4299 0.175% 93.793% ( 15, 22 ] 519 0.021% 93.814% ( 22, 34 ] 45 0.002% 93.816% ( 34, 51 ] 50 0.002% 93.818% ( 51, 76 ] 80 0.003% 93.821% ( 76, 110 ] 21096 0.858% 94.679% ( 110, 170 ] 83124 3.382% 98.061% # ( 170, 250 ] 27060 1.101% 99.161% ( 250, 380 ] 9154 0.372% 99.534% ( 380, 580 ] 7323 0.298% 99.832% ( 580, 870 ] 2943 0.120% 99.951% ( 870, 1300 ] 1100 0.045% 99.996% ( 1300, 1900 ] 517 0.021% 100.017% ( 1900, 2900 ] 397 0.016% 100.033% ( 2900, 4400 ] 330 0.013% 100.047% ( 4400, 6600 ] 68 0.003% 100.049% ( 6600, 9900 ] 7 0.000% 100.050% ( 9900, 14000 ] 6 0.000% 100.050% ( 14000, 22000 ] 10 0.000% 100.050% ( 22000, 33000 ] 4 0.000% 100.051% ( 33000, 50000 ] 1 0.000% 100.051% ** Level 1 read latency histogram (micros): Count: 369306 Average: 11.3388 StdDev: 147.70 Min: 1 Median: 2.1058 Max: 31299 Percentiles: P50: 2.11 P75: 3.50 P99: 152.88 P99.9: 1742.52 P99.99: 4378.86 ------------------------------------------------------ [ 0, 1 ] 7368 1.995% 1.995% ( 1, 2 ] 168945 45.747% 47.742% ######### ( 2, 3 ] 78840 21.348% 69.090% #### ( 3, 4 ] 43814 11.864% 80.954% ## ( 4, 6 ] 50700 13.728% 94.682% ### ( 6, 10 ] 10074 2.728% 97.410% # ( 10, 15 ] 2375 0.643% 98.053% ( 15, 22 ] 803 0.217% 98.271% ( 22, 34 ] 108 0.029% 98.300% ( 34, 51 ] 21 0.006% 98.305% ( 51, 76 ] 16 0.004% 98.310% ( 76, 110 ] 748 0.203% 98.512% ( 110, 170 ] 2520 0.682% 99.195% ( 170, 250 ] 907 0.246% 99.440% ( 250, 380 ] 419 0.113% 99.554% ( 380, 580 ] 495 0.134% 99.688% ( 580, 870 ] 390 0.106% 99.793% ( 870, 1300 ] 272 0.074% 99.867% ( 1300, 1900 ] 165 0.045% 99.912% ( 1900, 2900 ] 154 0.042% 99.953% ( 2900, 4400 ] 137 0.037% 99.991% ( 4400, 6600 ] 41 0.011% 100.002% ( 6600, 9900 ] 3 0.001% 100.002% ( 9900, 14000 ] 4 0.001% 100.004% ( 14000, 22000 ] 3 0.001% 100.004% ( 22000, 33000 ] 2 0.001% 100.005% ** Level 2 read latency histogram (micros): Count: 861109 Average: 14.9322 StdDev: 209.42 Min: 1 Median: 2.0091 Max: 36392 Percentiles: P50: 2.01 P75: 3.11 P99: 13.35 P99.9: 1137.83 P99.99: 1536.39 ------------------------------------------------------ [ 0, 1 ] 18518 2.150% 2.150% ( 1, 2 ] 410144 47.630% 49.780% ########## ( 2, 3 ] 207143 24.055% 73.836% ##### ( 3, 4 ] 93458 10.853% 84.689% ## ( 4, 6 ] 94895 11.020% 95.709% ## ( 6, 10 ] 23623 2.743% 98.452% # ( 10, 15 ] 7038 0.817% 99.270% ( 15, 22 ] 1011 0.117% 99.387% ( 22, 34 ] 114 0.013% 99.400% ( 34, 51 ] 56 0.007% 99.407% ( 51, 76 ] 60 0.007% 99.414% ( 76, 110 ] 165 0.019% 99.433% ( 110, 170 ] 493 0.057% 99.490% ( 170, 250 ] 348 0.040% 99.530% ( 250, 380 ] 488 0.057% 99.587% ( 380, 580 ] 849 0.099% 99.686% ( 580, 870 ] 1118 0.130% 99.816% ( 870, 1300 ] 1167 0.136% 99.951% ( 1300, 1900 ] 850 0.099% 100.050% ( 1900, 2900 ] 729 0.085% 100.134% ( 2900, 4400 ] 711 0.083% 100.217% ( 4400, 6600 ] 194 0.023% 100.240% ( 6600, 9900 ] 39 0.005% 100.244% ( 9900, 14000 ] 14 0.002% 100.246% ( 14000, 22000 ] 11 0.001% 100.247% ( 22000, 33000 ] 9 0.001% 100.248% ( 33000, 50000 ] 2 0.000% 100.248% ** Level 3 read latency histogram (micros): Count: 2231601 Average: 11.5391 StdDev: 188.91 Min: 1 Median: 1.9613 Max: 46665 Percentiles: P50: 1.96 P75: 2.97 P99: 12.39 P99.9: 617.83 P99.99: 909.70 ------------------------------------------------------ [ 0, 1 ] 38314 1.717% 1.717% ( 1, 2 ] 1120806 50.224% 51.941% ########## ( 2, 3 ] 531566 23.820% 75.761% ##### ( 3, 4 ] 229789 10.297% 86.058% ## ( 4, 6 ] 231438 10.371% 96.429% ## ( 6, 10 ] 51330 2.300% 98.729% ( 10, 15 ] 12623 0.566% 99.295% ( 15, 22 ] 1445 0.065% 99.360% ( 22, 34 ] 124 0.006% 99.365% ( 34, 51 ] 189 0.008% 99.374% ( 51, 76 ] 236 0.011% 99.384% ( 76, 110 ] 1474 0.066% 99.450% ( 110, 170 ] 4961 0.222% 99.673% ( 170, 250 ] 1836 0.082% 99.755% ( 250, 380 ] 1259 0.056% 99.811% ( 380, 580 ] 1706 0.076% 99.888% ( 580, 870 ] 2096 0.094% 99.982% ( 870, 1300 ] 2013 0.090% 100.072% ( 1300, 1900 ] 1314 0.059% 100.131% ( 1900, 2900 ] 1207 0.054% 100.185% ( 2900, 4400 ] 1237 0.055% 100.240% ( 4400, 6600 ] 319 0.014% 100.255% ( 6600, 9900 ] 67 0.003% 100.258% ( 9900, 14000 ] 24 0.001% 100.259% ( 14000, 22000 ] 47 0.002% 100.261% ( 22000, 33000 ] 17 0.001% 100.262% ( 33000, 50000 ] 6 0.000% 100.262% ** Level 4 read latency histogram (micros): Count: 1482310 Average: 24.0224 StdDev: 219.20 Min: 1 Median: 1.9456 Max: 34968 Percentiles: P50: 1.95 P75: 3.07 P99: 398.29 P99.9: 2196.41 P99.99: 3222.08 ------------------------------------------------------ [ 0, 1 ] 38082 2.569% 2.569% # ( 1, 2 ] 743527 50.160% 52.729% ########## ( 2, 3 ] 321044 21.658% 74.387% #### ( 3, 4 ] 135650 9.151% 83.539% ## ( 4, 6 ] 131740 8.887% 92.426% ## ( 6, 10 ] 33491 2.259% 94.686% ( 10, 15 ] 8036 0.542% 95.228% ( 15, 22 ] 866 0.058% 95.286% ( 22, 34 ] 72 0.005% 95.291% ( 34, 51 ] 122 0.008% 95.299% ( 51, 76 ] 172 0.012% 95.311% ( 76, 110 ] 9450 0.638% 95.948% ( 110, 170 ] 30188 2.037% 97.985% ( 170, 250 ] 10099 0.681% 98.666% ( 250, 380 ] 4525 0.305% 98.971% ( 380, 580 ] 4625 0.312% 99.283% ( 580, 870 ] 3979 0.268% 99.552% ( 870, 1300 ] 2974 0.201% 99.753% ( 1300, 1900 ] 1755 0.118% 99.871% ( 1900, 2900 ] 1453 0.098% 99.969% ( 2900, 4400 ] 1452 0.098% 100.067% ( 4400, 6600 ] 403 0.027% 100.094% ( 6600, 9900 ] 69 0.005% 100.099% ( 9900, 14000 ] 18 0.001% 100.100% ( 14000, 22000 ] 25 0.002% 100.102% ( 22000, 33000 ] 12 0.001% 100.102% ( 33000, 50000 ] 1 0.000% 100.103% ** Level 5 read latency histogram (micros): Count: 361906 Average: 321.2497 StdDev: 670.85 Min: 1 Median: 154.4021 Max: 44023 Percentiles: P50: 154.40 P75: 236.71 P99: 3482.70 P99.9: 6303.66 P99.99: 14749.17 ------------------------------------------------------ [ 0, 1 ] 12 0.003% 0.003% ( 1, 2 ] 112 0.031% 0.034% ( 2, 3 ] 240 0.066% 0.101% ( 3, 4 ] 227 0.063% 0.163% ( 4, 6 ] 292 0.081% 0.244% ( 6, 10 ] 32 0.009% 0.253% ( 10, 15 ] 10 0.003% 0.256% ( 51, 76 ] 23 0.006% 0.262% ( 76, 110 ] 52671 14.554% 14.816% ### ( 110, 170 ] 172065 47.544% 62.360% ########## ( 170, 250 ] 54857 15.158% 77.518% ### ( 250, 380 ] 22602 6.245% 83.763% # ( 380, 580 ] 21539 5.952% 89.714% # ( 580, 870 ] 13926 3.848% 93.562% # ( 870, 1300 ] 9089 2.511% 96.074% # ( 1300, 1900 ] 5081 1.404% 97.478% ( 1900, 2900 ] 4072 1.125% 98.603% ( 2900, 4400 ] 3699 1.022% 99.625% ( 4400, 6600 ] 1150 0.318% 99.943% ( 6600, 9900 ] 148 0.041% 99.984% ( 9900, 14000 ] 20 0.006% 99.989% ( 14000, 22000 ] 30 0.008% 99.998% ( 22000, 33000 ] 28 0.008% 100.005% ( 33000, 50000 ] 10 0.003% 100.008% ** DB Stats ** Uptime(secs): 43.8 total, 15.2 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 134.85 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:20.540 H:M:S, 46.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 64.69 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:11.477 H:M:S, 75.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 298.367 micros/op 13399 ops/sec 60.077 seconds 804996 operations; 13.0 MB/s (199999 of 199999 found) Microseconds per read: Count: 804996 Average: 298.3688 StdDev: 876.43 Min: 2 Median: 182.3126 Max: 111146 Percentiles: P50: 182.31 P75: 247.13 P99: 2436.65 P99.9: 13469.64 P99.99: 32358.92 ------------------------------------------------------ ( 1, 2 ] 6 0.001% 0.001% ( 2, 3 ] 31 0.004% 0.005% ( 3, 4 ] 47 0.006% 0.010% ( 4, 6 ] 75 0.009% 0.020% ( 6, 10 ] 367 0.046% 0.065% ( 10, 15 ] 2387 0.297% 0.362% ( 15, 22 ] 1878 0.233% 0.595% ( 22, 34 ] 4966 0.617% 1.212% ( 34, 51 ] 1502 0.187% 1.399% ( 51, 76 ] 192 0.024% 1.422% ( 76, 110 ] 16986 2.110% 3.533% ( 110, 170 ] 335831 41.718% 45.251% ######## ( 170, 250 ] 248396 30.857% 76.108% ###### ( 250, 380 ] 96590 11.999% 88.107% ## ( 380, 580 ] 44869 5.574% 93.680% # ( 580, 870 ] 22614 2.809% 96.490% # ( 870, 1300 ] 12097 1.503% 97.992% ( 1300, 1900 ] 5864 0.728% 98.721% ( 1900, 2900 ] 4189 0.520% 99.241% ( 2900, 4400 ] 3513 0.436% 99.678% ( 4400, 6600 ] 1040 0.129% 99.807% ( 6600, 9900 ] 327 0.041% 99.847% ( 9900, 14000 ] 487 0.060% 99.908% ( 14000, 22000 ] 508 0.063% 99.971% ( 22000, 33000 ] 163 0.020% 99.991% ( 33000, 50000 ] 53 0.007% 99.998% ( 50000, 75000 ] 17 0.002% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 12895986 rocksdb.bloom.filter.full.positive COUNT : 930152 rocksdb.bloom.filter.full.true.positive COUNT : 804611 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 385 rocksdb.memtable.miss COUNT : 804611 rocksdb.l0.hit COUNT : 32292 rocksdb.l1.hit COUNT : 3586 rocksdb.l2andup.hit COUNT : 768733 rocksdb.compaction.key.drop.new COUNT : 328385 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 6088133 rocksdb.number.keys.read COUNT : 804996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 6282953256 rocksdb.bytes.read COUNT : 804996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1801 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 36422472 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 31396561698 rocksdb.compact.write.bytes COUNT : 31051428855 rocksdb.flush.write.bytes COUNT : 6275472737 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 37 rocksdb.number.superversion_acquires COUNT : 1286 rocksdb.number.superversion_releases COUNT : 1245 rocksdb.number.superversion_cleanups COUNT : 322 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 406 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 6224798720 rocksdb.memtable.garbage.bytes.at.flush COUNT : 2348032 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 35128561669 rocksdb.non.last.level.read.count COUNT : 8344889 rocksdb.block.checksum.compute.count COUNT : 8543031 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 180.879526 P95 : 715.045160 P99 : 2435.732378 P100 : 111142.000000 COUNT : 804996 SUM : 239502268 rocksdb.db.write.micros P50 : 1.568111 P95 : 4.015692 P99 : 7.863872 P100 : 972259.000000 COUNT : 6088133 SUM : 51873392 rocksdb.compaction.times.micros P50 : 1410000.000000 P95 : 4916000.000000 P99 : 9404000.000000 P100 : 40471532.000000 COUNT : 112 SUM : 238076524 rocksdb.compaction.times.cpu_micros P50 : 905333.333333 P95 : 3390000.000000 P99 : 6368000.000000 P100 : 15602814.000000 COUNT : 112 SUM : 139664387 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 990.937500 P95 : 68750.000000 P99 : 182000.000000 P100 : 219849.000000 COUNT : 85 SUM : 911190 rocksdb.compaction.outfile.sync.micros P50 : 14347.826087 P95 : 220461.538462 P99 : 371018.181818 P100 : 550464.000000 COUNT : 376 SUM : 19401301 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 441.538462 P95 : 4145.000000 P99 : 5973.000000 P100 : 6326.000000 COUNT : 228 SUM : 253025 rocksdb.table.open.io.micros P50 : 8385.523614 P95 : 19591.525424 P99 : 22726.000000 P100 : 41211.000000 COUNT : 1801 SUM : 15353425 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.939575 P95 : 9.905810 P99 : 169.085518 P100 : 46671.000000 COUNT : 7601934 SUM : 111102591 rocksdb.read.block.get.micros P50 : 144.587783 P95 : 572.476001 P99 : 2207.459516 P100 : 44028.000000 COUNT : 939303 SUM : 209271530 rocksdb.write.raw.block.micros P50 : 0.528432 P95 : 1.107129 P99 : 3.319625 P100 : 63989.000000 COUNT : 9040862 SUM : 28340224 rocksdb.numfiles.in.singlecompaction P50 : 2.857143 P95 : 7.200000 P99 : 13.900000 P100 : 22.000000 COUNT : 144 SUM : 522 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.564966 P95 : 1231.365445 P99 : 2636.899524 P100 : 972073.000000 COUNT : 179014 SUM : 36433899 rocksdb.sst.read.micros P50 : 1.955099 P95 : 157.390155 P99 : 511.194837 P100 : 46665.000000 COUNT : 8344954 SUM : 299260984 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 804996 SUM : 804996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 6088133 SUM : 6282953256 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 229649.122807 P95 : 1015833.333333 P99 : 1216587.000000 P100 : 1216587.000000 COUNT : 85 SUM : 28934145 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 406 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0