[2024-07-15 00:23:53.192247] Starting SPDK v24.01.1-pre git sha1 4b94202c6 / DPDK 23.11.0 initialization... [2024-07-15 00:23:53.192390] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid508831 ] Set seed to 1720995833188859 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-15 00:23:53.298124] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-15 00:23:53.519900] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-15 00:23:53.519950] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 508831' to capture a snapshot of events at runtime. [2024-07-15 00:23:53.519960] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid508831 for offline analysis/debug. [2024-07-15 00:23:53.519982] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Jul 15 00:24:02 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/07/15-00:24:10 ... thread 0: (1000000,1000000) ops and (131065.8,131065.8) ops/second in (7.629758,7.629758) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 8/7 755.83 MB 6.3 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 116.0 7.06 2.41 9 0.785 0 0 0.0 0.0 L1 4/4 271.21 MB 0.0 0.3 0.2 0.1 0.3 0.2 0.0 1.7 106.2 106.1 2.56 1.26 1 2.556 275K 268 0.0 0.0 L2 2/0 257.56 MB 2.6 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 51/8 3.93 GB 3.1 3.9 2.2 1.7 3.8 2.1 0.0 1.8 105.8 104.9 37.57 18.64 17 2.210 2636K 7335 0.0 0.0 L4 184/14 11.31 GB 1.1 2.3 1.7 0.5 2.2 1.7 0.1 1.3 103.9 103.1 22.27 10.80 15 1.485 938K 0 0.0 0.0 L5 1088/0 89.81 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1337/33 106.31 GB 0.0 6.4 4.0 2.4 7.2 4.8 0.1 8.9 94.4 105.5 69.46 33.11 42 1.654 3850K 7603 0.0 0.0 Int 0/0 0.00 KB 0.0 6.4 4.0 2.4 7.2 4.8 0.1 8.9 94.4 105.5 69.46 33.11 42 1.654 3850K 7603 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.4 4.0 2.4 6.4 4.0 0.0 0.0 105.1 104.3 62.40 30.71 33 1.891 3850K 7603 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 116.0 7.06 2.41 9 0.785 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.8 total, 7.6 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.16 GB write, 939.77 MB/s write, 6.41 GB read, 841.39 MB/s read, 69.5 seconds Interval compaction: 7.16 GB write, 960.19 MB/s write, 6.41 GB read, 859.66 MB/s read, 69.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 26, interval: 26 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 136614 Average: 11.5020 StdDev: 161.52 Min: 1 Median: 1.9180 Max: 16681 Percentiles: P50: 1.92 P75: 2.92 P99: 10.91 P99.9: 2489.06 P99.99: 4350.10 ------------------------------------------------------ [ 0, 1 ] 4493 3.289% 3.289% # ( 1, 2 ] 69516 50.885% 54.174% ########## ( 2, 3 ] 30909 22.625% 76.799% ##### ( 3, 4 ] 12388 9.068% 85.867% ## ( 4, 6 ] 14718 10.773% 96.640% ## ( 6, 10 ] 3080 2.255% 98.895% ( 10, 15 ] 798 0.584% 99.479% ( 15, 22 ] 93 0.068% 99.547% ( 22, 34 ] 17 0.012% 99.559% ( 34, 51 ] 2 0.001% 99.561% ( 51, 76 ] 3 0.002% 99.563% ( 76, 110 ] 1 0.001% 99.564% ( 110, 170 ] 2 0.001% 99.565% ( 170, 250 ] 9 0.007% 99.572% ( 250, 380 ] 14 0.010% 99.582% ( 380, 580 ] 37 0.027% 99.609% ( 580, 870 ] 79 0.058% 99.667% ( 870, 1300 ] 105 0.077% 99.744% ( 1300, 1900 ] 150 0.110% 99.854% ( 1900, 2900 ] 111 0.081% 99.935% ( 2900, 4400 ] 80 0.059% 99.993% ( 4400, 6600 ] 29 0.021% 100.015% ( 6600, 9900 ] 2 0.001% 100.016% ( 14000, 22000 ] 2 0.001% 100.018% ** Level 1 read latency histogram (micros): Count: 92632 Average: 13.4201 StdDev: 175.11 Min: 1 Median: 1.9612 Max: 8649 Percentiles: P50: 1.96 P75: 2.98 P99: 11.94 P99.9: 3161.30 P99.99: 6155.51 ------------------------------------------------------ [ 0, 1 ] 2312 2.496% 2.496% ( 1, 2 ] 45779 49.420% 51.916% ########## ( 2, 3 ] 21919 23.662% 75.579% ##### ( 3, 4 ] 8807 9.508% 85.086% ## ( 4, 6 ] 10471 11.304% 96.390% ## ( 6, 10 ] 2167 2.339% 98.729% ( 10, 15 ] 645 0.696% 99.426% ( 15, 22 ] 73 0.079% 99.504% ( 22, 34 ] 8 0.009% 99.513% ( 51, 76 ] 1 0.001% 99.514% ( 170, 250 ] 2 0.002% 99.516% ( 250, 380 ] 6 0.006% 99.523% ( 380, 580 ] 15 0.016% 99.539% ( 580, 870 ] 37 0.040% 99.579% ( 870, 1300 ] 68 0.073% 99.652% ( 1300, 1900 ] 107 0.116% 99.768% ( 1900, 2900 ] 110 0.119% 99.887% ( 2900, 4400 ] 71 0.077% 99.963% ( 4400, 6600 ] 31 0.033% 99.997% ( 6600, 9900 ] 3 0.003% 100.000% ** Level 2 read latency histogram (micros): Count: 459789 Average: 18.9471 StdDev: 232.97 Min: 1 Median: 1.9487 Max: 26411 Percentiles: P50: 1.95 P75: 3.06 P99: 13.49 P99.9: 2631.20 P99.99: 3350.53 ------------------------------------------------------ [ 0, 1 ] 15629 3.399% 3.399% # ( 1, 2 ] 225840 49.118% 52.517% ########## ( 2, 3 ] 100489 21.855% 74.373% #### ( 3, 4 ] 45311 9.855% 84.228% ## ( 4, 6 ] 52232 11.360% 95.588% ## ( 6, 10 ] 12742 2.771% 98.359% # ( 10, 15 ] 4220 0.918% 99.277% ( 15, 22 ] 628 0.137% 99.413% ( 22, 34 ] 79 0.017% 99.430% ( 34, 51 ] 16 0.003% 99.434% ( 51, 76 ] 14 0.003% 99.437% ( 76, 110 ] 10 0.002% 99.439% ( 110, 170 ] 12 0.003% 99.442% ( 170, 250 ] 19 0.004% 99.446% ( 250, 380 ] 73 0.016% 99.462% ( 380, 580 ] 121 0.026% 99.488% ( 580, 870 ] 255 0.055% 99.543% ( 870, 1300 ] 419 0.091% 99.635% ( 1300, 1900 ] 606 0.132% 99.766% ( 1900, 2900 ] 840 0.183% 99.949% ( 2900, 4400 ] 626 0.136% 100.085% ( 4400, 6600 ] 210 0.046% 100.131% ( 6600, 9900 ] 33 0.007% 100.138% ( 9900, 14000 ] 4 0.001% 100.139% ( 14000, 22000 ] 1 0.000% 100.139% ( 22000, 33000 ] 3 0.001% 100.140% ** Level 3 read latency histogram (micros): Count: 867454 Average: 15.0973 StdDev: 188.95 Min: 1 Median: 2.0328 Max: 21706 Percentiles: P50: 2.03 P75: 3.06 P99: 11.58 P99.9: 1716.59 P99.99: 2302.58 ------------------------------------------------------ [ 0, 1 ] 16159 1.863% 1.863% ( 1, 2 ] 410406 47.312% 49.174% ######### ( 2, 3 ] 218503 25.189% 74.363% ##### ( 3, 4 ] 86617 9.985% 84.349% ## ( 4, 6 ] 103853 11.972% 96.321% ## ( 6, 10 ] 21148 2.438% 98.759% ( 10, 15 ] 6642 0.766% 99.524% ( 15, 22 ] 761 0.088% 99.612% ( 22, 34 ] 82 0.009% 99.622% ( 34, 51 ] 41 0.005% 99.626% ( 51, 76 ] 52 0.006% 99.632% ( 76, 110 ] 15 0.002% 99.634% ( 110, 170 ] 20 0.002% 99.636% ( 170, 250 ] 56 0.006% 99.643% ( 250, 380 ] 107 0.012% 99.655% ( 380, 580 ] 190 0.022% 99.677% ( 580, 870 ] 469 0.054% 99.731% ( 870, 1300 ] 774 0.089% 99.820% ( 1300, 1900 ] 996 0.115% 99.935% ( 1900, 2900 ] 1183 0.136% 100.071% ( 2900, 4400 ] 879 0.101% 100.173% ( 4400, 6600 ] 281 0.032% 100.205% ( 6600, 9900 ] 34 0.004% 100.209% ( 9900, 14000 ] 1 0.000% 100.209% ( 14000, 22000 ] 1 0.000% 100.209% ** Level 4 read latency histogram (micros): Count: 208730 Average: 40.3926 StdDev: 334.44 Min: 1 Median: 2.0368 Max: 17887 Percentiles: P50: 2.04 P75: 3.11 P99: 1420.69 P99.9: 4105.63 P99.99: 5094.43 ------------------------------------------------------ [ 0, 1 ] 4121 1.974% 1.974% ( 1, 2 ] 98338 47.113% 49.087% ######### ( 2, 3 ] 51843 24.837% 73.924% ##### ( 3, 4 ] 21216 10.164% 84.089% ## ( 4, 6 ] 22751 10.900% 94.988% ## ( 6, 10 ] 4947 2.370% 97.358% ( 10, 15 ] 1551 0.743% 98.101% ( 15, 22 ] 199 0.095% 98.197% ( 22, 34 ] 24 0.011% 98.208% ( 34, 51 ] 10 0.005% 98.213% ( 51, 76 ] 23 0.011% 98.224% ( 76, 110 ] 8 0.004% 98.228% ( 110, 170 ] 12 0.006% 98.234% ( 170, 250 ] 66 0.032% 98.265% ( 250, 380 ] 85 0.041% 98.306% ( 380, 580 ] 213 0.102% 98.408% ( 580, 870 ] 403 0.193% 98.601% ( 870, 1300 ] 680 0.326% 98.927% ( 1300, 1900 ] 769 0.368% 99.295% ( 1900, 2900 ] 778 0.373% 99.668% ( 2900, 4400 ] 605 0.290% 99.958% ( 4400, 6600 ] 219 0.105% 100.063% ( 6600, 9900 ] 34 0.016% 100.079% ( 9900, 14000 ] 2 0.001% 100.080% ( 14000, 22000 ] 1 0.000% 100.080% ** Level 5 read latency histogram (micros): Count: 9001 Average: 2404.3983 StdDev: 1753.80 Min: 96 Median: 1929.2874 Max: 27633 Percentiles: P50: 1929.29 P75: 3487.88 P99: 8600.89 P99.9: 11949.71 P99.99: 27633.00 ------------------------------------------------------ ( 76, 110 ] 2 0.022% 0.022% ( 110, 170 ] 18 0.200% 0.222% ( 170, 250 ] 65 0.722% 0.944% ( 250, 380 ] 174 1.933% 2.877% ( 380, 580 ] 462 5.133% 8.010% # ( 580, 870 ] 959 10.654% 18.665% ## ( 870, 1300 ] 1323 14.698% 33.363% ### ( 1300, 1900 ] 1449 16.098% 49.461% ### ( 1900, 2900 ] 1656 18.398% 67.859% #### ( 2900, 4400 ] 1640 18.220% 86.079% #### ( 4400, 6600 ] 1049 11.654% 97.734% ## ( 6600, 9900 ] 188 2.089% 99.822% ( 9900, 14000 ] 14 0.156% 99.978% ( 22000, 33000 ] 2 0.022% 100.000% ** DB Stats ** Uptime(secs): 7.8 total, 7.6 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 126.21 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.720 H:M:S, 47.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 128.95 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.720 H:M:S, 48.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/07/15-00:24:17 ... thread 0: (1000000,2000000) ops and (147722.3,138896.4) ops/second in (6.769459,14.399217) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/9 1.48 GB 50.3 0.7 0.0 0.7 2.6 1.8 0.0 1.4 42.2 147.6 17.92 8.43 24 0.747 767K 1054 0.0 0.0 L1 2/2 102.73 MB 0.0 0.8 0.5 0.4 0.8 0.5 0.0 1.8 120.5 120.2 7.12 3.97 2 3.560 870K 1472 0.0 0.0 L2 3/2 272.39 MB 1.3 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 27/17 2.04 GB 0.9 5.2 2.6 2.6 5.1 2.6 0.0 2.0 108.2 107.4 49.03 25.28 22 2.229 3988K 15K 0.0 0.0 L4 169/0 9.58 GB 1.0 6.2 3.8 2.3 6.1 3.7 0.3 1.6 111.0 109.4 56.89 29.99 40 1.422 2979K 3015 0.0 0.0 L5 1136/0 93.81 GB 1.0 0.0 0.0 0.0 0.0 0.0 4.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1349/30 107.28 GB 0.0 12.9 6.9 6.0 14.6 8.6 4.8 7.9 101.1 114.5 130.96 67.67 88 1.488 8605K 20K 0.0 0.0 Int 0/0 0.00 KB 0.0 12.9 6.9 6.0 14.6 8.6 4.8 7.9 101.1 114.5 130.96 67.67 88 1.488 8605K 20K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.9 6.9 6.0 12.8 6.8 0.0 0.0 112.9 111.7 117.25 62.25 66 1.777 8605K 20K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 137.9 13.70 5.42 22 0.623 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 14.6 total, 14.4 interval Flush(GB): cumulative 1.845, interval 1.845 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.64 GB write, 1028.96 MB/s write, 12.92 GB read, 908.54 MB/s read, 131.0 seconds Interval compaction: 14.64 GB write, 1040.81 MB/s write, 12.92 GB read, 919.00 MB/s read, 131.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 53, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 53, interval: 53 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 492651 Average: 9.4656 StdDev: 154.49 Min: 1 Median: 1.8648 Max: 46727 Percentiles: P50: 1.86 P75: 2.81 P99: 10.64 P99.9: 1529.17 P99.99: 3204.56 ------------------------------------------------------ [ 0, 1 ] 20061 4.072% 4.072% # ( 1, 2 ] 261633 53.107% 57.179% ########### ( 2, 3 ] 107825 21.887% 79.066% #### ( 3, 4 ] 40894 8.301% 87.367% ## ( 4, 6 ] 47202 9.581% 96.948% ## ( 6, 10 ] 9793 1.988% 98.936% ( 10, 15 ] 2483 0.504% 99.440% ( 15, 22 ] 238 0.048% 99.488% ( 22, 34 ] 31 0.006% 99.494% ( 34, 51 ] 10 0.002% 99.496% ( 51, 76 ] 20 0.004% 99.500% ( 76, 110 ] 12 0.002% 99.503% ( 110, 170 ] 117 0.024% 99.527% ( 170, 250 ] 197 0.040% 99.567% ( 250, 380 ] 293 0.059% 99.626% ( 380, 580 ] 358 0.073% 99.699% ( 580, 870 ] 480 0.097% 99.796% ( 870, 1300 ] 397 0.081% 99.877% ( 1300, 1900 ] 302 0.061% 99.938% ( 1900, 2900 ] 221 0.045% 99.983% ( 2900, 4400 ] 176 0.036% 100.019% ( 4400, 6600 ] 72 0.015% 100.033% ( 6600, 9900 ] 9 0.002% 100.035% ( 9900, 14000 ] 2 0.000% 100.036% ( 14000, 22000 ] 2 0.000% 100.036% ( 33000, 50000 ] 2 0.000% 100.036% ** Level 1 read latency histogram (micros): Count: 208773 Average: 15.9589 StdDev: 286.44 Min: 1 Median: 1.9929 Max: 57833 Percentiles: P50: 1.99 P75: 3.00 P99: 13.69 P99.9: 3137.44 P99.99: 5905.87 ------------------------------------------------------ [ 0, 1 ] 4407 2.111% 2.111% ( 1, 2 ] 100693 48.231% 50.342% ########## ( 2, 3 ] 51596 24.714% 75.056% ##### ( 3, 4 ] 19791 9.480% 84.535% ## ( 4, 6 ] 23646 11.326% 95.862% ## ( 6, 10 ] 5339 2.557% 98.419% # ( 10, 15 ] 1643 0.787% 99.206% ( 15, 22 ] 163 0.078% 99.284% ( 22, 34 ] 20 0.010% 99.293% ( 34, 51 ] 11 0.005% 99.299% ( 51, 76 ] 18 0.009% 99.307% ( 76, 110 ] 4 0.002% 99.309% ( 110, 170 ] 21 0.010% 99.319% ( 170, 250 ] 38 0.018% 99.338% ( 250, 380 ] 70 0.034% 99.371% ( 380, 580 ] 145 0.069% 99.441% ( 580, 870 ] 238 0.114% 99.555% ( 870, 1300 ] 266 0.127% 99.682% ( 1300, 1900 ] 226 0.108% 99.790% ( 1900, 2900 ] 202 0.097% 99.887% ( 2900, 4400 ] 172 0.082% 99.969% ( 4400, 6600 ] 63 0.030% 100.000% ( 6600, 9900 ] 9 0.004% 100.004% ( 9900, 14000 ] 1 0.000% 100.004% ( 14000, 22000 ] 4 0.002% 100.006% ( 22000, 33000 ] 2 0.001% 100.007% ( 33000, 50000 ] 3 0.001% 100.009% ( 50000, 75000 ] 1 0.000% 100.009% ** Level 2 read latency histogram (micros): Count: 525215 Average: 18.3483 StdDev: 237.28 Min: 1 Median: 1.9564 Max: 28668 Percentiles: P50: 1.96 P75: 3.06 P99: 13.40 P99.9: 2662.58 P99.99: 3483.41 ------------------------------------------------------ [ 0, 1 ] 16770 3.193% 3.193% # ( 1, 2 ] 257045 48.941% 52.134% ########## ( 2, 3 ] 117082 22.292% 74.426% #### ( 3, 4 ] 51478 9.801% 84.227% ## ( 4, 6 ] 59847 11.395% 95.622% ## ( 6, 10 ] 14469 2.755% 98.377% # ( 10, 15 ] 4818 0.917% 99.294% ( 15, 22 ] 696 0.133% 99.427% ( 22, 34 ] 82 0.016% 99.443% ( 34, 51 ] 24 0.005% 99.447% ( 51, 76 ] 24 0.005% 99.452% ( 76, 110 ] 10 0.002% 99.454% ( 110, 170 ] 12 0.002% 99.456% ( 170, 250 ] 21 0.004% 99.460% ( 250, 380 ] 83 0.016% 99.476% ( 380, 580 ] 147 0.028% 99.504% ( 580, 870 ] 295 0.056% 99.560% ( 870, 1300 ] 468 0.089% 99.649% ( 1300, 1900 ] 650 0.124% 99.773% ( 1900, 2900 ] 877 0.167% 99.940% ( 2900, 4400 ] 680 0.129% 100.069% ( 4400, 6600 ] 228 0.043% 100.113% ( 6600, 9900 ] 40 0.008% 100.120% ( 9900, 14000 ] 5 0.001% 100.121% ( 14000, 22000 ] 5 0.001% 100.122% ( 22000, 33000 ] 5 0.001% 100.123% ** Level 3 read latency histogram (micros): Count: 1747927 Average: 12.7935 StdDev: 177.82 Min: 1 Median: 2.0124 Max: 32163 Percentiles: P50: 2.01 P75: 3.02 P99: 11.41 P99.9: 1232.61 P99.99: 1821.75 ------------------------------------------------------ [ 0, 1 ] 29614 1.694% 1.694% ( 1, 2 ] 838918 47.995% 49.689% ########## ( 2, 3 ] 438865 25.108% 74.797% ##### ( 3, 4 ] 170510 9.755% 84.552% ## ( 4, 6 ] 205279 11.744% 96.296% ## ( 6, 10 ] 43512 2.489% 98.785% ( 10, 15 ] 13339 0.763% 99.549% ( 15, 22 ] 1428 0.082% 99.630% ( 22, 34 ] 114 0.007% 99.637% ( 34, 51 ] 95 0.005% 99.642% ( 51, 76 ] 166 0.009% 99.652% ( 76, 110 ] 42 0.002% 99.654% ( 110, 170 ] 124 0.007% 99.661% ( 170, 250 ] 259 0.015% 99.676% ( 250, 380 ] 418 0.024% 99.700% ( 380, 580 ] 767 0.044% 99.744% ( 580, 870 ] 1326 0.076% 99.820% ( 870, 1300 ] 1665 0.095% 99.915% ( 1300, 1900 ] 1509 0.086% 100.001% ( 1900, 2900 ] 1589 0.091% 100.092% ( 2900, 4400 ] 1309 0.075% 100.167% ( 4400, 6600 ] 415 0.024% 100.191% ( 6600, 9900 ] 61 0.003% 100.194% ( 9900, 14000 ] 9 0.001% 100.195% ( 14000, 22000 ] 12 0.001% 100.196% ( 22000, 33000 ] 5 0.000% 100.196% ** Level 4 read latency histogram (micros): Count: 613505 Average: 28.1082 StdDev: 292.37 Min: 1 Median: 2.0164 Max: 25939 Percentiles: P50: 2.02 P75: 3.13 P99: 686.13 P99.9: 3389.52 P99.99: 4160.68 ------------------------------------------------------ [ 0, 1 ] 12846 2.094% 2.094% ( 1, 2 ] 291484 47.511% 49.605% ########## ( 2, 3 ] 147953 24.116% 73.721% ##### ( 3, 4 ] 61447 10.016% 83.737% ## ( 4, 6 ] 69537 11.334% 95.071% ## ( 6, 10 ] 15868 2.586% 97.658% # ( 10, 15 ] 5118 0.834% 98.492% ( 15, 22 ] 544 0.089% 98.581% ( 22, 34 ] 47 0.008% 98.588% ( 34, 51 ] 40 0.007% 98.595% ( 51, 76 ] 74 0.012% 98.607% ( 76, 110 ] 24 0.004% 98.611% ( 110, 170 ] 148 0.024% 98.635% ( 170, 250 ] 356 0.058% 98.693% ( 250, 380 ] 484 0.079% 98.772% ( 380, 580 ] 877 0.143% 98.915% ( 580, 870 ] 1429 0.233% 99.148% ( 870, 1300 ] 1630 0.266% 99.413% ( 1300, 1900 ] 1421 0.232% 99.645% ( 1900, 2900 ] 1214 0.198% 99.843% ( 2900, 4400 ] 1074 0.175% 100.018% ( 4400, 6600 ] 375 0.061% 100.079% ( 6600, 9900 ] 61 0.010% 100.089% ( 9900, 14000 ] 14 0.002% 100.091% ( 14000, 22000 ] 14 0.002% 100.094% ( 22000, 33000 ] 11 0.002% 100.095% ** Level 5 read latency histogram (micros): Count: 27725 Average: 1524.1290 StdDev: 1758.15 Min: 87 Median: 949.9719 Max: 47069 Percentiles: P50: 949.97 P75: 1966.53 P99: 6824.70 P99.9: 17246.15 P99.99: 45715.23 ------------------------------------------------------ ( 76, 110 ] 83 0.299% 0.299% ( 110, 170 ] 869 3.134% 3.434% # ( 170, 250 ] 1628 5.872% 9.306% # ( 250, 380 ] 2323 8.379% 17.684% ## ( 380, 580 ] 3446 12.429% 30.114% ## ( 580, 870 ] 4687 16.905% 47.019% ### ( 870, 1300 ] 4444 16.029% 63.048% ### ( 1300, 1900 ] 3132 11.297% 74.344% ## ( 1900, 2900 ] 2732 9.854% 84.198% ## ( 2900, 4400 ] 2646 9.544% 93.742% ## ( 4400, 6600 ] 1441 5.197% 98.940% # ( 6600, 9900 ] 246 0.887% 99.827% ( 9900, 14000 ] 15 0.054% 99.881% ( 14000, 22000 ] 13 0.047% 99.928% ( 22000, 33000 ] 9 0.032% 99.960% ( 33000, 50000 ] 11 0.040% 100.000% ** DB Stats ** Uptime(secs): 14.6 total, 6.8 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 135.12 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.553 H:M:S, 45.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 145.39 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.833 H:M:S, 41.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 4 num-running-flushes: 1 2024/07/15-00:24:22 ... thread 0: (1000000,3000000) ops and (173769.3,148854.0) ops/second in (5.754758,20.153975) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/10 1.66 GB 25.2 1.3 0.0 1.3 4.0 2.7 0.0 1.5 51.6 159.6 25.64 13.26 37 0.693 1343K 1657 0.0 0.0 L1 2/2 219.87 MB 0.0 1.6 1.1 0.5 1.6 1.1 0.0 1.4 131.7 131.4 12.55 7.73 3 4.183 1678K 3825 0.0 0.0 L2 2/0 94.89 MB 0.9 0.3 0.2 0.1 0.3 0.2 0.8 1.7 160.5 160.0 1.97 1.56 1 1.974 321K 956 0.0 0.0 L3 23/6 1.43 GB 0.9 7.3 3.1 4.3 7.3 3.0 0.3 2.4 115.5 114.6 64.95 35.70 29 2.240 5274K 17K 0.0 0.0 L4 184/16 10.49 GB 1.0 8.9 5.1 3.8 8.7 4.9 0.4 1.7 116.7 114.8 77.70 43.29 58 1.340 4665K 11K 0.0 0.0 L5 1140/0 94.18 GB 1.0 0.0 0.0 0.0 0.0 0.0 4.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1363/34 108.07 GB 0.0 19.4 9.5 9.9 21.9 12.0 5.9 8.1 108.6 122.6 182.82 101.54 128 1.428 13M 35K 0.0 0.0 Int 0/0 0.00 KB 0.0 19.4 9.5 9.9 21.9 12.0 5.9 8.1 108.6 122.6 182.82 101.54 128 1.428 13M 35K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 19.4 9.5 9.9 19.2 9.2 0.0 0.0 121.2 119.9 163.92 93.55 95 1.725 13M 35K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.7 2.7 0.0 0.0 0.0 146.7 18.90 7.99 33 0.573 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.3 total, 20.2 interval Flush(GB): cumulative 2.707, interval 2.707 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 21.90 GB write, 1103.32 MB/s write, 19.40 GB read, 977.34 MB/s read, 182.8 seconds Interval compaction: 21.90 GB write, 1112.40 MB/s write, 19.40 GB read, 985.38 MB/s read, 182.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 66, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 66, interval: 66 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 803001 Average: 8.8499 StdDev: 137.26 Min: 1 Median: 1.8696 Max: 46727 Percentiles: P50: 1.87 P75: 2.82 P99: 9.99 P99.9: 1327.58 P99.99: 2987.50 ------------------------------------------------------ [ 0, 1 ] 31556 3.930% 3.930% # ( 1, 2 ] 425406 52.977% 56.907% ########### ( 2, 3 ] 176381 21.965% 78.872% #### ( 3, 4 ] 67403 8.394% 87.266% ## ( 4, 6 ] 78710 9.802% 97.068% ## ( 6, 10 ] 15570 1.939% 99.007% ( 10, 15 ] 3738 0.466% 99.472% ( 15, 22 ] 345 0.043% 99.515% ( 22, 34 ] 32 0.004% 99.519% ( 34, 51 ] 14 0.002% 99.521% ( 51, 76 ] 26 0.003% 99.524% ( 76, 110 ] 13 0.002% 99.526% ( 110, 170 ] 146 0.018% 99.544% ( 170, 250 ] 265 0.033% 99.577% ( 250, 380 ] 452 0.056% 99.633% ( 380, 580 ] 629 0.078% 99.712% ( 580, 870 ] 808 0.101% 99.812% ( 870, 1300 ] 684 0.085% 99.898% ( 1300, 1900 ] 435 0.054% 99.952% ( 1900, 2900 ] 293 0.036% 99.988% ( 2900, 4400 ] 252 0.031% 100.020% ( 4400, 6600 ] 101 0.013% 100.032% ( 6600, 9900 ] 13 0.002% 100.034% ( 9900, 14000 ] 3 0.000% 100.034% ( 14000, 22000 ] 2 0.000% 100.034% ( 33000, 50000 ] 2 0.000% 100.035% ** Level 1 read latency histogram (micros): Count: 369638 Average: 14.2792 StdDev: 267.44 Min: 1 Median: 2.0102 Max: 58942 Percentiles: P50: 2.01 P75: 3.03 P99: 12.74 P99.9: 2616.48 P99.99: 4263.34 ------------------------------------------------------ [ 0, 1 ] 7163 1.938% 1.938% ( 1, 2 ] 176717 47.808% 49.746% ########## ( 2, 3 ] 92130 24.924% 74.670% ##### ( 3, 4 ] 35925 9.719% 84.389% ## ( 4, 6 ] 43300 11.714% 96.103% ## ( 6, 10 ] 9243 2.501% 98.604% # ( 10, 15 ] 2670 0.722% 99.326% ( 15, 22 ] 254 0.069% 99.395% ( 22, 34 ] 26 0.007% 99.402% ( 34, 51 ] 20 0.005% 99.408% ( 51, 76 ] 34 0.009% 99.417% ( 76, 110 ] 9 0.002% 99.419% ( 110, 170 ] 31 0.008% 99.428% ( 170, 250 ] 68 0.018% 99.446% ( 250, 380 ] 118 0.032% 99.478% ( 380, 580 ] 231 0.062% 99.540% ( 580, 870 ] 383 0.104% 99.644% ( 870, 1300 ] 406 0.110% 99.754% ( 1300, 1900 ] 329 0.089% 99.843% ( 1900, 2900 ] 295 0.080% 99.923% ( 2900, 4400 ] 274 0.074% 99.997% ( 4400, 6600 ] 93 0.025% 100.022% ( 6600, 9900 ] 21 0.006% 100.028% ( 9900, 14000 ] 1 0.000% 100.028% ( 14000, 22000 ] 6 0.002% 100.029% ( 22000, 33000 ] 4 0.001% 100.031% ( 33000, 50000 ] 3 0.001% 100.031% ( 50000, 75000 ] 2 0.001% 100.032% ** Level 2 read latency histogram (micros): Count: 541797 Average: 18.0067 StdDev: 234.21 Min: 1 Median: 1.9562 Max: 28668 Percentiles: P50: 1.96 P75: 3.05 P99: 13.32 P99.9: 2650.79 P99.99: 3485.13 ------------------------------------------------------ [ 0, 1 ] 17168 3.169% 3.169% # ( 1, 2 ] 265349 48.976% 52.144% ########## ( 2, 3 ] 121141 22.359% 74.504% #### ( 3, 4 ] 53079 9.797% 84.300% ## ( 4, 6 ] 61589 11.368% 95.668% ## ( 6, 10 ] 14805 2.733% 98.401% # ( 10, 15 ] 4892 0.903% 99.303% ( 15, 22 ] 706 0.130% 99.434% ( 22, 34 ] 82 0.015% 99.449% ( 34, 51 ] 25 0.005% 99.453% ( 51, 76 ] 24 0.004% 99.458% ( 76, 110 ] 11 0.002% 99.460% ( 110, 170 ] 13 0.002% 99.462% ( 170, 250 ] 24 0.004% 99.467% ( 250, 380 ] 87 0.016% 99.483% ( 380, 580 ] 155 0.029% 99.511% ( 580, 870 ] 307 0.057% 99.568% ( 870, 1300 ] 477 0.088% 99.656% ( 1300, 1900 ] 656 0.121% 99.777% ( 1900, 2900 ] 886 0.164% 99.941% ( 2900, 4400 ] 684 0.126% 100.067% ( 4400, 6600 ] 228 0.042% 100.109% ( 6600, 9900 ] 40 0.007% 100.116% ( 9900, 14000 ] 5 0.001% 100.117% ( 14000, 22000 ] 5 0.001% 100.118% ( 22000, 33000 ] 5 0.001% 100.119% ** Level 3 read latency histogram (micros): Count: 2346047 Average: 11.7123 StdDev: 171.29 Min: 1 Median: 2.0123 Max: 32163 Percentiles: P50: 2.01 P75: 3.02 P99: 11.13 P99.9: 1108.27 P99.99: 1724.77 ------------------------------------------------------ [ 0, 1 ] 40049 1.707% 1.707% ( 1, 2 ] 1125737 47.984% 49.692% ########## ( 2, 3 ] 588056 25.066% 74.757% ##### ( 3, 4 ] 228897 9.757% 84.514% ## ( 4, 6 ] 277322 11.821% 96.335% ## ( 6, 10 ] 58603 2.498% 98.833% ( 10, 15 ] 17366 0.740% 99.573% ( 15, 22 ] 1789 0.076% 99.649% ( 22, 34 ] 154 0.007% 99.656% ( 34, 51 ] 158 0.007% 99.663% ( 51, 76 ] 252 0.011% 99.673% ( 76, 110 ] 63 0.003% 99.676% ( 110, 170 ] 172 0.007% 99.683% ( 170, 250 ] 373 0.016% 99.699% ( 250, 380 ] 676 0.029% 99.728% ( 380, 580 ] 1121 0.048% 99.776% ( 580, 870 ] 1783 0.076% 99.852% ( 870, 1300 ] 2041 0.087% 99.939% ( 1300, 1900 ] 1697 0.072% 100.011% ( 1900, 2900 ] 1807 0.077% 100.088% ( 2900, 4400 ] 1540 0.066% 100.154% ( 4400, 6600 ] 473 0.020% 100.174% ( 6600, 9900 ] 78 0.003% 100.177% ( 9900, 14000 ] 11 0.000% 100.178% ( 14000, 22000 ] 15 0.001% 100.178% ( 22000, 33000 ] 11 0.000% 100.179% ** Level 4 read latency histogram (micros): Count: 1131043 Average: 23.5584 StdDev: 285.51 Min: 1 Median: 2.0107 Max: 48333 Percentiles: P50: 2.01 P75: 3.10 P99: 430.59 P99.9: 2800.54 P99.99: 3736.04 ------------------------------------------------------ [ 0, 1 ] 21381 1.890% 1.890% ( 1, 2 ] 541213 47.851% 49.741% ########## ( 2, 3 ] 274453 24.265% 74.007% ##### ( 3, 4 ] 111553 9.863% 83.869% ## ( 4, 6 ] 129058 11.411% 95.280% ## ( 6, 10 ] 29441 2.603% 97.883% # ( 10, 15 ] 8994 0.795% 98.678% ( 15, 22 ] 927 0.082% 98.760% ( 22, 34 ] 83 0.007% 98.768% ( 34, 51 ] 100 0.009% 98.776% ( 51, 76 ] 170 0.015% 98.791% ( 76, 110 ] 42 0.004% 98.795% ( 110, 170 ] 251 0.022% 98.817% ( 170, 250 ] 619 0.055% 98.872% ( 250, 380 ] 974 0.086% 98.958% ( 380, 580 ] 1876 0.166% 99.124% ( 580, 870 ] 2725 0.241% 99.365% ( 870, 1300 ] 2692 0.238% 99.603% ( 1300, 1900 ] 1921 0.170% 99.773% ( 1900, 2900 ] 1599 0.141% 99.914% ( 2900, 4400 ] 1541 0.136% 100.050% ( 4400, 6600 ] 521 0.046% 100.096% ( 6600, 9900 ] 83 0.007% 100.104% ( 9900, 14000 ] 24 0.002% 100.106% ( 14000, 22000 ] 21 0.002% 100.108% ( 22000, 33000 ] 24 0.002% 100.110% ( 33000, 50000 ] 5 0.000% 100.110% ** Level 5 read latency histogram (micros): Count: 42863 Average: 1392.0348 StdDev: 1689.55 Min: 87 Median: 859.1405 Max: 51015 Percentiles: P50: 859.14 P75: 1707.39 P99: 6502.87 P99.9: 16888.00 P99.99: 46713.70 ------------------------------------------------------ ( 76, 110 ] 100 0.233% 0.233% ( 110, 170 ] 1137 2.653% 2.886% # ( 170, 250 ] 2381 5.555% 8.441% # ( 250, 380 ] 3862 9.010% 17.451% ## ( 380, 580 ] 6073 14.168% 31.619% ### ( 580, 870 ] 8185 19.096% 50.715% #### ( 870, 1300 ] 7345 17.136% 67.851% ### ( 1300, 1900 ] 4513 10.529% 78.380% ## ( 1900, 2900 ] 3618 8.441% 86.821% ## ( 2900, 4400 ] 3518 8.208% 95.028% ## ( 4400, 6600 ] 1781 4.155% 99.183% # ( 6600, 9900 ] 282 0.658% 99.841% ( 9900, 14000 ] 19 0.044% 99.886% ( 14000, 22000 ] 17 0.040% 99.925% ( 22000, 33000 ] 14 0.033% 99.958% ( 33000, 50000 ] 17 0.040% 99.998% ( 50000, 75000 ] 1 0.002% 100.000% ** DB Stats ** Uptime(secs): 20.3 total, 5.8 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 145.29 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:8.290 H:M:S, 40.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 171.02 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.737 H:M:S, 30.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 8 num-running-flushes: 1 2024/07/15-00:24:26 ... thread 0: (1000000,4000000) ops and (262217.0,166891.9) ops/second in (3.813635,23.967610) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/10 2.70 GB 132.2 2.4 0.0 2.4 6.1 3.7 0.0 1.6 71.9 184.3 34.16 20.55 57 0.599 2494K 2908 0.0 0.0 L1 2/2 219.87 MB 0.0 1.6 1.1 0.5 1.6 1.1 0.0 1.4 131.7 131.4 12.55 7.73 3 4.183 1678K 3825 0.0 0.0 L2 2/0 94.89 MB 0.9 0.3 0.2 0.1 0.3 0.2 0.8 1.7 160.5 160.0 1.97 1.56 1 1.974 321K 956 0.0 0.0 L3 17/0 930.27 MB 0.9 7.3 3.1 4.3 7.3 3.0 0.3 2.4 115.5 114.6 64.95 35.70 29 2.240 5274K 17K 0.0 0.0 L4 173/0 9.69 GB 1.0 10.1 5.6 4.5 9.9 5.4 0.4 1.8 120.1 118.0 86.27 49.19 64 1.348 5385K 11K 0.0 0.0 L5 1156/0 95.48 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1366/12 109.08 GB 0.0 21.8 10.0 11.8 25.3 13.5 7.2 6.7 111.5 129.5 199.90 114.73 154 1.298 15M 36K 0.0 0.0 Int 0/0 0.00 KB 0.0 21.8 10.0 11.8 25.3 13.5 7.2 6.7 111.5 129.5 199.90 114.73 154 1.298 15M 36K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 21.8 10.0 11.8 21.5 9.7 0.0 0.0 125.8 124.5 177.08 103.80 105 1.686 15M 36K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 168.3 22.83 10.93 49 0.466 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 24.1 total, 24.0 interval Flush(GB): cumulative 3.752, interval 3.752 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 25.28 GB write, 1072.51 MB/s write, 21.76 GB read, 923.28 MB/s read, 199.9 seconds Interval compaction: 25.28 GB write, 1079.93 MB/s write, 21.76 GB read, 929.67 MB/s read, 199.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 75, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 75, interval: 75 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1087786 Average: 10.5997 StdDev: 123.58 Min: 1 Median: 1.7668 Max: 46727 Percentiles: P50: 1.77 P75: 2.66 P99: 199.92 P99.9: 1105.95 P99.99: 2386.33 ------------------------------------------------------ [ 0, 1 ] 87391 8.034% 8.034% ## ( 1, 2 ] 595343 54.730% 62.764% ########### ( 2, 3 ] 203186 18.679% 81.442% #### ( 3, 4 ] 75511 6.942% 88.384% # ( 4, 6 ] 85010 7.815% 96.199% ## ( 6, 10 ] 16843 1.548% 97.748% ( 10, 15 ] 4079 0.375% 98.123% ( 15, 22 ] 388 0.036% 98.158% ( 22, 34 ] 35 0.003% 98.161% ( 34, 51 ] 20 0.002% 98.163% ( 51, 76 ] 28 0.003% 98.166% ( 76, 110 ] 529 0.049% 98.214% ( 110, 170 ] 6354 0.584% 98.799% ( 170, 250 ] 5862 0.539% 99.337% ( 250, 380 ] 2425 0.223% 99.560% ( 380, 580 ] 1981 0.182% 99.743% ( 580, 870 ] 1267 0.116% 99.859% ( 870, 1300 ] 815 0.075% 99.934% ( 1300, 1900 ] 459 0.042% 99.976% ( 1900, 2900 ] 313 0.029% 100.005% ( 2900, 4400 ] 264 0.024% 100.029% ( 4400, 6600 ] 102 0.009% 100.039% ( 6600, 9900 ] 13 0.001% 100.040% ( 9900, 14000 ] 3 0.000% 100.040% ( 14000, 22000 ] 2 0.000% 100.040% ( 33000, 50000 ] 2 0.000% 100.040% ** Level 1 read latency histogram (micros): Count: 388223 Average: 13.9827 StdDev: 261.17 Min: 1 Median: 2.0297 Max: 58942 Percentiles: P50: 2.03 P75: 3.09 P99: 13.35 P99.9: 2558.03 P99.99: 4253.70 ------------------------------------------------------ [ 0, 1 ] 7636 1.967% 1.967% ( 1, 2 ] 183610 47.295% 49.262% ######### ( 2, 3 ] 96627 24.890% 74.151% ##### ( 3, 4 ] 38380 9.886% 84.038% ## ( 4, 6 ] 46143 11.886% 95.923% ## ( 6, 10 ] 10009 2.578% 98.501% # ( 10, 15 ] 2887 0.744% 99.245% ( 15, 22 ] 322 0.083% 99.328% ( 22, 34 ] 36 0.009% 99.337% ( 34, 51 ] 21 0.005% 99.343% ( 51, 76 ] 36 0.009% 99.352% ( 76, 110 ] 28 0.007% 99.359% ( 110, 170 ] 182 0.047% 99.406% ( 170, 250 ] 181 0.047% 99.453% ( 250, 380 ] 155 0.040% 99.493% ( 380, 580 ] 255 0.066% 99.558% ( 580, 870 ] 391 0.101% 99.659% ( 870, 1300 ] 411 0.106% 99.765% ( 1300, 1900 ] 330 0.085% 99.850% ( 1900, 2900 ] 296 0.076% 99.926% ( 2900, 4400 ] 275 0.071% 99.997% ( 4400, 6600 ] 93 0.024% 100.021% ( 6600, 9900 ] 21 0.005% 100.026% ( 9900, 14000 ] 1 0.000% 100.027% ( 14000, 22000 ] 6 0.002% 100.028% ( 22000, 33000 ] 4 0.001% 100.029% ( 33000, 50000 ] 3 0.001% 100.030% ( 50000, 75000 ] 2 0.001% 100.030% ** Level 2 read latency histogram (micros): Count: 542027 Average: 18.1040 StdDev: 234.24 Min: 1 Median: 1.9566 Max: 28668 Percentiles: P50: 1.96 P75: 3.05 P99: 13.55 P99.9: 2650.53 P99.99: 3485.08 ------------------------------------------------------ [ 0, 1 ] 17168 3.167% 3.167% # ( 1, 2 ] 265349 48.955% 52.122% ########## ( 2, 3 ] 121141 22.350% 74.472% #### ( 3, 4 ] 53079 9.793% 84.265% ## ( 4, 6 ] 61589 11.363% 95.627% ## ( 6, 10 ] 14805 2.731% 98.359% # ( 10, 15 ] 4892 0.903% 99.261% ( 15, 22 ] 706 0.130% 99.392% ( 22, 34 ] 82 0.015% 99.407% ( 34, 51 ] 25 0.005% 99.411% ( 51, 76 ] 24 0.004% 99.416% ( 76, 110 ] 20 0.004% 99.419% ( 110, 170 ] 96 0.018% 99.437% ( 170, 250 ] 106 0.020% 99.457% ( 250, 380 ] 113 0.021% 99.478% ( 380, 580 ] 170 0.031% 99.509% ( 580, 870 ] 317 0.058% 99.567% ( 870, 1300 ] 482 0.089% 99.656% ( 1300, 1900 ] 656 0.121% 99.777% ( 1900, 2900 ] 886 0.163% 99.941% ( 2900, 4400 ] 684 0.126% 100.067% ( 4400, 6600 ] 228 0.042% 100.109% ( 6600, 9900 ] 40 0.007% 100.116% ( 9900, 14000 ] 5 0.001% 100.117% ( 14000, 22000 ] 5 0.001% 100.118% ( 22000, 33000 ] 5 0.001% 100.119% ** Level 3 read latency histogram (micros): Count: 2397146 Average: 11.5963 StdDev: 169.61 Min: 1 Median: 2.0069 Max: 32163 Percentiles: P50: 2.01 P75: 3.02 P99: 11.21 P99.9: 1090.95 P99.99: 1710.11 ------------------------------------------------------ [ 0, 1 ] 41827 1.745% 1.745% ( 1, 2 ] 1152628 48.083% 49.828% ########## ( 2, 3 ] 599470 25.008% 74.836% ##### ( 3, 4 ] 233194 9.728% 84.564% ## ( 4, 6 ] 282364 11.779% 96.343% ## ( 6, 10 ] 59454 2.480% 98.823% ( 10, 15 ] 17529 0.731% 99.554% ( 15, 22 ] 1805 0.075% 99.630% ( 22, 34 ] 156 0.007% 99.636% ( 34, 51 ] 164 0.007% 99.643% ( 51, 76 ] 253 0.011% 99.654% ( 76, 110 ] 87 0.004% 99.657% ( 110, 170 ] 450 0.019% 99.676% ( 170, 250 ] 588 0.025% 99.701% ( 250, 380 ] 756 0.032% 99.732% ( 380, 580 ] 1168 0.049% 99.781% ( 580, 870 ] 1803 0.075% 99.856% ( 870, 1300 ] 2049 0.085% 99.942% ( 1300, 1900 ] 1699 0.071% 100.012% ( 1900, 2900 ] 1809 0.075% 100.088% ( 2900, 4400 ] 1544 0.064% 100.152% ( 4400, 6600 ] 473 0.020% 100.172% ( 6600, 9900 ] 78 0.003% 100.175% ( 9900, 14000 ] 11 0.000% 100.176% ( 14000, 22000 ] 15 0.001% 100.176% ( 22000, 33000 ] 11 0.000% 100.177% ** Level 4 read latency histogram (micros): Count: 1192657 Average: 23.9313 StdDev: 279.70 Min: 0 Median: 2.0167 Max: 48333 Percentiles: P50: 2.02 P75: 3.13 P99: 437.43 P99.9: 2733.44 P99.99: 3670.87 ------------------------------------------------------ [ 0, 1 ] 23126 1.939% 1.939% ( 1, 2 ] 568411 47.659% 49.598% ########## ( 2, 3 ] 287485 24.105% 73.703% ##### ( 3, 4 ] 116599 9.776% 83.479% ## ( 4, 6 ] 135056 11.324% 94.803% ## ( 6, 10 ] 30982 2.598% 97.401% # ( 10, 15 ] 9444 0.792% 98.193% ( 15, 22 ] 981 0.082% 98.275% ( 22, 34 ] 86 0.007% 98.282% ( 34, 51 ] 104 0.009% 98.291% ( 51, 76 ] 179 0.015% 98.306% ( 76, 110 ] 275 0.023% 98.329% ( 110, 170 ] 2861 0.240% 98.569% ( 170, 250 ] 2764 0.232% 98.801% ( 250, 380 ] 1694 0.142% 98.943% ( 380, 580 ] 2380 0.200% 99.142% ( 580, 870 ] 2976 0.250% 99.392% ( 870, 1300 ] 2768 0.232% 99.624% ( 1300, 1900 ] 1939 0.163% 99.786% ( 1900, 2900 ] 1625 0.136% 99.923% ( 2900, 4400 ] 1562 0.131% 100.054% ( 4400, 6600 ] 527 0.044% 100.098% ( 6600, 9900 ] 84 0.007% 100.105% ( 9900, 14000 ] 24 0.002% 100.107% ( 14000, 22000 ] 21 0.002% 100.109% ( 22000, 33000 ] 24 0.002% 100.111% ( 33000, 50000 ] 5 0.000% 100.111% ** Level 5 read latency histogram (micros): Count: 82452 Average: 841.3670 StdDev: 1355.95 Min: 84 Median: 370.0759 Max: 51015 Percentiles: P50: 370.08 P75: 933.09 P99: 6017.44 P99.9: 9719.82 P99.99: 41754.80 ------------------------------------------------------ ( 76, 110 ] 1472 1.785% 1.785% ( 110, 170 ] 16658 20.203% 21.989% #### ( 170, 250 ] 15425 18.708% 40.696% #### ( 250, 380 ] 8305 10.073% 50.769% ## ( 380, 580 ] 9345 11.334% 62.103% ## ( 580, 870 ] 9497 11.518% 73.621% ## ( 870, 1300 ] 7749 9.398% 83.019% ## ( 1300, 1900 ] 4632 5.618% 88.637% # ( 1900, 2900 ] 3669 4.450% 93.087% # ( 2900, 4400 ] 3558 4.315% 97.402% # ( 4400, 6600 ] 1792 2.173% 99.576% ( 6600, 9900 ] 283 0.343% 99.919% ( 9900, 14000 ] 19 0.023% 99.942% ( 14000, 22000 ] 17 0.021% 99.962% ( 22000, 33000 ] 14 0.017% 99.979% ( 33000, 50000 ] 17 0.021% 100.000% ( 50000, 75000 ] 1 0.001% 100.001% ** DB Stats ** Uptime(secs): 24.1 total, 3.8 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 163.11 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:8.545 H:M:S, 35.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 258.07 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.255 H:M:S, 6.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/15-00:24:31 ... thread 0: (1000000,5000000) ops and (205548.4,173414.5) ops/second in (4.865035,28.832645) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/6 3.75 GB 264.3 3.6 0.0 3.6 8.4 4.8 0.0 1.8 87.5 203.0 42.50 28.22 79 0.538 3773K 4129 0.0 0.0 L1 2/2 219.87 MB 0.0 1.6 1.1 0.5 1.6 1.1 0.0 1.4 131.7 131.4 12.55 7.73 3 4.183 1678K 3825 0.0 0.0 L2 2/0 94.89 MB 0.9 0.3 0.2 0.1 0.3 0.2 0.8 1.7 160.5 160.0 1.97 1.56 1 1.974 321K 956 0.0 0.0 L3 17/0 930.27 MB 0.9 7.3 3.1 4.3 7.3 3.0 0.3 2.4 115.5 114.6 64.95 35.70 29 2.240 5274K 17K 0.0 0.0 L4 173/0 9.69 GB 1.0 10.1 5.6 4.5 9.9 5.4 0.4 1.8 120.1 118.0 86.27 49.19 64 1.348 5385K 11K 0.0 0.0 L5 1156/0 95.48 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1368/8 110.13 GB 0.0 23.0 10.0 13.0 27.6 14.5 7.2 5.7 113.1 135.5 208.24 122.40 176 1.183 16M 37K 0.0 0.0 Int 0/0 0.00 KB 0.0 23.0 10.0 13.0 27.6 14.5 7.2 5.7 113.1 135.5 208.24 122.40 176 1.183 16M 37K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 23.0 10.0 13.0 22.8 9.7 0.0 0.0 129.4 128.1 181.95 108.54 110 1.654 16M 37K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.8 4.8 0.0 0.0 0.0 186.9 26.29 13.87 66 0.398 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 29.0 total, 28.8 interval Flush(GB): cumulative 4.798, interval 4.798 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 27.55 GB write, 972.92 MB/s write, 22.99 GB read, 811.84 MB/s read, 208.2 seconds Interval compaction: 27.55 GB write, 978.52 MB/s write, 22.99 GB read, 816.51 MB/s read, 208.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 9, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 9, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 75, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 9, total-stops: 75, interval: 84 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1425360 Average: 12.3654 StdDev: 113.21 Min: 1 Median: 1.6959 Max: 46727 Percentiles: P50: 1.70 P75: 2.51 P99: 235.98 P99.9: 896.68 P99.99: 1840.97 ------------------------------------------------------ [ 0, 1 ] 164016 11.507% 11.507% ## ( 1, 2 ] 788399 55.312% 66.819% ########### ( 2, 3 ] 229009 16.067% 82.886% ### ( 3, 4 ] 84182 5.906% 88.792% # ( 4, 6 ] 91394 6.412% 95.204% # ( 6, 10 ] 17903 1.256% 96.460% ( 10, 15 ] 4341 0.305% 96.765% ( 15, 22 ] 451 0.032% 96.796% ( 22, 34 ] 50 0.004% 96.800% ( 34, 51 ] 32 0.002% 96.802% ( 51, 76 ] 42 0.003% 96.805% ( 76, 110 ] 1554 0.109% 96.914% ( 110, 170 ] 17904 1.256% 98.170% ( 170, 250 ] 14344 1.006% 99.176% ( 250, 380 ] 4922 0.345% 99.522% ( 380, 580 ] 3571 0.251% 99.772% ( 580, 870 ] 1764 0.124% 99.896% ( 870, 1300 ] 913 0.064% 99.960% ( 1300, 1900 ] 473 0.033% 99.993% ( 1900, 2900 ] 327 0.023% 100.016% ( 2900, 4400 ] 274 0.019% 100.035% ( 4400, 6600 ] 102 0.007% 100.043% ( 6600, 9900 ] 13 0.001% 100.043% ( 9900, 14000 ] 3 0.000% 100.044% ( 14000, 22000 ] 2 0.000% 100.044% ( 33000, 50000 ] 2 0.000% 100.044% ** Level 1 read latency histogram (micros): Count: 402243 Average: 13.9304 StdDev: 256.72 Min: 1 Median: 2.0715 Max: 58942 Percentiles: P50: 2.07 P75: 3.17 P99: 14.38 P99.9: 2510.67 P99.99: 4246.05 ------------------------------------------------------ [ 0, 1 ] 7715 1.918% 1.918% ( 1, 2 ] 186189 46.288% 48.206% ######### ( 2, 3 ] 100907 25.086% 73.292% ##### ( 3, 4 ] 40667 10.110% 83.402% ## ( 4, 6 ] 49333 12.264% 95.666% ## ( 6, 10 ] 10678 2.655% 98.321% # ( 10, 15 ] 3117 0.775% 99.096% ( 15, 22 ] 412 0.102% 99.198% ( 22, 34 ] 46 0.011% 99.210% ( 34, 51 ] 24 0.006% 99.216% ( 51, 76 ] 37 0.009% 99.225% ( 76, 110 ] 68 0.017% 99.242% ( 110, 170 ] 439 0.109% 99.351% ( 170, 250 ] 384 0.095% 99.446% ( 250, 380 ] 218 0.054% 99.501% ( 380, 580 ] 284 0.071% 99.571% ( 580, 870 ] 399 0.099% 99.670% ( 870, 1300 ] 413 0.103% 99.773% ( 1300, 1900 ] 330 0.082% 99.855% ( 1900, 2900 ] 296 0.074% 99.929% ( 2900, 4400 ] 275 0.068% 99.997% ( 4400, 6600 ] 93 0.023% 100.020% ( 6600, 9900 ] 21 0.005% 100.025% ( 9900, 14000 ] 1 0.000% 100.026% ( 14000, 22000 ] 6 0.001% 100.027% ( 22000, 33000 ] 4 0.001% 100.028% ( 33000, 50000 ] 3 0.001% 100.029% ( 50000, 75000 ] 2 0.000% 100.029% ** Level 2 read latency histogram (micros): Count: 542351 Average: 18.2115 StdDev: 234.22 Min: 1 Median: 1.9573 Max: 28668 Percentiles: P50: 1.96 P75: 3.06 P99: 13.88 P99.9: 2650.17 P99.99: 3485.01 ------------------------------------------------------ [ 0, 1 ] 17168 3.165% 3.165% # ( 1, 2 ] 265349 48.926% 52.091% ########## ( 2, 3 ] 121141 22.336% 74.427% #### ( 3, 4 ] 53079 9.787% 84.214% ## ( 4, 6 ] 61589 11.356% 95.570% ## ( 6, 10 ] 14805 2.730% 98.300% # ( 10, 15 ] 4892 0.902% 99.202% ( 15, 22 ] 706 0.130% 99.332% ( 22, 34 ] 82 0.015% 99.347% ( 34, 51 ] 25 0.005% 99.352% ( 51, 76 ] 24 0.004% 99.356% ( 76, 110 ] 32 0.006% 99.362% ( 110, 170 ] 262 0.048% 99.411% ( 170, 250 ] 199 0.037% 99.447% ( 250, 380 ] 143 0.026% 99.474% ( 380, 580 ] 188 0.035% 99.508% ( 580, 870 ] 322 0.059% 99.568% ( 870, 1300 ] 482 0.089% 99.656% ( 1300, 1900 ] 656 0.121% 99.777% ( 1900, 2900 ] 886 0.163% 99.941% ( 2900, 4400 ] 684 0.126% 100.067% ( 4400, 6600 ] 228 0.042% 100.109% ( 6600, 9900 ] 40 0.007% 100.116% ( 9900, 14000 ] 5 0.001% 100.117% ( 14000, 22000 ] 5 0.001% 100.118% ( 22000, 33000 ] 5 0.001% 100.119% ** Level 3 read latency histogram (micros): Count: 2398214 Average: 11.6868 StdDev: 169.67 Min: 1 Median: 2.0078 Max: 32163 Percentiles: P50: 2.01 P75: 3.02 P99: 11.51 P99.9: 1091.56 P99.99: 1711.13 ------------------------------------------------------ [ 0, 1 ] 41827 1.744% 1.744% ( 1, 2 ] 1152628 48.062% 49.806% ########## ( 2, 3 ] 599471 24.997% 74.803% ##### ( 3, 4 ] 233195 9.724% 84.526% ## ( 4, 6 ] 282366 11.774% 96.300% ## ( 6, 10 ] 59455 2.479% 98.779% ( 10, 15 ] 17529 0.731% 99.510% ( 15, 22 ] 1805 0.075% 99.586% ( 22, 34 ] 156 0.007% 99.592% ( 34, 51 ] 164 0.007% 99.599% ( 51, 76 ] 253 0.011% 99.610% ( 76, 110 ] 131 0.005% 99.615% ( 110, 170 ] 917 0.038% 99.653% ( 170, 250 ] 958 0.040% 99.693% ( 250, 380 ] 838 0.035% 99.728% ( 380, 580 ] 1243 0.052% 99.780% ( 580, 870 ] 1823 0.076% 99.856% ( 870, 1300 ] 2051 0.086% 99.941% ( 1300, 1900 ] 1699 0.071% 100.012% ( 1900, 2900 ] 1811 0.076% 100.088% ( 2900, 4400 ] 1545 0.064% 100.152% ( 4400, 6600 ] 473 0.020% 100.172% ( 6600, 9900 ] 78 0.003% 100.175% ( 9900, 14000 ] 11 0.000% 100.176% ( 14000, 22000 ] 15 0.001% 100.176% ( 22000, 33000 ] 11 0.000% 100.177% ** Level 4 read latency histogram (micros): Count: 1203500 Average: 25.4784 StdDev: 279.20 Min: 0 Median: 2.0354 Max: 48333 Percentiles: P50: 2.04 P75: 3.20 P99: 474.79 P99.9: 2727.91 P99.99: 3670.30 ------------------------------------------------------ [ 0, 1 ] 23126 1.922% 1.922% ( 1, 2 ] 568433 47.232% 49.153% ######### ( 2, 3 ] 287659 23.902% 73.055% ##### ( 3, 4 ] 116693 9.696% 82.751% ## ( 4, 6 ] 135260 11.239% 93.990% ## ( 6, 10 ] 31010 2.577% 96.567% # ( 10, 15 ] 9457 0.786% 97.353% ( 15, 22 ] 984 0.082% 97.434% ( 22, 34 ] 86 0.007% 97.441% ( 34, 51 ] 104 0.009% 97.450% ( 51, 76 ] 179 0.015% 97.465% ( 76, 110 ] 782 0.065% 97.530% ( 110, 170 ] 7714 0.641% 98.171% ( 170, 250 ] 5955 0.495% 98.666% ( 250, 380 ] 2594 0.216% 98.881% ( 380, 580 ] 3015 0.251% 99.132% ( 580, 870 ] 3154 0.262% 99.394% ( 870, 1300 ] 2799 0.233% 99.626% ( 1300, 1900 ] 1943 0.161% 99.788% ( 1900, 2900 ] 1630 0.135% 99.923% ( 2900, 4400 ] 1563 0.130% 100.053% ( 4400, 6600 ] 527 0.044% 100.097% ( 6600, 9900 ] 84 0.007% 100.104% ( 9900, 14000 ] 24 0.002% 100.106% ( 14000, 22000 ] 21 0.002% 100.108% ( 22000, 33000 ] 24 0.002% 100.110% ( 33000, 50000 ] 5 0.000% 100.110% ** Level 5 read latency histogram (micros): Count: 140694 Average: 578.1070 StdDev: 1087.61 Min: 82 Median: 222.3960 Max: 51015 Percentiles: P50: 222.40 P75: 538.08 P99: 5298.73 P99.9: 9005.69 P99.99: 32945.47 ------------------------------------------------------ ( 76, 110 ] 4119 2.928% 2.928% # ( 110, 170 ] 43887 31.193% 34.121% ###### ( 170, 250 ] 34111 24.245% 58.366% ##### ( 250, 380 ] 13326 9.472% 67.837% ## ( 380, 580 ] 12750 9.062% 76.900% ## ( 580, 870 ] 10532 7.486% 84.385% # ( 870, 1300 ] 7911 5.623% 90.008% # ( 1300, 1900 ] 4657 3.310% 93.318% # ( 1900, 2900 ] 3700 2.630% 95.948% # ( 2900, 4400 ] 3562 2.532% 98.480% # ( 4400, 6600 ] 1792 1.274% 99.753% ( 6600, 9900 ] 283 0.201% 99.955% ( 9900, 14000 ] 19 0.014% 99.968% ( 14000, 22000 ] 17 0.012% 99.980% ( 22000, 33000 ] 14 0.010% 99.990% ( 33000, 50000 ] 17 0.012% 100.002% ( 50000, 75000 ] 1 0.001% 100.003% ** DB Stats ** Uptime(secs): 29.0 total, 4.9 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 169.68 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.825 H:M:S, 33.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 202.30 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.279 H:M:S, 26.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 2024/07/15-00:24:46 ... thread 0: (1000000,6000000) ops and (65843.6,136301.2) ops/second in (15.187508,44.020153) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/9 3.49 GB 0.0 6.2 0.0 6.2 11.9 5.7 0.0 2.1 114.0 219.8 55.25 40.34 99 0.558 6394K 15K 0.0 0.0 L1 16/16 1.37 GB 0.0 3.0 2.3 0.7 3.0 2.3 0.0 1.3 85.3 84.9 35.98 17.28 4 8.994 3115K 12K 0.0 0.0 L2 2/0 94.89 MB 0.9 0.3 0.2 0.1 0.3 0.2 0.8 1.7 160.5 160.0 1.97 1.56 1 1.974 321K 956 0.0 0.0 L3 17/0 930.27 MB 0.9 7.3 3.1 4.3 7.3 3.0 0.3 2.4 115.5 114.6 64.95 35.70 29 2.240 5274K 17K 0.0 0.0 L4 173/0 9.69 GB 1.0 10.1 5.6 4.5 9.9 5.4 0.4 1.8 120.1 118.0 86.27 49.19 64 1.348 5385K 11K 0.0 0.0 L5 1156/0 95.48 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1373/25 111.03 GB 0.0 26.9 11.1 15.8 32.4 16.6 7.2 5.7 112.7 135.6 244.41 144.07 197 1.241 20M 58K 0.0 0.0 Int 0/0 0.00 KB 0.0 26.9 11.1 15.8 32.4 16.6 7.2 5.7 112.7 135.6 244.41 144.07 197 1.241 20M 58K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 26.9 11.1 15.8 26.6 10.9 0.0 0.0 128.0 126.8 215.12 127.66 116 1.854 20M 58K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 200.0 29.30 16.41 81 0.362 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 44.2 total, 44.0 interval Flush(GB): cumulative 5.721, interval 5.721 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 32.36 GB write, 749.90 MB/s write, 26.90 GB read, 623.26 MB/s read, 244.4 seconds Interval compaction: 32.36 GB write, 752.73 MB/s write, 26.90 GB read, 625.61 MB/s read, 244.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 17, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 17, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 75, pending-compaction-bytes-delays: 3, pending-compaction-bytes-stops: 0, total-delays: 20, total-stops: 75, interval: 95 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2296929 Average: 15.3956 StdDev: 98.10 Min: 0 Median: 1.6146 Max: 46727 Percentiles: P50: 1.61 P75: 2.31 P99: 245.45 P99.9: 740.06 P99.99: 1236.91 ------------------------------------------------------ [ 0, 1 ] 375514 16.349% 16.349% ### ( 1, 2 ] 1257623 54.752% 71.101% ########### ( 2, 3 ] 288502 12.560% 83.661% ### ( 3, 4 ] 108133 4.708% 88.369% # ( 4, 6 ] 107843 4.695% 93.064% # ( 6, 10 ] 21770 0.948% 94.012% ( 10, 15 ] 5475 0.238% 94.250% ( 15, 22 ] 652 0.028% 94.279% ( 22, 34 ] 63 0.003% 94.281% ( 34, 51 ] 39 0.002% 94.283% ( 51, 76 ] 45 0.002% 94.285% ( 76, 110 ] 5235 0.228% 94.513% ( 110, 170 ] 63435 2.762% 97.275% # ( 170, 250 ] 42018 1.829% 99.104% ( 250, 380 ] 10118 0.441% 99.544% ( 380, 580 ] 6700 0.292% 99.836% ( 580, 870 ] 2658 0.116% 99.952% ( 870, 1300 ] 1027 0.045% 99.997% ( 1300, 1900 ] 493 0.021% 100.018% ( 1900, 2900 ] 350 0.015% 100.033% ( 2900, 4400 ] 290 0.013% 100.046% ( 4400, 6600 ] 104 0.005% 100.050% ( 6600, 9900 ] 13 0.001% 100.051% ( 9900, 14000 ] 3 0.000% 100.051% ( 14000, 22000 ] 2 0.000% 100.051% ( 33000, 50000 ] 2 0.000% 100.051% ** Level 1 read latency histogram (micros): Count: 427091 Average: 14.2926 StdDev: 249.70 Min: 1 Median: 2.1760 Max: 58942 Percentiles: P50: 2.18 P75: 3.46 P99: 136.63 P99.9: 2435.05 P99.99: 4233.71 ------------------------------------------------------ [ 0, 1 ] 7864 1.841% 1.841% ( 1, 2 ] 187541 43.911% 45.753% ######### ( 2, 3 ] 103072 24.133% 69.886% ##### ( 3, 4 ] 47639 11.154% 81.040% ## ( 4, 6 ] 59557 13.945% 94.985% ### ( 6, 10 ] 11964 2.801% 97.786% # ( 10, 15 ] 3492 0.818% 98.604% ( 15, 22 ] 715 0.167% 98.771% ( 22, 34 ] 73 0.017% 98.789% ( 34, 51 ] 28 0.007% 98.795% ( 51, 76 ] 37 0.009% 98.804% ( 76, 110 ] 179 0.042% 98.846% ( 110, 170 ] 1485 0.348% 99.193% ( 170, 250 ] 965 0.226% 99.419% ( 250, 380 ] 352 0.082% 99.502% ( 380, 580 ] 368 0.086% 99.588% ( 580, 870 ] 422 0.099% 99.687% ( 870, 1300 ] 422 0.099% 99.786% ( 1300, 1900 ] 330 0.077% 99.863% ( 1900, 2900 ] 297 0.070% 99.932% ( 2900, 4400 ] 277 0.065% 99.997% ( 4400, 6600 ] 93 0.022% 100.019% ( 6600, 9900 ] 21 0.005% 100.024% ( 9900, 14000 ] 1 0.000% 100.024% ( 14000, 22000 ] 6 0.001% 100.026% ( 22000, 33000 ] 4 0.001% 100.026% ( 33000, 50000 ] 3 0.001% 100.027% ( 50000, 75000 ] 2 0.000% 100.028% ** Level 2 read latency histogram (micros): Count: 543586 Average: 18.5917 StdDev: 234.17 Min: 1 Median: 1.9596 Max: 28668 Percentiles: P50: 1.96 P75: 3.08 P99: 16.26 P99.9: 2649.06 P99.99: 3484.74 ------------------------------------------------------ [ 0, 1 ] 17168 3.158% 3.158% # ( 1, 2 ] 265349 48.815% 51.973% ########## ( 2, 3 ] 121141 22.286% 74.258% #### ( 3, 4 ] 53079 9.765% 84.023% ## ( 4, 6 ] 61589 11.330% 95.353% ## ( 6, 10 ] 14805 2.724% 98.077% # ( 10, 15 ] 4892 0.900% 98.977% ( 15, 22 ] 706 0.130% 99.106% ( 22, 34 ] 82 0.015% 99.122% ( 34, 51 ] 25 0.005% 99.126% ( 51, 76 ] 24 0.004% 99.131% ( 76, 110 ] 98 0.018% 99.149% ( 110, 170 ] 929 0.171% 99.320% ( 170, 250 ] 565 0.104% 99.423% ( 250, 380 ] 224 0.041% 99.465% ( 380, 580 ] 229 0.042% 99.507% ( 580, 870 ] 331 0.061% 99.568% ( 870, 1300 ] 484 0.089% 99.657% ( 1300, 1900 ] 658 0.121% 99.778% ( 1900, 2900 ] 887 0.163% 99.941% ( 2900, 4400 ] 684 0.126% 100.067% ( 4400, 6600 ] 228 0.042% 100.109% ( 6600, 9900 ] 40 0.007% 100.116% ( 9900, 14000 ] 5 0.001% 100.117% ( 14000, 22000 ] 5 0.001% 100.118% ( 22000, 33000 ] 5 0.001% 100.119% ** Level 3 read latency histogram (micros): Count: 2402025 Average: 11.9579 StdDev: 169.71 Min: 1 Median: 2.0109 Max: 32163 Percentiles: P50: 2.01 P75: 3.03 P99: 12.59 P99.9: 1091.38 P99.99: 1711.35 ------------------------------------------------------ [ 0, 1 ] 41827 1.741% 1.741% ( 1, 2 ] 1152628 47.986% 49.727% ########## ( 2, 3 ] 599471 24.957% 74.684% ##### ( 3, 4 ] 233195 9.708% 84.392% ## ( 4, 6 ] 282366 11.755% 96.148% ## ( 6, 10 ] 59455 2.475% 98.623% ( 10, 15 ] 17529 0.730% 99.352% ( 15, 22 ] 1805 0.075% 99.428% ( 22, 34 ] 156 0.006% 99.434% ( 34, 51 ] 164 0.007% 99.441% ( 51, 76 ] 253 0.011% 99.451% ( 76, 110 ] 328 0.014% 99.465% ( 110, 170 ] 3026 0.126% 99.591% ( 170, 250 ] 2074 0.086% 99.677% ( 250, 380 ] 1057 0.044% 99.721% ( 380, 580 ] 1371 0.057% 99.779% ( 580, 870 ] 1860 0.077% 99.856% ( 870, 1300 ] 2055 0.086% 99.942% ( 1300, 1900 ] 1699 0.071% 100.012% ( 1900, 2900 ] 1812 0.075% 100.088% ( 2900, 4400 ] 1545 0.064% 100.152% ( 4400, 6600 ] 473 0.020% 100.172% ( 6600, 9900 ] 78 0.003% 100.175% ( 9900, 14000 ] 11 0.000% 100.175% ( 14000, 22000 ] 15 0.001% 100.176% ( 22000, 33000 ] 11 0.000% 100.176% ** Level 4 read latency histogram (micros): Count: 1242247 Average: 30.3132 StdDev: 276.81 Min: 0 Median: 2.1027 Max: 48333 Percentiles: P50: 2.10 P75: 3.45 P99: 506.90 P99.9: 2706.91 P99.99: 3668.01 ------------------------------------------------------ [ 0, 1 ] 23126 1.862% 1.862% ( 1, 2 ] 568447 45.760% 47.621% ######### ( 2, 3 ] 287707 23.160% 70.781% ##### ( 3, 4 ] 116718 9.396% 80.177% ## ( 4, 6 ] 135300 10.892% 91.069% ## ( 6, 10 ] 31016 2.497% 93.565% ( 10, 15 ] 9457 0.761% 94.327% ( 15, 22 ] 985 0.079% 94.406% ( 22, 34 ] 86 0.007% 94.413% ( 34, 51 ] 104 0.008% 94.421% ( 51, 76 ] 179 0.014% 94.436% ( 76, 110 ] 3024 0.243% 94.679% ( 110, 170 ] 29303 2.359% 97.038% ( 170, 250 ] 16987 1.367% 98.405% ( 250, 380 ] 4694 0.378% 98.783% ( 380, 580 ] 4242 0.341% 99.125% ( 580, 870 ] 3502 0.282% 99.407% ( 870, 1300 ] 2848 0.229% 99.636% ( 1300, 1900 ] 1954 0.157% 99.793% ( 1900, 2900 ] 1643 0.132% 99.926% ( 2900, 4400 ] 1564 0.126% 100.051% ( 4400, 6600 ] 529 0.043% 100.094% ( 6600, 9900 ] 84 0.007% 100.101% ( 9900, 14000 ] 24 0.002% 100.103% ( 14000, 22000 ] 21 0.002% 100.104% ( 22000, 33000 ] 24 0.002% 100.106% ( 33000, 50000 ] 5 0.000% 100.107% ** Level 5 read latency histogram (micros): Count: 347892 Average: 340.8792 StdDev: 722.80 Min: 80 Median: 168.9230 Max: 51015 Percentiles: P50: 168.92 P75: 242.43 P99: 3836.80 P99.9: 6598.91 P99.99: 18805.08 ------------------------------------------------------ ( 76, 110 ] 16493 4.741% 4.741% # ( 110, 170 ] 160331 46.086% 50.827% ######### ( 170, 250 ] 92878 26.697% 77.525% ##### ( 250, 380 ] 24206 6.958% 84.483% # ( 380, 580 ] 19317 5.553% 90.035% # ( 580, 870 ] 12371 3.556% 93.591% # ( 870, 1300 ] 8147 2.342% 95.933% ( 1300, 1900 ] 4693 1.349% 97.282% ( 1900, 2900 ] 3750 1.078% 98.360% ( 2900, 4400 ] 3566 1.025% 99.385% ( 4400, 6600 ] 1793 0.515% 99.900% ( 6600, 9900 ] 283 0.081% 99.982% ( 9900, 14000 ] 19 0.005% 99.987% ( 14000, 22000 ] 17 0.005% 99.992% ( 22000, 33000 ] 14 0.004% 99.996% ( 33000, 50000 ] 17 0.005% 100.001% ( 50000, 75000 ] 1 0.000% 100.001% ** DB Stats ** Uptime(secs): 44.2 total, 15.2 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 133.64 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:21.153 H:M:S, 47.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 64.80 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:11.329 H:M:S, 74.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 318.309 micros/op 12560 ops/sec 60.108 seconds 754996 operations; 12.2 MB/s (187999 of 187999 found) Microseconds per read: Count: 754996 Average: 318.3116 StdDev: 1046.77 Min: 2 Median: 203.9246 Max: 89777 Percentiles: P50: 203.92 P75: 248.19 P99: 2393.48 P99.9: 18544.22 P99.99: 38361.64 ------------------------------------------------------ ( 1, 2 ] 1 0.000% 0.000% ( 2, 3 ] 15 0.002% 0.002% ( 3, 4 ] 38 0.005% 0.007% ( 4, 6 ] 79 0.010% 0.018% ( 6, 10 ] 209 0.028% 0.045% ( 10, 15 ] 2166 0.287% 0.332% ( 15, 22 ] 3740 0.495% 0.828% ( 22, 34 ] 6464 0.856% 1.684% ( 34, 51 ] 1821 0.241% 1.925% ( 51, 76 ] 259 0.034% 1.959% ( 76, 110 ] 1845 0.244% 2.204% ( 110, 170 ] 216190 28.635% 30.838% ###### ( 170, 250 ] 341159 45.187% 76.025% ######### ( 250, 380 ] 87835 11.634% 87.659% ## ( 380, 580 ] 46231 6.123% 93.782% # ( 580, 870 ] 21198 2.808% 96.590% # ( 870, 1300 ] 11111 1.472% 98.062% ( 1300, 1900 ] 5346 0.708% 98.770% ( 1900, 2900 ] 3524 0.467% 99.236% ( 2900, 4400 ] 2981 0.395% 99.631% ( 4400, 6600 ] 1235 0.164% 99.795% ( 6600, 9900 ] 217 0.029% 99.824% ( 9900, 14000 ] 172 0.023% 99.846% ( 14000, 22000 ] 713 0.094% 99.941% ( 22000, 33000 ] 351 0.046% 99.987% ( 33000, 50000 ] 65 0.009% 99.996% ( 50000, 75000 ] 29 0.004% 100.000% ( 75000, 110000 ] 2 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 11381607 rocksdb.bloom.filter.full.positive COUNT : 866184 rocksdb.bloom.filter.full.true.positive COUNT : 754622 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 374 rocksdb.memtable.miss COUNT : 754622 rocksdb.l0.hit COUNT : 26627 rocksdb.l1.hit COUNT : 6207 rocksdb.l2andup.hit COUNT : 721788 rocksdb.compaction.key.drop.new COUNT : 277190 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 6085799 rocksdb.number.keys.read COUNT : 754996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 6280544568 rocksdb.bytes.read COUNT : 754996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1758 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 36469584 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 29509412361 rocksdb.compact.write.bytes COUNT : 29218626701 rocksdb.flush.write.bytes COUNT : 6275283680 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 24 rocksdb.number.superversion_acquires COUNT : 1261 rocksdb.number.superversion_releases COUNT : 1234 rocksdb.number.superversion_cleanups COUNT : 321 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 383 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 6224788480 rocksdb.memtable.garbage.bytes.at.flush COUNT : 2511872 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 32773077821 rocksdb.non.last.level.read.count COUNT : 7775922 rocksdb.block.checksum.compute.count COUNT : 8022056 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 203.099866 P95 : 704.612013 P99 : 2392.349603 P100 : 89775.000000 COUNT : 754996 SUM : 239701680 rocksdb.db.write.micros P50 : 1.559031 P95 : 4.057947 P99 : 8.573355 P100 : 946171.000000 COUNT : 6085799 SUM : 51779755 rocksdb.compaction.times.micros P50 : 1394444.444444 P95 : 4076000.000000 P99 : 9272000.000000 P100 : 23427175.000000 COUNT : 116 SUM : 215119148 rocksdb.compaction.times.cpu_micros P50 : 870000.000000 P95 : 2450000.000000 P99 : 6324000.000000 P100 : 9553420.000000 COUNT : 116 SUM : 127662549 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 945.250000 P95 : 30470.000000 P99 : 68083.333333 P100 : 71035.000000 COUNT : 83 SUM : 560000 rocksdb.compaction.outfile.sync.micros P50 : 17555.555556 P95 : 130117.647059 P99 : 223440.000000 P100 : 692518.000000 COUNT : 366 SUM : 13030361 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 692.777778 P95 : 4322.000000 P99 : 6528.500000 P100 : 6930.000000 COUNT : 226 SUM : 291809 rocksdb.table.open.io.micros P50 : 11634.930140 P95 : 21550.375940 P99 : 30084.000000 P100 : 40516.000000 COUNT : 1758 SUM : 19337371 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.964565 P95 : 9.902974 P99 : 205.754710 P100 : 57838.000000 COUNT : 7145431 SUM : 114380482 rocksdb.read.block.get.micros P50 : 159.174297 P95 : 567.143012 P99 : 2360.825147 P100 : 58950.000000 COUNT : 874852 SUM : 208255831 rocksdb.write.raw.block.micros P50 : 0.527633 P95 : 1.069082 P99 : 3.323541 P100 : 95981.000000 COUNT : 8597041 SUM : 29066149 rocksdb.numfiles.in.singlecompaction P50 : 2.609756 P95 : 6.850000 P99 : 9.770000 P100 : 25.000000 COUNT : 146 SUM : 501 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.568939 P95 : 1198.204322 P99 : 2546.396851 P100 : 946001.000000 COUNT : 177575 SUM : 36483508 rocksdb.sst.read.micros P50 : 1.993215 P95 : 176.403577 P99 : 519.091010 P100 : 58942.000000 COUNT : 7775936 SUM : 303297776 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 754996 SUM : 754996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 6085799 SUM : 6280544568 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 231481.481481 P95 : 998428.571429 P99 : 1159685.714286 P100 : 1183472.000000 COUNT : 83 SUM : 29750667 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 384 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0