[2024-11-20 13:37:16.961510] Starting SPDK v25.01-pre git sha1 557f022f6 / DPDK 24.11.0-rc3 initialization... [2024-11-20 13:37:16.961657] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid607840 ] Set seed to 1732106236958004 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-20 13:37:17.124951] pci_dpdk.c: 37:dpdk_pci_init: *NOTICE*: In-development DPDK 24.11.0-rc3 is used. There is no support for it in SPDK. Enabled only for validation. [2024-11-20 13:37:17.170105] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-20 13:37:17.193553] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-20 13:37:17.193590] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 607840' to capture a snapshot of events at runtime. [2024-11-20 13:37:17.193602] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-20 13:37:17.193611] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-20 13:37:17.193621] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid607840 for offline analysis/debug. [2024-11-20 13:37:17.194171] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Wed Nov 20 13:37:21 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/20-13:37:25 ... thread 0: (1000000,1000000) ops and (248286.7,248286.7) ops/second in (4.027602,4.027602) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/14 3.16 GB 62.9 0.6 0.0 0.6 1.4 0.9 0.0 1.6 101.2 258.4 5.60 4.56 15 0.374 575K 571 0.0 0.0 L1 1/1 82.29 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 14/0 934.01 MB 0.9 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 160/0 9.72 GB 1.0 1.4 0.7 0.7 1.4 0.7 0.2 2.0 140.0 139.0 9.98 6.65 6 1.664 261K 0 0.0 0.0 L5 1120/0 92.15 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1312/15 106.02 GB 0.0 1.9 0.7 1.2 2.8 1.5 2.2 3.2 126.1 182.0 15.59 11.21 21 0.742 837K 571 0.0 0.0 Int 0/0 0.00 KB 0.0 1.9 0.7 1.2 2.8 1.5 2.2 3.2 126.1 182.0 15.59 11.21 21 0.742 837K 571 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.9 0.7 1.2 1.9 0.7 0.0 0.0 161.2 160.4 12.19 8.80 8 1.523 837K 571 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 259.4 3.40 2.41 13 0.262 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 4.2 total, 4.0 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 2.77 GB write, 674.48 MB/s write, 1.92 GB read, 467.29 MB/s read, 15.6 seconds Interval compaction: 2.77 GB write, 703.95 MB/s write, 1.92 GB read, 487.71 MB/s read, 15.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 6, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 6, interval: 6 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 311273 Average: 9.3510 StdDev: 94.82 Min: 0 Median: 1.6151 Max: 9818 Percentiles: P50: 1.62 P75: 2.13 P99: 167.50 P99.9: 866.89 P99.99: 1479.87 ------------------------------------------------------ [ 0, 1 ] 41432 13.311% 13.311% ### ( 1, 2 ] 185654 59.643% 72.954% ############ ( 2, 3 ] 48917 15.715% 88.669% ### ( 3, 4 ] 13601 4.369% 93.039% # ( 4, 6 ] 11237 3.610% 96.649% # ( 6, 10 ] 4020 1.291% 97.940% ( 10, 15 ] 698 0.224% 98.164% ( 15, 22 ] 106 0.034% 98.198% ( 22, 34 ] 27 0.009% 98.207% ( 34, 51 ] 3 0.001% 98.208% ( 51, 76 ] 2 0.001% 98.209% ( 76, 110 ] 581 0.187% 98.395% ( 110, 170 ] 1964 0.631% 99.026% ( 170, 250 ] 1111 0.357% 99.383% ( 250, 380 ] 624 0.200% 99.584% ( 380, 580 ] 591 0.190% 99.774% ( 580, 870 ] 398 0.128% 99.901% ( 870, 1300 ] 239 0.077% 99.978% ( 1300, 1900 ] 123 0.040% 100.018% ( 1900, 2900 ] 75 0.024% 100.042% ( 2900, 4400 ] 50 0.016% 100.058% ( 4400, 6600 ] 8 0.003% 100.060% ( 6600, 9900 ] 7 0.002% 100.063% ** Level 1 read latency histogram (micros): Count: 18 Average: 895.8889 StdDev: 1935.97 Min: 108 Median: 282.5000 Max: 8577 Percentiles: P50: 282.50 P75: 628.33 P99: 8577.00 P99.9: 8577.00 P99.99: 8577.00 ------------------------------------------------------ ( 76, 110 ] 1 5.556% 5.556% # ( 110, 170 ] 6 33.333% 38.889% ####### ( 170, 250 ] 1 5.556% 44.444% # ( 250, 380 ] 4 22.222% 66.667% #### ( 380, 580 ] 1 5.556% 72.222% # ( 580, 870 ] 3 16.667% 88.889% ### ( 1900, 2900 ] 1 5.556% 94.444% # ( 6600, 9900 ] 1 5.556% 100.000% # ** Level 2 read latency histogram (micros): Count: 34 Average: 764.4706 StdDev: 1396.21 Min: 94 Median: 170.0000 Max: 6383 Percentiles: P50: 170.00 P75: 555.00 P99: 6226.00 P99.9: 6383.00 P99.99: 6383.00 ------------------------------------------------------ ( 76, 110 ] 2 5.882% 5.882% # ( 110, 170 ] 15 44.118% 50.000% ######### ( 170, 250 ] 1 2.941% 52.941% # ( 250, 380 ] 4 11.765% 64.706% ## ( 380, 580 ] 4 11.765% 76.471% ## ( 580, 870 ] 2 5.882% 82.353% # ( 870, 1300 ] 2 5.882% 88.235% # ( 1300, 1900 ] 1 2.941% 91.176% # ( 2900, 4400 ] 1 2.941% 94.118% # ( 4400, 6600 ] 2 5.882% 100.000% # ** Level 3 read latency histogram (micros): Count: 12038 Average: 20.0369 StdDev: 183.00 Min: 1 Median: 1.9156 Max: 10110 Percentiles: P50: 1.92 P75: 2.96 P99: 489.42 P99.9: 2396.20 P99.99: 9227.46 ------------------------------------------------------ [ 0, 1 ] 562 4.669% 4.669% # ( 1, 2 ] 5960 49.510% 54.178% ########## ( 2, 3 ] 2624 21.798% 75.976% #### ( 3, 4 ] 1065 8.847% 84.823% ## ( 4, 6 ] 1070 8.889% 93.712% ## ( 6, 10 ] 326 2.708% 96.420% # ( 10, 15 ] 61 0.507% 96.926% ( 15, 22 ] 13 0.108% 97.034% ( 22, 34 ] 3 0.025% 97.059% ( 34, 51 ] 2 0.017% 97.076% ( 76, 110 ] 25 0.208% 97.284% ( 110, 170 ] 62 0.515% 97.799% ( 170, 250 ] 58 0.482% 98.280% ( 250, 380 ] 62 0.515% 98.795% ( 380, 580 ] 45 0.374% 99.169% ( 580, 870 ] 35 0.291% 99.460% ( 870, 1300 ] 31 0.258% 99.718% ( 1300, 1900 ] 17 0.141% 99.859% ( 1900, 2900 ] 10 0.083% 99.942% ( 2900, 4400 ] 5 0.042% 99.983% ( 6600, 9900 ] 1 0.008% 99.992% ( 9900, 14000 ] 1 0.008% 100.000% ** Level 4 read latency histogram (micros): Count: 129177 Average: 22.0265 StdDev: 175.89 Min: 1 Median: 1.9875 Max: 6899 Percentiles: P50: 1.99 P75: 3.00 P99: 530.23 P99.9: 1714.81 P99.99: 2133.88 ------------------------------------------------------ [ 0, 1 ] 4323 3.347% 3.347% # ( 1, 2 ] 61028 47.244% 50.590% ######### ( 2, 3 ] 31498 24.384% 74.974% ##### ( 3, 4 ] 12290 9.514% 84.488% ## ( 4, 6 ] 12126 9.387% 93.875% ## ( 6, 10 ] 3824 2.960% 96.835% # ( 10, 15 ] 726 0.562% 97.397% ( 15, 22 ] 118 0.091% 97.489% ( 22, 34 ] 25 0.019% 97.508% ( 34, 51 ] 13 0.010% 97.518% ( 51, 76 ] 5 0.004% 97.522% ( 76, 110 ] 172 0.133% 97.655% ( 110, 170 ] 606 0.469% 98.124% ( 170, 250 ] 437 0.338% 98.463% ( 250, 380 ] 378 0.293% 98.755% ( 380, 580 ] 421 0.326% 99.081% ( 580, 870 ] 481 0.372% 99.453% ( 870, 1300 ] 395 0.306% 99.759% ( 1300, 1900 ] 263 0.204% 99.963% ( 1900, 2900 ] 150 0.116% 100.079% ( 2900, 4400 ] 95 0.074% 100.153% ( 4400, 6600 ] 22 0.017% 100.170% ( 6600, 9900 ] 1 0.001% 100.170% ** Level 5 read latency histogram (micros): Count: 18772 Average: 594.7865 StdDev: 910.80 Min: 75 Median: 230.2683 Max: 8449 Percentiles: P50: 230.27 P75: 562.86 P99: 4477.18 P99.9: 6553.64 P99.99: 8449.00 ------------------------------------------------------ ( 51, 76 ] 4 0.021% 0.021% ( 76, 110 ] 1556 8.289% 8.310% ## ( 110, 170 ] 5355 28.527% 36.837% ###### ( 170, 250 ] 3280 17.473% 54.310% ### ( 250, 380 ] 2156 11.485% 65.795% ## ( 380, 580 ] 1890 10.068% 75.863% ## ( 580, 870 ] 1377 7.335% 83.198% # ( 870, 1300 ] 888 4.730% 87.929% # ( 1300, 1900 ] 698 3.718% 91.647% # ( 1900, 2900 ] 746 3.974% 95.621% # ( 2900, 4400 ] 628 3.345% 98.967% # ( 4400, 6600 ] 179 0.954% 99.920% ( 6600, 9900 ] 15 0.080% 100.000% ** DB Stats ** Uptime(secs): 4.2 total, 4.0 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 234.03 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.437 H:M:S, 10.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 244.19 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.437 H:M:S, 10.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/20-13:37:31 ... thread 0: (1000000,2000000) ops and (172130.5,203311.0) ops/second in (5.809545,9.837147) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/17 4.14 GB 31.5 1.8 0.0 1.8 3.6 1.8 0.0 2.0 136.9 278.5 13.34 11.81 36 0.371 1855K 1805 0.0 0.0 L1 1/1 82.29 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 14/0 934.01 MB 0.9 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 160/0 9.72 GB 1.0 1.4 0.7 0.7 1.4 0.7 0.2 2.0 140.0 139.0 9.98 6.65 6 1.664 261K 0 0.0 0.0 L5 1120/0 92.15 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1313/18 107.01 GB 0.0 3.1 0.7 2.5 5.0 2.5 2.2 2.7 138.3 218.8 23.32 18.46 42 0.555 2116K 1805 0.0 0.0 Int 0/0 0.00 KB 0.0 3.1 0.7 2.5 5.0 2.5 2.2 2.7 138.3 218.8 23.32 18.46 42 0.555 2116K 1805 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.1 0.7 2.5 3.1 0.7 0.0 0.0 191.0 190.3 16.89 13.42 13 1.299 2116K 1805 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 293.5 6.44 5.03 29 0.222 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 10.0 total, 9.8 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 4.98 GB write, 509.55 MB/s write, 3.15 GB read, 322.04 MB/s read, 23.3 seconds Interval compaction: 4.98 GB write, 518.67 MB/s write, 3.15 GB read, 327.80 MB/s read, 23.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 9, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 9, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 6, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 9, total-stops: 6, interval: 15 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 681611 Average: 15.7541 StdDev: 82.24 Min: 0 Median: 1.5222 Max: 9818 Percentiles: P50: 1.52 P75: 1.98 P99: 243.51 P99.9: 692.19 P99.99: 1003.21 ------------------------------------------------------ [ 0, 1 ] 146007 21.421% 21.421% #### ( 1, 2 ] 373010 54.725% 76.146% ########### ( 2, 3 ] 71016 10.419% 86.564% ## ( 3, 4 ] 22534 3.306% 89.870% # ( 4, 6 ] 16025 2.351% 92.222% ( 6, 10 ] 5132 0.753% 92.974% ( 10, 15 ] 1017 0.149% 93.124% ( 15, 22 ] 151 0.022% 93.146% ( 22, 34 ] 34 0.005% 93.151% ( 34, 51 ] 7 0.001% 93.152% ( 51, 76 ] 9 0.001% 93.153% ( 76, 110 ] 7509 1.102% 94.255% ( 110, 170 ] 24733 3.629% 97.883% # ( 170, 250 ] 8284 1.215% 99.099% ( 250, 380 ] 2836 0.416% 99.515% ( 380, 580 ] 2296 0.337% 99.852% ( 580, 870 ] 854 0.125% 99.977% ( 870, 1300 ] 290 0.043% 100.020% ( 1300, 1900 ] 133 0.020% 100.039% ( 1900, 2900 ] 98 0.014% 100.053% ( 2900, 4400 ] 63 0.009% 100.063% ( 4400, 6600 ] 8 0.001% 100.064% ( 6600, 9900 ] 7 0.001% 100.065% ** Level 1 read latency histogram (micros): Count: 103 Average: 315.2233 StdDev: 863.62 Min: 87 Median: 151.6667 Max: 8577 Percentiles: P50: 151.67 P75: 226.92 P99: 2870.00 P99.9: 8577.00 P99.99: 8577.00 ------------------------------------------------------ ( 76, 110 ] 14 13.592% 13.592% ### ( 110, 170 ] 54 52.427% 66.019% ########## ( 170, 250 ] 13 12.621% 78.641% ### ( 250, 380 ] 10 9.709% 88.350% ## ( 380, 580 ] 6 5.825% 94.175% # ( 580, 870 ] 3 2.913% 97.087% # ( 870, 1300 ] 1 0.971% 98.058% ( 1900, 2900 ] 1 0.971% 99.029% ( 6600, 9900 ] 1 0.971% 100.000% ** Level 2 read latency histogram (micros): Count: 205 Average: 261.6341 StdDev: 616.41 Min: 80 Median: 145.5462 Max: 6383 Percentiles: P50: 145.55 P75: 179.17 P99: 4325.00 P99.9: 6374.50 P99.99: 6383.00 ------------------------------------------------------ ( 76, 110 ] 32 15.610% 15.610% ### ( 110, 170 ] 119 58.049% 73.659% ############ ( 170, 250 ] 24 11.707% 85.366% ## ( 250, 380 ] 10 4.878% 90.244% # ( 380, 580 ] 11 5.366% 95.610% # ( 580, 870 ] 3 1.463% 97.073% ( 870, 1300 ] 2 0.976% 98.049% ( 1300, 1900 ] 1 0.488% 98.537% ( 2900, 4400 ] 1 0.488% 99.024% ( 4400, 6600 ] 2 0.976% 100.000% ** Level 3 read latency histogram (micros): Count: 13264 Average: 33.9746 StdDev: 182.61 Min: 1 Median: 2.0419 Max: 10110 Percentiles: P50: 2.04 P75: 3.75 P99: 539.03 P99.9: 2273.60 P99.99: 8822.88 ------------------------------------------------------ [ 0, 1 ] 562 4.237% 4.237% # ( 1, 2 ] 5960 44.934% 49.171% ######### ( 2, 3 ] 2624 19.783% 68.954% #### ( 3, 4 ] 1065 8.029% 76.983% ## ( 4, 6 ] 1070 8.067% 85.050% ## ( 6, 10 ] 326 2.458% 87.508% ( 10, 15 ] 61 0.460% 87.967% ( 15, 22 ] 13 0.098% 88.065% ( 22, 34 ] 3 0.023% 88.088% ( 34, 51 ] 2 0.015% 88.103% ( 76, 110 ] 238 1.794% 89.897% ( 110, 170 ] 748 5.639% 95.537% # ( 170, 250 ] 259 1.953% 97.489% ( 250, 380 ] 128 0.965% 98.454% ( 380, 580 ] 91 0.686% 99.141% ( 580, 870 ] 45 0.339% 99.480% ( 870, 1300 ] 34 0.256% 99.736% ( 1300, 1900 ] 18 0.136% 99.872% ( 1900, 2900 ] 10 0.075% 99.947% ( 2900, 4400 ] 5 0.038% 99.985% ( 6600, 9900 ] 1 0.008% 99.992% ( 9900, 14000 ] 1 0.008% 100.000% ** Level 4 read latency histogram (micros): Count: 140099 Average: 32.6858 StdDev: 176.22 Min: 1 Median: 2.1445 Max: 6899 Percentiles: P50: 2.14 P75: 3.63 P99: 556.88 P99.9: 1708.87 P99.99: 2167.45 ------------------------------------------------------ [ 0, 1 ] 4324 3.086% 3.086% # ( 1, 2 ] 61134 43.636% 46.723% ######### ( 2, 3 ] 31784 22.687% 69.409% ##### ( 3, 4 ] 12526 8.941% 78.350% ## ( 4, 6 ] 12374 8.832% 87.183% ## ( 6, 10 ] 3848 2.747% 89.929% # ( 10, 15 ] 736 0.525% 90.455% ( 15, 22 ] 120 0.086% 90.540% ( 22, 34 ] 25 0.018% 90.558% ( 34, 51 ] 13 0.009% 90.567% ( 51, 76 ] 6 0.004% 90.572% ( 76, 110 ] 1831 1.307% 91.879% ( 110, 170 ] 6216 4.437% 96.315% # ( 170, 250 ] 2104 1.502% 97.817% ( 250, 380 ] 915 0.653% 98.470% ( 380, 580 ] 839 0.599% 99.069% ( 580, 870 ] 576 0.411% 99.480% ( 870, 1300 ] 408 0.291% 99.772% ( 1300, 1900 ] 264 0.188% 99.960% ( 1900, 2900 ] 157 0.112% 100.072% ( 2900, 4400 ] 96 0.069% 100.141% ( 4400, 6600 ] 22 0.016% 100.156% ( 6600, 9900 ] 1 0.001% 100.157% ** Level 5 read latency histogram (micros): Count: 103804 Average: 249.3625 StdDev: 433.39 Min: 73 Median: 151.1299 Max: 8449 Percentiles: P50: 151.13 P75: 212.48 P99: 2623.73 P99.9: 5410.23 P99.99: 6558.45 ------------------------------------------------------ ( 51, 76 ] 19 0.018% 0.018% ( 76, 110 ] 15943 15.359% 15.377% ### ( 110, 170 ] 52429 50.508% 65.885% ########## ( 170, 250 ] 17819 17.166% 83.051% ### ( 250, 380 ] 6589 6.348% 89.398% # ( 380, 580 ] 5376 5.179% 94.577% # ( 580, 870 ] 2291 2.207% 96.784% ( 870, 1300 ] 1002 0.965% 97.750% ( 1300, 1900 ] 732 0.705% 98.455% ( 1900, 2900 ] 782 0.753% 99.208% ( 2900, 4400 ] 636 0.613% 99.821% ( 4400, 6600 ] 179 0.172% 99.993% ( 6600, 9900 ] 15 0.014% 100.008% ** DB Stats ** Uptime(secs): 10.0 total, 5.8 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 196.54 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.754 H:M:S, 27.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 169.41 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.317 H:M:S, 39.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/20-13:37:40 ... thread 0: (1000000,3000000) ops and (107425.1,156691.1) ops/second in (9.308809,19.145956) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/10 5.05 GB 282.2 4.1 0.0 4.1 6.9 2.8 0.0 2.4 165.0 279.5 25.19 23.10 56 0.450 4220K 13K 0.0 0.0 L1 1/1 82.29 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 14/0 934.01 MB 0.9 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 160/0 9.72 GB 1.0 1.4 0.7 0.7 1.4 0.7 0.2 2.0 140.0 139.0 9.98 6.65 6 1.664 261K 0 0.0 0.0 L5 1120/0 92.15 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1313/11 107.92 GB 0.0 5.4 0.7 4.8 8.2 3.5 2.2 2.9 157.9 239.6 35.18 29.75 62 0.567 4482K 13K 0.0 0.0 Int 0/0 0.00 KB 0.0 5.4 0.7 4.8 8.2 3.5 2.2 2.9 157.9 239.6 35.18 29.75 62 0.567 4482K 13K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.4 0.7 4.8 5.4 0.7 0.0 0.0 216.8 215.8 25.63 22.02 17 1.507 4482K 13K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 303.5 9.55 7.73 45 0.212 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 19.3 total, 19.1 interval Flush(GB): cumulative 2.830, interval 2.830 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.23 GB write, 436.20 MB/s write, 5.42 GB read, 287.45 MB/s read, 35.2 seconds Interval compaction: 8.23 GB write, 440.21 MB/s write, 5.42 GB read, 290.09 MB/s read, 35.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 15, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 15, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 6, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 15, total-stops: 6, interval: 21 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1346532 Average: 17.5155 StdDev: 73.64 Min: 0 Median: 1.4673 Max: 9818 Percentiles: P50: 1.47 P75: 1.94 P99: 246.36 P99.9: 578.14 P99.99: 808.23 ------------------------------------------------------ [ 0, 1 ] 343217 25.489% 25.489% ##### ( 1, 2 ] 706307 52.454% 77.943% ########## ( 2, 3 ] 110799 8.228% 86.171% ## ( 3, 4 ] 38879 2.887% 89.059% # ( 4, 6 ] 23745 1.763% 90.822% ( 6, 10 ] 6857 0.509% 91.331% ( 10, 15 ] 1555 0.115% 91.447% ( 15, 22 ] 244 0.018% 91.465% ( 22, 34 ] 50 0.004% 91.469% ( 34, 51 ] 11 0.001% 91.469% ( 51, 76 ] 26 0.002% 91.471% ( 76, 110 ] 18474 1.372% 92.843% ( 110, 170 ] 63573 4.721% 97.564% # ( 170, 250 ] 20252 1.504% 99.068% ( 250, 380 ] 6343 0.471% 99.540% ( 380, 580 ] 4899 0.364% 99.903% ( 580, 870 ] 1482 0.110% 100.013% ( 870, 1300 ] 372 0.028% 100.041% ( 1300, 1900 ] 149 0.011% 100.052% ( 1900, 2900 ] 115 0.009% 100.061% ( 2900, 4400 ] 77 0.006% 100.066% ( 4400, 6600 ] 9 0.001% 100.067% ( 6600, 9900 ] 7 0.001% 100.068% ** Level 1 read latency histogram (micros): Count: 223 Average: 233.7803 StdDev: 596.07 Min: 82 Median: 145.5645 Max: 8577 Percentiles: P50: 145.56 P75: 185.00 P99: 1201.10 P99.9: 8577.00 P99.99: 8577.00 ------------------------------------------------------ ( 76, 110 ] 38 17.040% 17.040% ### ( 110, 170 ] 124 55.605% 72.646% ########### ( 170, 250 ] 28 12.556% 85.202% ### ( 250, 380 ] 15 6.726% 91.928% # ( 380, 580 ] 11 4.933% 96.861% # ( 580, 870 ] 4 1.794% 98.655% ( 870, 1300 ] 1 0.448% 99.103% ( 1900, 2900 ] 1 0.448% 99.552% ( 6600, 9900 ] 1 0.448% 100.000% ** Level 2 read latency histogram (micros): Count: 428 Average: 213.8551 StdDev: 446.30 Min: 80 Median: 143.1646 Max: 6383 Percentiles: P50: 143.16 P75: 171.36 P99: 1732.00 P99.9: 6129.20 P99.99: 6383.00 ------------------------------------------------------ ( 76, 110 ] 83 19.393% 19.393% #### ( 110, 170 ] 237 55.374% 74.766% ########### ( 170, 250 ] 59 13.785% 88.551% ### ( 250, 380 ] 19 4.439% 92.991% # ( 380, 580 ] 19 4.439% 97.430% # ( 580, 870 ] 4 0.935% 98.364% ( 870, 1300 ] 2 0.467% 98.832% ( 1300, 1900 ] 1 0.234% 99.065% ( 1900, 2900 ] 1 0.234% 99.299% ( 2900, 4400 ] 1 0.234% 99.533% ( 4400, 6600 ] 2 0.467% 100.000% ** Level 3 read latency histogram (micros): Count: 15184 Average: 50.7443 StdDev: 180.56 Min: 1 Median: 2.4078 Max: 10110 Percentiles: P50: 2.41 P75: 7.31 P99: 556.61 P99.9: 2156.00 P99.99: 8189.28 ------------------------------------------------------ [ 0, 1 ] 562 3.701% 3.701% # ( 1, 2 ] 5960 39.252% 42.953% ######## ( 2, 3 ] 2624 17.281% 60.234% ### ( 3, 4 ] 1065 7.014% 67.248% # ( 4, 6 ] 1070 7.047% 74.295% # ( 6, 10 ] 326 2.147% 76.442% ( 10, 15 ] 61 0.402% 76.844% ( 15, 22 ] 13 0.086% 76.930% ( 22, 34 ] 3 0.020% 76.949% ( 34, 51 ] 2 0.013% 76.963% ( 76, 110 ] 543 3.576% 80.539% # ( 110, 170 ] 1886 12.421% 92.960% ## ( 170, 250 ] 574 3.780% 96.740% # ( 250, 380 ] 216 1.423% 98.163% ( 380, 580 ] 144 0.948% 99.111% ( 580, 870 ] 61 0.402% 99.513% ( 870, 1300 ] 37 0.244% 99.756% ( 1300, 1900 ] 19 0.125% 99.881% ( 1900, 2900 ] 11 0.072% 99.954% ( 2900, 4400 ] 5 0.033% 99.987% ( 6600, 9900 ] 1 0.007% 99.993% ( 9900, 14000 ] 1 0.007% 100.000% ** Level 4 read latency histogram (micros): Count: 157032 Average: 47.2947 StdDev: 175.75 Min: 1 Median: 2.4096 Max: 6899 Percentiles: P50: 2.41 P75: 5.25 P99: 569.93 P99.9: 1685.80 P99.99: 2181.35 ------------------------------------------------------ [ 0, 1 ] 4324 2.754% 2.754% # ( 1, 2 ] 61146 38.939% 41.692% ######## ( 2, 3 ] 31848 20.281% 61.973% #### ( 3, 4 ] 12654 8.058% 70.032% ## ( 4, 6 ] 12501 7.961% 77.992% ## ( 6, 10 ] 3855 2.455% 80.447% ( 10, 15 ] 737 0.469% 80.917% ( 15, 22 ] 121 0.077% 80.994% ( 22, 34 ] 25 0.016% 81.010% ( 34, 51 ] 13 0.008% 81.018% ( 51, 76 ] 7 0.004% 81.022% ( 76, 110 ] 4597 2.927% 83.950% # ( 110, 170 ] 15625 9.950% 93.900% ## ( 170, 250 ] 4860 3.095% 96.995% # ( 250, 380 ] 1690 1.076% 98.071% ( 380, 580 ] 1536 0.978% 99.049% ( 580, 870 ] 730 0.465% 99.514% ( 870, 1300 ] 433 0.276% 99.790% ( 1300, 1900 ] 269 0.171% 99.961% ( 1900, 2900 ] 161 0.103% 100.064% ( 2900, 4400 ] 97 0.062% 100.125% ( 4400, 6600 ] 22 0.014% 100.139% ( 6600, 9900 ] 1 0.001% 100.140% ** Level 5 read latency histogram (micros): Count: 240230 Average: 203.7845 StdDev: 298.93 Min: 73 Median: 147.3922 Max: 8449 Percentiles: P50: 147.39 P75: 192.20 P99: 1306.83 P99.9: 4272.35 P99.99: 6378.49 ------------------------------------------------------ ( 51, 76 ] 27 0.011% 0.011% ( 76, 110 ] 39334 16.373% 16.385% ### ( 110, 170 ] 129579 53.940% 70.324% ########### ( 170, 250 ] 40474 16.848% 87.172% ### ( 250, 380 ] 13157 5.477% 92.649% # ( 380, 580 ] 10615 4.419% 97.068% # ( 580, 870 ] 3494 1.454% 98.522% ( 870, 1300 ] 1139 0.474% 98.996% ( 1300, 1900 ] 764 0.318% 99.314% ( 1900, 2900 ] 813 0.338% 99.653% ( 2900, 4400 ] 649 0.270% 99.923% ( 4400, 6600 ] 179 0.075% 99.998% ( 6600, 9900 ] 15 0.006% 100.004% ** DB Stats ** Uptime(secs): 19.3 total, 9.3 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 152.79 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:8.436 H:M:S, 43.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 105.72 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.682 H:M:S, 61.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 211.474 micros/op 18895 ops/sec 60.173 seconds 1136996 operations; 18.3 MB/s (284999 of 284999 found) Microseconds per read: Count: 1136996 Average: 211.4759 StdDev: 401.17 Min: 2 Median: 172.7935 Max: 59805 Percentiles: P50: 172.79 P75: 231.26 P99: 700.60 P99.9: 4928.02 P99.99: 19738.09 ------------------------------------------------------ ( 1, 2 ] 11 0.001% 0.001% ( 2, 3 ] 56 0.005% 0.006% ( 3, 4 ] 58 0.005% 0.011% ( 4, 6 ] 100 0.009% 0.020% ( 6, 10 ] 307 0.027% 0.047% ( 10, 15 ] 3525 0.310% 0.357% ( 15, 22 ] 3001 0.264% 0.621% ( 22, 34 ] 9820 0.864% 1.484% ( 34, 51 ] 1610 0.142% 1.626% ( 51, 76 ] 224 0.020% 1.646% ( 76, 110 ] 18864 1.659% 3.305% ( 110, 170 ] 517342 45.501% 48.806% ######### ( 170, 250 ] 388906 34.205% 83.010% ####### ( 250, 380 ] 132674 11.669% 94.679% ## ( 380, 580 ] 44229 3.890% 98.569% # ( 580, 870 ] 11780 1.036% 99.605% ( 870, 1300 ] 2242 0.197% 99.802% ( 1300, 1900 ] 358 0.031% 99.834% ( 1900, 2900 ] 360 0.032% 99.866% ( 2900, 4400 ] 296 0.026% 99.892% ( 4400, 6600 ] 400 0.035% 99.927% ( 6600, 9900 ] 318 0.028% 99.955% ( 9900, 14000 ] 260 0.023% 99.978% ( 14000, 22000 ] 197 0.017% 99.995% ( 22000, 33000 ] 53 0.005% 100.000% ( 33000, 50000 ] 4 0.000% 100.000% ( 50000, 75000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 19413172 rocksdb.bloom.filter.full.positive COUNT : 1325532 rocksdb.bloom.filter.full.true.positive COUNT : 1136533 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 463 rocksdb.memtable.miss COUNT : 1136533 rocksdb.l0.hit COUNT : 60749 rocksdb.l1.hit COUNT : 876 rocksdb.l2andup.hit COUNT : 1074908 rocksdb.compaction.key.drop.new COUNT : 49503 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3631695 rocksdb.number.keys.read COUNT : 1136996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3747909240 rocksdb.bytes.read COUNT : 1136996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1402 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 46128071 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 10217211936 rocksdb.compact.write.bytes COUNT : 10171050698 rocksdb.flush.write.bytes COUNT : 3699458983 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 48 rocksdb.number.superversion_acquires COUNT : 562 rocksdb.number.superversion_releases COUNT : 530 rocksdb.number.superversion_cleanups COUNT : 137 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3669388288 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1214464 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 15495892030 rocksdb.non.last.level.read.count COUNT : 3599062 rocksdb.block.checksum.compute.count COUNT : 3812660 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 170.947545 P95 : 394.234115 P99 : 699.047878 P100 : 59803.000000 COUNT : 1136996 SUM : 239381946 rocksdb.db.write.micros P50 : 1.570261 P95 : 5.197441 P99 : 20.070706 P100 : 410528.000000 COUNT : 3631695 SUM : 55550912 rocksdb.compaction.times.micros P50 : 1129166.666667 P95 : 5345000.000000 P99 : 5794776.000000 P100 : 5794776.000000 COUNT : 21 SUM : 33389261 rocksdb.compaction.times.cpu_micros P50 : 1061875.000000 P95 : 5345000.000000 P99 : 5722024.000000 P100 : 5722024.000000 COUNT : 21 SUM : 29629696 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 763.382353 P95 : 6875.000000 P99 : 9295.000000 P100 : 9380.000000 COUNT : 55 SUM : 65259 rocksdb.compaction.outfile.sync.micros P50 : 1700.000000 P95 : 12360.000000 P99 : 22928.000000 P100 : 22928.000000 COUNT : 56 SUM : 204752 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 141.200000 P95 : 1840.000000 P99 : 3162.000000 P100 : 3162.000000 COUNT : 84 SUM : 29933 rocksdb.table.open.io.micros P50 : 5994.273128 P95 : 13296.203209 P99 : 19048.421053 P100 : 19422.000000 COUNT : 1402 SUM : 9199397 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.625619 P95 : 150.177717 P99 : 236.473449 P100 : 15441.000000 COUNT : 2475428 SUM : 60335823 rocksdb.read.block.get.micros P50 : 139.707088 P95 : 248.498221 P99 : 551.176107 P100 : 22126.000000 COUNT : 1335824 SUM : 203679854 rocksdb.write.raw.block.micros P50 : 0.531178 P95 : 1.212730 P99 : 3.337197 P100 : 7382.000000 COUNT : 3359059 SUM : 7126904 rocksdb.numfiles.in.singlecompaction P50 : 3.522727 P95 : 7.800000 P99 : 11.000000 P100 : 11.000000 COUNT : 31 SUM : 131 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.577196 P95 : 1901.634681 P99 : 2700.462806 P100 : 410349.000000 COUNT : 235175 SUM : 46150764 rocksdb.sst.read.micros P50 : 2.688993 P95 : 205.707378 P99 : 423.296127 P100 : 10110.000000 COUNT : 3599133 SUM : 251322476 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1136996 SUM : 1136996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3631695 SUM : 3747909240 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 214000.000000 P95 : 323125.000000 P99 : 631409.000000 P100 : 631409.000000 COUNT : 55 SUM : 11635671 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 92 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0