[2024-11-19 09:36:10.627937] Starting SPDK v25.01-pre git sha1 d47eb51c9 / DPDK 22.11.4 initialization... [2024-11-19 09:36:10.628148] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2140001 ] Set seed to 1732005370624760 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-19 09:36:10.758483] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-19 09:36:10.805865] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-19 09:36:10.805964] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2140001' to capture a snapshot of events at runtime. [2024-11-19 09:36:10.805987] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-19 09:36:10.806008] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-19 09:36:10.806025] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2140001 for offline analysis/debug. [2024-11-19 09:36:10.806867] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Tue Nov 19 09:36:15 2024 CPU: 48 * Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz CPUCache: 30720 KB 2024/11/19-09:36:20 ... thread 0: (1000000,1000000) ops and (209031.2,209031.2) ops/second in (4.783974,4.783974) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/8 567.02 MB 6.3 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 258.0 3.42 3.10 14 0.244 0 0 0.0 0.0 L1 3/3 188.88 MB 0.0 0.2 0.2 0.0 0.2 0.2 0.1 1.0 163.0 162.9 1.16 1.13 1 1.160 191K 126 0.0 0.0 L2 2/1 125.96 MB 0.0 0.1 0.1 0.1 0.1 0.1 0.1 2.0 150.8 150.7 0.84 0.81 1 0.836 127K 41 0.0 0.0 L3 14/0 993.33 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 140/0 9.72 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1256/0 85.45 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1424/12 97.00 GB 0.0 0.3 0.2 0.1 1.2 1.1 0.3 1.4 58.2 221.1 5.41 5.05 16 0.338 319K 167 0.0 0.0 Int 0/0 0.00 KB 0.0 0.3 0.2 0.1 1.2 1.1 0.2 1.4 58.2 221.1 5.41 5.05 16 0.338 319K 167 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.3 0.2 0.1 0.3 0.2 0.0 0.0 157.9 157.8 2.00 1.94 2 0.998 319K 167 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 258.0 3.42 3.10 14 0.244 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 5.0 total, 4.8 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.17 GB write, 237.94 MB/s write, 0.31 GB read, 62.63 MB/s read, 5.4 seconds Interval compaction: 1.17 GB write, 250.12 MB/s write, 0.31 GB read, 65.83 MB/s read, 5.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 0, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 0, ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 143571 Average: 2.9364 StdDev: 16.51 Min: 1 Median: 2.0711 Max: 2666 Percentiles: P50: 2.07 P75: 2.63 P99: 8.74 P99.9: 20.97 P99.99: 192.42 ------------------------------------------------------ [ 0, 1 ] 22 0.015% 0.015% ( 1, 2 ] 67194 46.802% 46.817% ######### ( 2, 3 ] 64232 44.739% 91.556% ######### ( 3, 4 ] 5240 3.650% 95.206% # ( 4, 6 ] 4603 3.206% 98.412% # ( 6, 10 ] 1237 0.862% 99.274% ( 10, 15 ] 340 0.237% 99.510% ( 15, 22 ] 658 0.458% 99.969% ( 22, 34 ] 27 0.019% 99.987% ( 34, 51 ] 1 0.001% 99.988% ( 51, 76 ] 1 0.001% 99.989% ( 170, 250 ] 13 0.009% 99.998% ( 250, 380 ] 4 0.003% 100.001% ( 380, 580 ] 2 0.001% 100.002% ( 870, 1300 ] 2 0.001% 100.003% ( 1300, 1900 ] 1 0.001% 100.004% ( 1900, 2900 ] 6 0.004% 100.008% ** Level 1 read latency histogram (micros): Count: 29028 Average: 2.8554 StdDev: 5.86 Min: 1 Median: 2.1406 Max: 808 Percentiles: P50: 2.14 P75: 2.66 P99: 7.84 P99.9: 20.71 P99.99: 262.64 ------------------------------------------------------ [ 0, 1 ] 6 0.021% 0.021% ( 1, 2 ] 12536 43.186% 43.207% ######### ( 2, 3 ] 14021 48.302% 91.508% ########## ( 3, 4 ] 1155 3.979% 95.487% # ( 4, 6 ] 913 3.145% 98.632% # ( 6, 10 ] 232 0.799% 99.432% ( 10, 15 ] 43 0.148% 99.580% ( 15, 22 ] 114 0.393% 99.972% ( 22, 34 ] 2 0.007% 99.979% ( 34, 51 ] 1 0.003% 99.983% ( 51, 76 ] 1 0.003% 99.986% ( 170, 250 ] 1 0.003% 99.990% ( 250, 380 ] 1 0.003% 99.993% ( 380, 580 ] 1 0.003% 99.997% ( 580, 870 ] 1 0.003% 100.000% ** Level 2 read latency histogram (micros): Count: 21968 Average: 3.8322 StdDev: 44.73 Min: 1 Median: 2.5179 Max: 5586 Percentiles: P50: 2.52 P75: 2.87 P99: 9.67 P99.9: 20.51 P99.99: 1840.96 ------------------------------------------------------ [ 0, 1 ] 4 0.018% 0.018% ( 1, 2 ] 2982 13.574% 13.592% ### ( 2, 3 ] 15442 70.293% 83.886% ############## ( 3, 4 ] 1925 8.763% 92.648% ## ( 4, 6 ] 1025 4.666% 97.314% # ( 6, 10 ] 404 1.839% 99.153% ( 10, 15 ] 131 0.596% 99.750% ( 15, 22 ] 42 0.191% 99.941% ( 22, 34 ] 5 0.023% 99.964% ( 34, 51 ] 1 0.005% 99.968% ( 380, 580 ] 1 0.005% 99.973% ( 580, 870 ] 2 0.009% 99.982% ( 1300, 1900 ] 2 0.009% 99.991% ( 1900, 2900 ] 1 0.005% 99.995% ( 4400, 6600 ] 1 0.005% 100.000% ** Level 3 read latency histogram (micros): Count: 65 Average: 2489.0923 StdDev: 2040.53 Min: 385 Median: 1562.5000 Max: 7292 Percentiles: P50: 1562.50 P75: 4191.67 P99: 7292.00 P99.9: 7292.00 P99.99: 7292.00 ------------------------------------------------------ ( 380, 580 ] 7 10.769% 10.769% ## ( 580, 870 ] 13 20.000% 30.769% #### ( 870, 1300 ] 9 13.846% 44.615% ### ( 1300, 1900 ] 8 12.308% 56.923% ## ( 1900, 2900 ] 4 6.154% 63.077% # ( 2900, 4400 ] 9 13.846% 76.923% ### ( 4400, 6600 ] 12 18.462% 95.385% #### ( 6600, 9900 ] 3 4.615% 100.000% # ** Level 4 read latency histogram (micros): Count: 10 Average: 2677.3000 StdDev: 2410.79 Min: 578 Median: 1900.0000 Max: 7453 Percentiles: P50: 1900.00 P75: 3650.00 P99: 7453.00 P99.9: 7453.00 P99.99: 7453.00 ------------------------------------------------------ ( 380, 580 ] 1 10.000% 10.000% ## ( 580, 870 ] 2 20.000% 30.000% #### ( 870, 1300 ] 1 10.000% 40.000% ## ( 1300, 1900 ] 1 10.000% 50.000% ## ( 1900, 2900 ] 2 20.000% 70.000% #### ( 2900, 4400 ] 1 10.000% 80.000% ## ( 6600, 9900 ] 2 20.000% 100.000% #### ** DB Stats ** Uptime(secs): 5.0 total, 4.8 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 195.64 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 205.64 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/11/19-09:36:25 ... thread 0: (1000000,2000000) ops and (203306.2,206129.0) ops/second in (4.918689,9.702663) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/4 1.29 GB 107.0 1.0 0.0 1.0 2.8 1.8 0.0 1.5 76.4 219.5 13.20 12.35 34 0.388 1023K 927 0.0 0.0 L1 1/0 4.53 MB 0.5 0.6 0.4 0.2 0.6 0.4 0.1 1.4 150.3 150.1 4.19 4.08 2 2.095 639K 889 0.0 0.0 L2 2/0 50.54 MB 0.5 0.1 0.1 0.1 0.1 0.1 0.5 2.0 150.8 150.7 0.84 0.81 1 0.836 127K 41 0.0 0.0 L3 16/0 825.46 MB 0.8 0.1 0.1 0.0 0.1 0.1 0.5 1.0 134.9 134.9 0.92 0.86 1 0.924 126K 0 0.0 0.0 L4 192/0 9.58 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.6 1.0 83.9 83.9 1.33 0.93 1 1.334 113K 0 0.0 0.0 L5 1268/0 86.25 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.8 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1488/4 97.98 GB 0.0 2.0 0.7 1.2 3.8 2.6 2.4 2.1 97.7 189.9 20.48 19.04 39 0.525 2031K 1857 0.0 0.0 Int 0/0 0.00 KB 0.0 2.0 0.7 1.2 3.8 2.6 2.3 2.1 97.7 189.9 20.48 19.04 39 0.525 2031K 1857 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.0 0.7 1.2 2.0 0.7 0.0 0.0 149.9 149.7 13.35 12.65 9 1.483 2031K 1857 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 265.0 7.13 6.39 30 0.238 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.9 total, 9.7 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.80 GB write, 390.86 MB/s write, 1.95 GB read, 201.08 MB/s read, 20.5 seconds Interval compaction: 3.80 GB write, 400.74 MB/s write, 1.95 GB read, 206.16 MB/s read, 20.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 0, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 0, ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 419071 Average: 2.9947 StdDev: 14.26 Min: 1 Median: 2.2069 Max: 3255 Percentiles: P50: 2.21 P75: 2.70 P99: 8.93 P99.9: 20.97 P99.99: 44.96 ------------------------------------------------------ [ 0, 1 ] 44 0.010% 0.010% ( 1, 2 ] 165220 39.425% 39.436% ######## ( 2, 3 ] 214021 51.070% 90.506% ########## ( 3, 4 ] 18963 4.525% 95.031% # ( 4, 6 ] 13918 3.321% 98.352% # ( 6, 10 ] 3710 0.885% 99.238% ( 10, 15 ] 1061 0.253% 99.491% ( 15, 22 ] 2011 0.480% 99.971% ( 22, 34 ] 75 0.018% 99.989% ( 34, 51 ] 11 0.003% 99.991% ( 51, 76 ] 2 0.000% 99.992% ( 76, 110 ] 2 0.000% 99.992% ( 110, 170 ] 1 0.000% 99.992% ( 170, 250 ] 27 0.006% 99.999% ( 250, 380 ] 6 0.001% 100.000% ( 380, 580 ] 12 0.003% 100.003% ( 580, 870 ] 3 0.001% 100.004% ( 870, 1300 ] 8 0.002% 100.006% ( 1300, 1900 ] 2 0.000% 100.006% ( 1900, 2900 ] 8 0.002% 100.008% ( 2900, 4400 ] 2 0.000% 100.009% ** Level 1 read latency histogram (micros): Count: 47990 Average: 3.1646 StdDev: 17.65 Min: 1 Median: 2.3360 Max: 3250 Percentiles: P50: 2.34 P75: 2.77 P99: 8.34 P99.9: 20.72 P99.99: 609.14 ------------------------------------------------------ [ 0, 1 ] 10 0.021% 0.021% ( 1, 2 ] 14710 30.652% 30.673% ###### ( 2, 3 ] 27605 57.522% 88.195% ############ ( 3, 4 ] 3024 6.301% 94.497% # ( 4, 6 ] 1864 3.884% 98.381% # ( 6, 10 ] 507 1.056% 99.437% ( 10, 15 ] 93 0.194% 99.631% ( 15, 22 ] 158 0.329% 99.960% ( 22, 34 ] 4 0.008% 99.969% ( 34, 51 ] 1 0.002% 99.971% ( 51, 76 ] 1 0.002% 99.973% ( 76, 110 ] 1 0.002% 99.975% ( 170, 250 ] 2 0.004% 99.979% ( 250, 380 ] 1 0.002% 99.981% ( 380, 580 ] 4 0.008% 99.990% ( 580, 870 ] 2 0.004% 99.994% ( 870, 1300 ] 2 0.004% 99.998% ( 2900, 4400 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 31668 Average: 3.6677 StdDev: 37.26 Min: 1 Median: 2.5360 Max: 5586 Percentiles: P50: 2.54 P75: 2.89 P99: 9.46 P99.9: 20.08 P99.99: 1549.96 ------------------------------------------------------ [ 0, 1 ] 4 0.013% 0.013% ( 1, 2 ] 3763 11.883% 11.895% ## ( 2, 3 ] 22514 71.094% 82.989% ############## ( 3, 4 ] 3047 9.622% 92.611% ## ( 4, 6 ] 1567 4.948% 97.559% # ( 6, 10 ] 528 1.667% 99.226% ( 10, 15 ] 164 0.518% 99.744% ( 15, 22 ] 68 0.215% 99.959% ( 22, 34 ] 5 0.016% 99.975% ( 34, 51 ] 1 0.003% 99.978% ( 380, 580 ] 1 0.003% 99.981% ( 580, 870 ] 2 0.006% 99.987% ( 1300, 1900 ] 2 0.006% 99.994% ( 1900, 2900 ] 1 0.003% 99.997% ( 4400, 6600 ] 1 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 28514 Average: 9.0516 StdDev: 153.57 Min: 1 Median: 2.5681 Max: 7292 Percentiles: P50: 2.57 P75: 2.92 P99: 11.50 P99.9: 1861.45 P99.99: 6763.46 ------------------------------------------------------ [ 0, 1 ] 6 0.021% 0.021% ( 1, 2 ] 2830 9.925% 9.946% ## ( 2, 3 ] 20104 70.506% 80.452% ############## ( 3, 4 ] 3454 12.113% 92.565% ## ( 4, 6 ] 1239 4.345% 96.910% # ( 6, 10 ] 547 1.918% 98.829% ( 10, 15 ] 163 0.572% 99.400% ( 15, 22 ] 91 0.319% 99.719% ( 22, 34 ] 10 0.035% 99.755% ( 34, 51 ] 1 0.004% 99.758% ( 76, 110 ] 1 0.004% 99.762% ( 170, 250 ] 1 0.004% 99.765% ( 380, 580 ] 8 0.028% 99.793% ( 580, 870 ] 13 0.046% 99.839% ( 870, 1300 ] 10 0.035% 99.874% ( 1300, 1900 ] 8 0.028% 99.902% ( 1900, 2900 ] 4 0.014% 99.916% ( 2900, 4400 ] 9 0.032% 99.947% ( 4400, 6600 ] 12 0.042% 99.989% ( 6600, 9900 ] 3 0.011% 100.000% ** Level 4 read latency histogram (micros): Count: 290 Average: 104.2241 StdDev: 664.24 Min: 1 Median: 2.6111 Max: 7453 Percentiles: P50: 2.61 P75: 4.72 P99: 3050.00 P99.9: 7453.00 P99.99: 7453.00 ------------------------------------------------------ [ 0, 1 ] 42 14.483% 14.483% ### ( 1, 2 ] 81 27.931% 42.414% ###### ( 2, 3 ] 36 12.414% 54.828% ## ( 3, 4 ] 48 16.552% 71.379% ### ( 4, 6 ] 29 10.000% 81.379% ## ( 6, 10 ] 20 6.897% 88.276% # ( 10, 15 ] 9 3.103% 91.379% # ( 15, 22 ] 6 2.069% 93.448% ( 22, 34 ] 1 0.345% 93.793% ( 110, 170 ] 3 1.034% 94.828% ( 170, 250 ] 3 1.034% 95.862% ( 380, 580 ] 2 0.690% 96.552% ( 580, 870 ] 2 0.690% 97.241% ( 870, 1300 ] 2 0.690% 97.931% ( 1300, 1900 ] 1 0.345% 98.276% ( 1900, 2900 ] 2 0.690% 98.966% ( 2900, 4400 ] 1 0.345% 99.310% ( 6600, 9900 ] 2 0.690% 100.000% ** Level 5 read latency histogram (micros): Count: 60 Average: 526.3667 StdDev: 569.76 Min: 97 Median: 213.6364 Max: 2039 Percentiles: P50: 213.64 P75: 754.00 P99: 2039.00 P99.9: 2039.00 P99.99: 2039.00 ------------------------------------------------------ ( 76, 110 ] 4 6.667% 6.667% # ( 110, 170 ] 20 33.333% 40.000% ####### ( 170, 250 ] 11 18.333% 58.333% #### ( 250, 380 ] 2 3.333% 61.667% # ( 380, 580 ] 5 8.333% 70.000% ## ( 580, 870 ] 5 8.333% 78.333% ## ( 870, 1300 ] 2 3.333% 81.667% # ( 1300, 1900 ] 9 15.000% 96.667% ### ( 1900, 2900 ] 2 3.333% 100.000% # ** DB Stats ** Uptime(secs): 9.9 total, 4.9 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 197.84 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 200.10 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/19-09:36:30 ... thread 0: (1000000,3000000) ops and (207400.8,206551.2) ops/second in (4.821583,14.524246) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/4 2.27 GB 207.7 2.0 0.0 2.0 4.8 2.8 0.0 1.7 88.6 215.9 22.75 21.41 54 0.421 2046K 1961 0.0 0.0 L1 1/0 4.53 MB 0.5 0.6 0.4 0.2 0.6 0.4 0.1 1.4 150.3 150.1 4.19 4.08 2 2.095 639K 889 0.0 0.0 L2 2/0 50.54 MB 0.5 0.1 0.1 0.1 0.1 0.1 0.5 2.0 150.8 150.7 0.84 0.81 1 0.836 127K 41 0.0 0.0 L3 16/0 825.46 MB 0.8 0.1 0.1 0.0 0.1 0.1 0.5 1.0 134.9 134.9 0.92 0.86 1 0.924 126K 0 0.0 0.0 L4 192/0 9.58 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.6 1.0 83.9 83.9 1.33 0.93 1 1.334 113K 0 0.0 0.0 L5 1268/0 86.25 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.8 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1492/4 98.96 GB 0.0 2.9 0.7 2.2 5.8 3.6 2.4 2.0 100.2 196.6 30.03 28.10 59 0.509 3054K 2891 0.0 0.0 Int 0/0 0.00 KB 0.0 2.9 0.7 2.2 5.8 3.6 2.3 2.0 100.2 196.6 30.03 28.10 59 0.509 3054K 2891 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.9 0.7 2.2 2.9 0.7 0.0 0.0 156.5 156.4 19.22 18.41 13 1.479 3054K 2891 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 268.0 10.81 9.70 46 0.235 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 14.8 total, 14.5 interval Flush(GB): cumulative 2.830, interval 2.830 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.77 GB write, 399.69 MB/s write, 2.94 GB read, 203.69 MB/s read, 30.0 seconds Interval compaction: 5.77 GB write, 406.43 MB/s write, 2.94 GB read, 207.13 MB/s read, 30.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 0, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 0, ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 671108 Average: 2.9256 StdDev: 12.60 Min: 1 Median: 2.1483 Max: 3255 Percentiles: P50: 2.15 P75: 2.66 P99: 8.62 P99.9: 21.04 P99.99: 48.44 ------------------------------------------------------ [ 0, 1 ] 66 0.010% 0.010% ( 1, 2 ] 286752 42.728% 42.738% ######### ( 2, 3 ] 328526 48.953% 91.691% ########## ( 3, 4 ] 25214 3.757% 95.448% # ( 4, 6 ] 20533 3.060% 98.507% # ( 6, 10 ] 5048 0.752% 99.260% ( 10, 15 ] 1376 0.205% 99.465% ( 15, 22 ] 3390 0.505% 99.970% ( 22, 34 ] 125 0.019% 99.988% ( 34, 51 ] 14 0.002% 99.990% ( 51, 76 ] 3 0.000% 99.991% ( 76, 110 ] 4 0.001% 99.992% ( 110, 170 ] 1 0.000% 99.992% ( 170, 250 ] 43 0.006% 99.998% ( 250, 380 ] 12 0.002% 100.000% ( 380, 580 ] 21 0.003% 100.003% ( 580, 870 ] 5 0.001% 100.004% ( 870, 1300 ] 13 0.002% 100.006% ( 1300, 1900 ] 2 0.000% 100.006% ( 1900, 2900 ] 10 0.001% 100.007% ( 2900, 4400 ] 2 0.000% 100.008% ** Level 1 read latency histogram (micros): Count: 47990 Average: 3.1646 StdDev: 17.65 Min: 1 Median: 2.3360 Max: 3250 Percentiles: P50: 2.34 P75: 2.77 P99: 8.34 P99.9: 20.72 P99.99: 609.14 ------------------------------------------------------ [ 0, 1 ] 10 0.021% 0.021% ( 1, 2 ] 14710 30.652% 30.673% ###### ( 2, 3 ] 27605 57.522% 88.195% ############ ( 3, 4 ] 3024 6.301% 94.497% # ( 4, 6 ] 1864 3.884% 98.381% # ( 6, 10 ] 507 1.056% 99.437% ( 10, 15 ] 93 0.194% 99.631% ( 15, 22 ] 158 0.329% 99.960% ( 22, 34 ] 4 0.008% 99.969% ( 34, 51 ] 1 0.002% 99.971% ( 51, 76 ] 1 0.002% 99.973% ( 76, 110 ] 1 0.002% 99.975% ( 170, 250 ] 2 0.004% 99.979% ( 250, 380 ] 1 0.002% 99.981% ( 380, 580 ] 4 0.008% 99.990% ( 580, 870 ] 2 0.004% 99.994% ( 870, 1300 ] 2 0.004% 99.998% ( 2900, 4400 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 31668 Average: 3.6677 StdDev: 37.26 Min: 1 Median: 2.5360 Max: 5586 Percentiles: P50: 2.54 P75: 2.89 P99: 9.46 P99.9: 20.08 P99.99: 1549.96 ------------------------------------------------------ [ 0, 1 ] 4 0.013% 0.013% ( 1, 2 ] 3763 11.883% 11.895% ## ( 2, 3 ] 22514 71.094% 82.989% ############## ( 3, 4 ] 3047 9.622% 92.611% ## ( 4, 6 ] 1567 4.948% 97.559% # ( 6, 10 ] 528 1.667% 99.226% ( 10, 15 ] 164 0.518% 99.744% ( 15, 22 ] 68 0.215% 99.959% ( 22, 34 ] 5 0.016% 99.975% ( 34, 51 ] 1 0.003% 99.978% ( 380, 580 ] 1 0.003% 99.981% ( 580, 870 ] 2 0.006% 99.987% ( 1300, 1900 ] 2 0.006% 99.994% ( 1900, 2900 ] 1 0.003% 99.997% ( 4400, 6600 ] 1 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 28514 Average: 9.0516 StdDev: 153.57 Min: 1 Median: 2.5681 Max: 7292 Percentiles: P50: 2.57 P75: 2.92 P99: 11.50 P99.9: 1861.45 P99.99: 6763.46 ------------------------------------------------------ [ 0, 1 ] 6 0.021% 0.021% ( 1, 2 ] 2830 9.925% 9.946% ## ( 2, 3 ] 20104 70.506% 80.452% ############## ( 3, 4 ] 3454 12.113% 92.565% ## ( 4, 6 ] 1239 4.345% 96.910% # ( 6, 10 ] 547 1.918% 98.829% ( 10, 15 ] 163 0.572% 99.400% ( 15, 22 ] 91 0.319% 99.719% ( 22, 34 ] 10 0.035% 99.755% ( 34, 51 ] 1 0.004% 99.758% ( 76, 110 ] 1 0.004% 99.762% ( 170, 250 ] 1 0.004% 99.765% ( 380, 580 ] 8 0.028% 99.793% ( 580, 870 ] 13 0.046% 99.839% ( 870, 1300 ] 10 0.035% 99.874% ( 1300, 1900 ] 8 0.028% 99.902% ( 1900, 2900 ] 4 0.014% 99.916% ( 2900, 4400 ] 9 0.032% 99.947% ( 4400, 6600 ] 12 0.042% 99.989% ( 6600, 9900 ] 3 0.011% 100.000% ** Level 4 read latency histogram (micros): Count: 290 Average: 104.2241 StdDev: 664.24 Min: 1 Median: 2.6111 Max: 7453 Percentiles: P50: 2.61 P75: 4.72 P99: 3050.00 P99.9: 7453.00 P99.99: 7453.00 ------------------------------------------------------ [ 0, 1 ] 42 14.483% 14.483% ### ( 1, 2 ] 81 27.931% 42.414% ###### ( 2, 3 ] 36 12.414% 54.828% ## ( 3, 4 ] 48 16.552% 71.379% ### ( 4, 6 ] 29 10.000% 81.379% ## ( 6, 10 ] 20 6.897% 88.276% # ( 10, 15 ] 9 3.103% 91.379% # ( 15, 22 ] 6 2.069% 93.448% ( 22, 34 ] 1 0.345% 93.793% ( 110, 170 ] 3 1.034% 94.828% ( 170, 250 ] 3 1.034% 95.862% ( 380, 580 ] 2 0.690% 96.552% ( 580, 870 ] 2 0.690% 97.241% ( 870, 1300 ] 2 0.690% 97.931% ( 1300, 1900 ] 1 0.345% 98.276% ( 1900, 2900 ] 2 0.690% 98.966% ( 2900, 4400 ] 1 0.345% 99.310% ( 6600, 9900 ] 2 0.690% 100.000% ** Level 5 read latency histogram (micros): Count: 60 Average: 526.3667 StdDev: 569.76 Min: 97 Median: 213.6364 Max: 2039 Percentiles: P50: 213.64 P75: 754.00 P99: 2039.00 P99.9: 2039.00 P99.99: 2039.00 ------------------------------------------------------ ( 76, 110 ] 4 6.667% 6.667% # ( 110, 170 ] 20 33.333% 40.000% ####### ( 170, 250 ] 11 18.333% 58.333% #### ( 250, 380 ] 2 3.333% 61.667% # ( 380, 580 ] 5 8.333% 70.000% ## ( 580, 870 ] 5 8.333% 78.333% ## ( 870, 1300 ] 2 3.333% 81.667% # ( 1300, 1900 ] 9 15.000% 96.667% ### ( 1900, 2900 ] 2 3.333% 100.000% # ** DB Stats ** Uptime(secs): 14.8 total, 4.8 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 199.89 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 204.12 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 2024/11/19-09:36:34 ... thread 0: (1000000,4000000) ops and (204901.7,206136.3) ops/second in (4.880388,19.404634) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/8 3.20 GB 276.9 2.7 0.0 2.7 6.5 3.8 0.0 1.7 90.0 214.8 30.79 29.02 72 0.428 2814K 2661 0.0 0.0 L1 1/0 4.53 MB 0.5 0.6 0.4 0.2 0.6 0.4 0.1 1.4 150.3 150.1 4.19 4.08 2 2.095 639K 889 0.0 0.0 L2 2/0 50.54 MB 0.5 0.1 0.1 0.1 0.1 0.1 0.5 2.0 150.8 150.7 0.84 0.81 1 0.836 127K 41 0.0 0.0 L3 16/0 825.46 MB 0.8 0.1 0.1 0.0 0.1 0.1 0.5 1.0 134.9 134.9 0.92 0.86 1 0.924 126K 0 0.0 0.0 L4 192/0 9.58 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.6 1.0 83.9 83.9 1.33 0.93 1 1.334 113K 0 0.0 0.0 L5 1268/0 86.25 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.8 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1498/8 99.89 GB 0.0 3.7 0.7 3.0 7.4 4.5 2.4 2.0 98.9 199.7 38.07 35.70 77 0.494 3822K 3591 0.0 0.0 Int 0/0 0.00 KB 0.0 3.7 0.7 3.0 7.4 4.5 2.3 2.0 98.9 199.7 38.07 35.70 77 0.494 3822K 3591 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.7 0.7 3.0 3.7 0.7 0.0 0.0 158.5 158.3 23.75 22.87 16 1.484 3822K 3591 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 268.4 14.32 12.83 61 0.235 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 19.7 total, 19.4 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.43 GB write, 386.95 MB/s write, 3.68 GB read, 191.57 MB/s read, 38.1 seconds Interval compaction: 7.43 GB write, 391.83 MB/s write, 3.68 GB read, 193.99 MB/s read, 38.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 0, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 0, ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 920737 Average: 2.9101 StdDev: 12.45 Min: 1 Median: 2.1339 Max: 3660 Percentiles: P50: 2.13 P75: 2.65 P99: 8.64 P99.9: 21.10 P99.99: 49.86 ------------------------------------------------------ [ 0, 1 ] 83 0.009% 0.009% ( 1, 2 ] 400361 43.483% 43.492% ######### ( 2, 3 ] 447495 48.602% 92.094% ########## ( 3, 4 ] 32295 3.508% 95.601% # ( 4, 6 ] 26919 2.924% 98.525% # ( 6, 10 ] 6639 0.721% 99.246% ( 10, 15 ] 1708 0.186% 99.431% ( 15, 22 ] 4958 0.538% 99.970% ( 22, 34 ] 174 0.019% 99.989% ( 34, 51 ] 16 0.002% 99.990% ( 51, 76 ] 4 0.000% 99.991% ( 76, 110 ] 6 0.001% 99.991% ( 110, 170 ] 1 0.000% 99.992% ( 170, 250 ] 56 0.006% 99.998% ( 250, 380 ] 16 0.002% 99.999% ( 380, 580 ] 28 0.003% 100.002% ( 580, 870 ] 10 0.001% 100.003% ( 870, 1300 ] 17 0.002% 100.005% ( 1300, 1900 ] 2 0.000% 100.006% ( 1900, 2900 ] 11 0.001% 100.007% ( 2900, 4400 ] 4 0.000% 100.007% ** Level 1 read latency histogram (micros): Count: 47990 Average: 3.1646 StdDev: 17.65 Min: 1 Median: 2.3360 Max: 3250 Percentiles: P50: 2.34 P75: 2.77 P99: 8.34 P99.9: 20.72 P99.99: 609.14 ------------------------------------------------------ [ 0, 1 ] 10 0.021% 0.021% ( 1, 2 ] 14710 30.652% 30.673% ###### ( 2, 3 ] 27605 57.522% 88.195% ############ ( 3, 4 ] 3024 6.301% 94.497% # ( 4, 6 ] 1864 3.884% 98.381% # ( 6, 10 ] 507 1.056% 99.437% ( 10, 15 ] 93 0.194% 99.631% ( 15, 22 ] 158 0.329% 99.960% ( 22, 34 ] 4 0.008% 99.969% ( 34, 51 ] 1 0.002% 99.971% ( 51, 76 ] 1 0.002% 99.973% ( 76, 110 ] 1 0.002% 99.975% ( 170, 250 ] 2 0.004% 99.979% ( 250, 380 ] 1 0.002% 99.981% ( 380, 580 ] 4 0.008% 99.990% ( 580, 870 ] 2 0.004% 99.994% ( 870, 1300 ] 2 0.004% 99.998% ( 2900, 4400 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 31668 Average: 3.6677 StdDev: 37.26 Min: 1 Median: 2.5360 Max: 5586 Percentiles: P50: 2.54 P75: 2.89 P99: 9.46 P99.9: 20.08 P99.99: 1549.96 ------------------------------------------------------ [ 0, 1 ] 4 0.013% 0.013% ( 1, 2 ] 3763 11.883% 11.895% ## ( 2, 3 ] 22514 71.094% 82.989% ############## ( 3, 4 ] 3047 9.622% 92.611% ## ( 4, 6 ] 1567 4.948% 97.559% # ( 6, 10 ] 528 1.667% 99.226% ( 10, 15 ] 164 0.518% 99.744% ( 15, 22 ] 68 0.215% 99.959% ( 22, 34 ] 5 0.016% 99.975% ( 34, 51 ] 1 0.003% 99.978% ( 380, 580 ] 1 0.003% 99.981% ( 580, 870 ] 2 0.006% 99.987% ( 1300, 1900 ] 2 0.006% 99.994% ( 1900, 2900 ] 1 0.003% 99.997% ( 4400, 6600 ] 1 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 28514 Average: 9.0516 StdDev: 153.57 Min: 1 Median: 2.5681 Max: 7292 Percentiles: P50: 2.57 P75: 2.92 P99: 11.50 P99.9: 1861.45 P99.99: 6763.46 ------------------------------------------------------ [ 0, 1 ] 6 0.021% 0.021% ( 1, 2 ] 2830 9.925% 9.946% ## ( 2, 3 ] 20104 70.506% 80.452% ############## ( 3, 4 ] 3454 12.113% 92.565% ## ( 4, 6 ] 1239 4.345% 96.910% # ( 6, 10 ] 547 1.918% 98.829% ( 10, 15 ] 163 0.572% 99.400% ( 15, 22 ] 91 0.319% 99.719% ( 22, 34 ] 10 0.035% 99.755% ( 34, 51 ] 1 0.004% 99.758% ( 76, 110 ] 1 0.004% 99.762% ( 170, 250 ] 1 0.004% 99.765% ( 380, 580 ] 8 0.028% 99.793% ( 580, 870 ] 13 0.046% 99.839% ( 870, 1300 ] 10 0.035% 99.874% ( 1300, 1900 ] 8 0.028% 99.902% ( 1900, 2900 ] 4 0.014% 99.916% ( 2900, 4400 ] 9 0.032% 99.947% ( 4400, 6600 ] 12 0.042% 99.989% ( 6600, 9900 ] 3 0.011% 100.000% ** Level 4 read latency histogram (micros): Count: 290 Average: 104.2241 StdDev: 664.24 Min: 1 Median: 2.6111 Max: 7453 Percentiles: P50: 2.61 P75: 4.72 P99: 3050.00 P99.9: 7453.00 P99.99: 7453.00 ------------------------------------------------------ [ 0, 1 ] 42 14.483% 14.483% ### ( 1, 2 ] 81 27.931% 42.414% ###### ( 2, 3 ] 36 12.414% 54.828% ## ( 3, 4 ] 48 16.552% 71.379% ### ( 4, 6 ] 29 10.000% 81.379% ## ( 6, 10 ] 20 6.897% 88.276% # ( 10, 15 ] 9 3.103% 91.379% # ( 15, 22 ] 6 2.069% 93.448% ( 22, 34 ] 1 0.345% 93.793% ( 110, 170 ] 3 1.034% 94.828% ( 170, 250 ] 3 1.034% 95.862% ( 380, 580 ] 2 0.690% 96.552% ( 580, 870 ] 2 0.690% 97.241% ( 870, 1300 ] 2 0.690% 97.931% ( 1300, 1900 ] 1 0.345% 98.276% ( 1900, 2900 ] 2 0.690% 98.966% ( 2900, 4400 ] 1 0.345% 99.310% ( 6600, 9900 ] 2 0.690% 100.000% ** Level 5 read latency histogram (micros): Count: 60 Average: 526.3667 StdDev: 569.76 Min: 97 Median: 213.6364 Max: 2039 Percentiles: P50: 213.64 P75: 754.00 P99: 2039.00 P99.9: 2039.00 P99.99: 2039.00 ------------------------------------------------------ ( 76, 110 ] 4 6.667% 6.667% # ( 110, 170 ] 20 33.333% 40.000% ####### ( 170, 250 ] 11 18.333% 58.333% #### ( 250, 380 ] 2 3.333% 61.667% # ( 380, 580 ] 5 8.333% 70.000% ## ( 580, 870 ] 5 8.333% 78.333% ## ( 870, 1300 ] 2 3.333% 81.667% # ( 1300, 1900 ] 9 15.000% 96.667% ### ( 1900, 2900 ] 2 3.333% 100.000% # ** DB Stats ** Uptime(secs): 19.7 total, 4.9 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 200.33 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 201.66 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/19-09:37:03 ... thread 0: (1000000,5000000) ops and (35084.1,104367.6) ops/second in (28.502935,47.907569) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 4/3 629.48 MB 6.3 3.9 0.0 3.9 8.7 4.8 0.0 1.8 95.1 211.0 42.38 39.97 94 0.451 4093K 3870 0.0 0.0 L1 6/6 561.01 MB 0.0 4.2 4.1 0.2 4.2 4.0 0.1 1.0 144.4 142.2 30.11 28.75 3 10.036 4414K 66K 0.0 0.0 L2 26/20 3.07 GB 7.7 0.1 0.1 0.1 0.1 0.1 3.5 2.0 150.8 150.7 0.84 0.81 1 0.836 127K 41 0.0 0.0 L3 16/16 825.46 MB 0.0 0.1 0.1 0.0 0.1 0.1 0.5 1.0 134.9 134.9 0.92 0.86 1 0.924 126K 0 0.0 0.0 L4 192/0 9.58 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.6 1.0 83.9 83.9 1.33 0.93 1 1.334 113K 0 0.0 0.0 L5 1268/0 86.25 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.8 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1512/45 100.87 GB 0.0 8.5 4.3 4.2 13.3 9.1 5.5 2.8 115.7 179.7 75.58 71.33 100 0.756 8877K 70K 0.0 0.0 Int 0/0 0.00 KB 0.0 8.5 4.3 4.2 13.3 9.1 5.4 2.8 115.7 179.7 75.58 71.33 100 0.756 8877K 70K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 8.5 4.3 4.2 8.5 4.3 0.0 0.0 152.8 151.5 57.22 54.89 22 2.601 8877K 70K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.8 4.8 0.0 0.0 0.0 267.6 18.36 16.44 78 0.235 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 48.2 total, 47.9 interval Flush(GB): cumulative 4.799, interval 4.799 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.27 GB write, 282.12 MB/s write, 8.54 GB read, 181.55 MB/s read, 75.6 seconds Interval compaction: 13.27 GB write, 283.57 MB/s write, 8.54 GB read, 182.48 MB/s read, 75.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 12, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 12, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 0, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 12, total-stops: 0, interval: 12 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2110248 Average: 3.4230 StdDev: 14.68 Min: 1 Median: 2.1545 Max: 3660 Percentiles: P50: 2.15 P75: 2.65 P99: 16.49 P99.9: 162.53 P99.99: 459.56 ------------------------------------------------------ [ 0, 1 ] 129 0.006% 0.006% ( 1, 2 ] 891251 42.234% 42.241% ######## ( 2, 3 ] 1059541 50.209% 92.450% ########## ( 3, 4 ] 62725 2.972% 95.422% # ( 4, 6 ] 46809 2.218% 97.640% ( 6, 10 ] 19697 0.933% 98.574% ( 10, 15 ] 5280 0.250% 98.824% ( 15, 22 ] 17417 0.825% 99.649% ( 22, 34 ] 1020 0.048% 99.698% ( 34, 51 ] 45 0.002% 99.700% ( 51, 76 ] 14 0.001% 99.701% ( 76, 110 ] 889 0.042% 99.743% ( 110, 170 ] 3794 0.180% 99.922% ( 170, 250 ] 978 0.046% 99.969% ( 250, 380 ] 336 0.016% 99.985% ( 380, 580 ] 284 0.013% 99.998% ( 580, 870 ] 73 0.003% 100.002% ( 870, 1300 ] 29 0.001% 100.003% ( 1300, 1900 ] 3 0.000% 100.003% ( 1900, 2900 ] 22 0.001% 100.004% ( 2900, 4400 ] 4 0.000% 100.004% ** Level 1 read latency histogram (micros): Count: 49451 Average: 6.2485 StdDev: 52.59 Min: 1 Median: 2.3370 Max: 3269 Percentiles: P50: 2.34 P75: 2.78 P99: 16.24 P99.9: 856.84 P99.99: 1733.28 ------------------------------------------------------ [ 0, 1 ] 34 0.069% 0.069% ( 1, 2 ] 15230 30.798% 30.867% ###### ( 2, 3 ] 28073 56.769% 87.636% ########### ( 3, 4 ] 3081 6.230% 93.867% # ( 4, 6 ] 1885 3.812% 97.679% # ( 6, 10 ] 526 1.064% 98.742% ( 10, 15 ] 97 0.196% 98.938% ( 15, 22 ] 172 0.348% 99.286% ( 22, 34 ] 5 0.010% 99.296% ( 34, 51 ] 1 0.002% 99.298% ( 51, 76 ] 1 0.002% 99.300% ( 76, 110 ] 4 0.008% 99.308% ( 110, 170 ] 97 0.196% 99.505% ( 170, 250 ] 44 0.089% 99.594% ( 250, 380 ] 34 0.069% 99.662% ( 380, 580 ] 66 0.133% 99.796% ( 580, 870 ] 54 0.109% 99.905% ( 870, 1300 ] 37 0.075% 99.980% ( 1300, 1900 ] 7 0.014% 99.994% ( 1900, 2900 ] 2 0.004% 99.998% ( 2900, 4400 ] 2 0.004% 100.002% ** Level 2 read latency histogram (micros): Count: 31689 Average: 4.0830 StdDev: 41.12 Min: 1 Median: 2.5364 Max: 5586 Percentiles: P50: 2.54 P75: 2.89 P99: 9.61 P99.9: 27.55 P99.99: 1549.33 ------------------------------------------------------ [ 0, 1 ] 4 0.013% 0.013% ( 1, 2 ] 3763 11.875% 11.887% ## ( 2, 3 ] 22514 71.047% 82.934% ############## ( 3, 4 ] 3047 9.615% 92.549% ## ( 4, 6 ] 1567 4.945% 97.494% # ( 6, 10 ] 528 1.666% 99.161% ( 10, 15 ] 164 0.518% 99.678% ( 15, 22 ] 68 0.215% 99.893% ( 22, 34 ] 5 0.016% 99.908% ( 34, 51 ] 1 0.003% 99.912% ( 250, 380 ] 3 0.009% 99.921% ( 380, 580 ] 7 0.022% 99.943% ( 580, 870 ] 11 0.035% 99.978% ( 870, 1300 ] 3 0.009% 99.987% ( 1300, 1900 ] 2 0.006% 99.994% ( 1900, 2900 ] 1 0.003% 99.997% ( 4400, 6600 ] 1 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 28516 Average: 9.0786 StdDev: 153.60 Min: 1 Median: 2.5681 Max: 7292 Percentiles: P50: 2.57 P75: 2.92 P99: 11.56 P99.9: 1861.30 P99.99: 6763.24 ------------------------------------------------------ [ 0, 1 ] 6 0.021% 0.021% ( 1, 2 ] 2830 9.924% 9.945% ## ( 2, 3 ] 20104 70.501% 80.446% ############## ( 3, 4 ] 3454 12.112% 92.559% ## ( 4, 6 ] 1239 4.345% 96.903% # ( 6, 10 ] 547 1.918% 98.822% ( 10, 15 ] 163 0.572% 99.393% ( 15, 22 ] 91 0.319% 99.712% ( 22, 34 ] 10 0.035% 99.748% ( 34, 51 ] 1 0.004% 99.751% ( 76, 110 ] 1 0.004% 99.755% ( 170, 250 ] 1 0.004% 99.758% ( 380, 580 ] 10 0.035% 99.793% ( 580, 870 ] 13 0.046% 99.839% ( 870, 1300 ] 10 0.035% 99.874% ( 1300, 1900 ] 8 0.028% 99.902% ( 1900, 2900 ] 4 0.014% 99.916% ( 2900, 4400 ] 9 0.032% 99.947% ( 4400, 6600 ] 12 0.042% 99.989% ( 6600, 9900 ] 3 0.011% 100.000% ** Level 4 read latency histogram (micros): Count: 290 Average: 104.2241 StdDev: 664.24 Min: 1 Median: 2.6111 Max: 7453 Percentiles: P50: 2.61 P75: 4.72 P99: 3050.00 P99.9: 7453.00 P99.99: 7453.00 ------------------------------------------------------ [ 0, 1 ] 42 14.483% 14.483% ### ( 1, 2 ] 81 27.931% 42.414% ###### ( 2, 3 ] 36 12.414% 54.828% ## ( 3, 4 ] 48 16.552% 71.379% ### ( 4, 6 ] 29 10.000% 81.379% ## ( 6, 10 ] 20 6.897% 88.276% # ( 10, 15 ] 9 3.103% 91.379% # ( 15, 22 ] 6 2.069% 93.448% ( 22, 34 ] 1 0.345% 93.793% ( 110, 170 ] 3 1.034% 94.828% ( 170, 250 ] 3 1.034% 95.862% ( 380, 580 ] 2 0.690% 96.552% ( 580, 870 ] 2 0.690% 97.241% ( 870, 1300 ] 2 0.690% 97.931% ( 1300, 1900 ] 1 0.345% 98.276% ( 1900, 2900 ] 2 0.690% 98.966% ( 2900, 4400 ] 1 0.345% 99.310% ( 6600, 9900 ] 2 0.690% 100.000% ** Level 5 read latency histogram (micros): Count: 60 Average: 526.3667 StdDev: 569.76 Min: 97 Median: 213.6364 Max: 2039 Percentiles: P50: 213.64 P75: 754.00 P99: 2039.00 P99.9: 2039.00 P99.99: 2039.00 ------------------------------------------------------ ( 76, 110 ] 4 6.667% 6.667% # ( 110, 170 ] 20 33.333% 40.000% ####### ( 170, 250 ] 11 18.333% 58.333% #### ( 250, 380 ] 2 3.333% 61.667% # ( 380, 580 ] 5 8.333% 70.000% ## ( 580, 870 ] 5 8.333% 78.333% ## ( 870, 1300 ] 2 3.333% 81.667% # ( 1300, 1900 ] 9 15.000% 96.667% ### ( 1900, 2900 ] 2 3.333% 100.000% # ** DB Stats ** Uptime(secs): 48.2 total, 28.5 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 102.19 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:23.274 H:M:S, 48.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 34.53 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:23.274 H:M:S, 81.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 7 num-running-flushes: 0 2024/11/19-09:37:08 ... thread 0: (1000000,6000000) ops and (182487.8,112386.1) ops/second in (5.479819,53.387388) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/7 1.48 GB 69.2 4.4 0.0 4.4 10.1 5.7 0.0 1.8 90.3 205.7 50.22 46.30 110 0.457 4605K 4384 0.0 0.0 L1 6/6 561.01 MB 0.0 4.2 4.1 0.2 4.2 4.0 0.1 1.0 144.4 142.2 30.11 28.75 3 10.036 4414K 66K 0.0 0.0 L2 20/17 2.52 GB 3.9 0.1 0.1 0.1 0.1 0.1 3.5 2.0 150.8 150.7 0.84 0.81 1 0.836 127K 41 0.0 0.0 L3 20/3 1.06 GB 0.8 1.2 0.7 0.6 1.2 0.7 0.5 1.8 119.9 119.7 10.49 9.01 4 2.623 1278K 2978 0.0 0.0 L4 191/0 9.57 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.8 1.0 83.9 83.9 1.33 0.93 1 1.334 113K 0 0.0 0.0 L5 1272/0 86.55 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1521/33 101.72 GB 0.0 10.1 4.9 5.2 15.7 10.5 6.1 2.8 111.6 173.2 92.99 85.79 119 0.781 10M 74K 0.0 0.0 Int 0/0 0.00 KB 0.0 10.1 4.9 5.2 15.7 10.5 6.0 2.8 111.6 173.2 92.99 85.79 119 0.781 10M 74K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.1 4.9 5.2 10.1 4.8 0.0 0.0 148.1 147.0 70.08 66.18 27 2.596 10M 74K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 253.1 22.91 19.62 92 0.249 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 53.6 total, 53.4 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 15.72 GB write, 300.21 MB/s write, 10.14 GB read, 193.54 MB/s read, 93.0 seconds Interval compaction: 15.72 GB write, 301.59 MB/s write, 10.14 GB read, 194.43 MB/s read, 93.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 12, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 12, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 0, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 12, total-stops: 0, interval: 12 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2411491 Average: 3.4983 StdDev: 15.78 Min: 1 Median: 2.2428 Max: 3660 Percentiles: P50: 2.24 P75: 2.72 P99: 15.48 P99.9: 160.77 P99.99: 477.29 ------------------------------------------------------ [ 0, 1 ] 129 0.005% 0.005% ( 1, 2 ] 900621 37.347% 37.352% ####### ( 2, 3 ] 1256257 52.095% 89.447% ########## ( 3, 4 ] 121662 5.045% 94.492% # ( 4, 6 ] 73202 3.036% 97.528% # ( 6, 10 ] 26122 1.083% 98.611% ( 10, 15 ] 8169 0.339% 98.950% ( 15, 22 ] 17684 0.733% 99.683% ( 22, 34 ] 1041 0.043% 99.726% ( 34, 51 ] 48 0.002% 99.728% ( 51, 76 ] 15 0.001% 99.729% ( 76, 110 ] 891 0.037% 99.766% ( 110, 170 ] 3827 0.159% 99.924% ( 170, 250 ] 1023 0.042% 99.967% ( 250, 380 ] 392 0.016% 99.983% ( 380, 580 ] 343 0.014% 99.997% ( 580, 870 ] 115 0.005% 100.002% ( 870, 1300 ] 45 0.002% 100.004% ( 1300, 1900 ] 9 0.000% 100.004% ( 1900, 2900 ] 24 0.001% 100.005% ( 2900, 4400 ] 10 0.000% 100.006% ** Level 1 read latency histogram (micros): Count: 637943 Average: 4.8209 StdDev: 39.89 Min: 1 Median: 2.7626 Max: 4456 Percentiles: P50: 2.76 P75: 3.47 P99: 12.58 P99.9: 213.94 P99.99: 460.79 ------------------------------------------------------ [ 0, 1 ] 34 0.005% 0.005% ( 1, 2 ] 20482 3.211% 3.216% # ( 2, 3 ] 391342 61.344% 64.560% ############ ( 3, 4 ] 141415 22.167% 86.728% #### ( 4, 6 ] 59002 9.249% 95.976% ## ( 6, 10 ] 14668 2.299% 98.276% ( 10, 15 ] 8952 1.403% 99.679% ( 15, 22 ] 856 0.134% 99.813% ( 22, 34 ] 36 0.006% 99.819% ( 34, 51 ] 5 0.001% 99.820% ( 51, 76 ] 4 0.001% 99.820% ( 76, 110 ] 21 0.003% 99.823% ( 110, 170 ] 283 0.044% 99.868% ( 170, 250 ] 377 0.059% 99.927% ( 250, 380 ] 302 0.047% 99.974% ( 380, 580 ] 253 0.040% 100.014% ( 580, 870 ] 160 0.025% 100.039% ( 870, 1300 ] 56 0.009% 100.048% ( 1300, 1900 ] 18 0.003% 100.051% ( 1900, 2900 ] 10 0.002% 100.052% ( 2900, 4400 ] 57 0.009% 100.061% ( 4400, 6600 ] 1 0.000% 100.061% ** Level 2 read latency histogram (micros): Count: 60285 Average: 5.0272 StdDev: 51.46 Min: 1 Median: 2.6617 Max: 5586 Percentiles: P50: 2.66 P75: 3.16 P99: 12.12 P99.9: 490.72 P99.99: 3322.46 ------------------------------------------------------ [ 0, 1 ] 4 0.007% 0.007% ( 1, 2 ] 3892 6.456% 6.463% # ( 2, 3 ] 39668 65.801% 72.263% ############# ( 3, 4 ] 10575 17.542% 89.805% #### ( 4, 6 ] 4060 6.735% 96.540% # ( 6, 10 ] 1208 2.004% 98.544% ( 10, 15 ] 649 1.077% 99.620% ( 15, 22 ] 87 0.144% 99.764% ( 22, 34 ] 5 0.008% 99.773% ( 34, 51 ] 1 0.002% 99.774% ( 110, 170 ] 3 0.005% 99.779% ( 170, 250 ] 28 0.046% 99.826% ( 250, 380 ] 27 0.045% 99.871% ( 380, 580 ] 32 0.053% 99.924% ( 580, 870 ] 23 0.038% 99.962% ( 870, 1300 ] 10 0.017% 99.978% ( 1300, 1900 ] 4 0.007% 99.985% ( 1900, 2900 ] 1 0.002% 99.987% ( 2900, 4400 ] 7 0.012% 99.998% ( 4400, 6600 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 116070 Average: 9.2549 StdDev: 116.75 Min: 1 Median: 2.7486 Max: 7292 Percentiles: P50: 2.75 P75: 3.43 P99: 14.67 P99.9: 1285.16 P99.99: 5143.43 ------------------------------------------------------ [ 0, 1 ] 13 0.011% 0.011% ( 1, 2 ] 3382 2.914% 2.925% # ( 2, 3 ] 72986 62.881% 65.806% ############# ( 3, 4 ] 25072 21.601% 87.407% #### ( 4, 6 ] 9469 8.158% 95.565% ## ( 6, 10 ] 2499 2.153% 97.718% ( 10, 15 ] 1592 1.372% 99.089% ( 15, 22 ] 155 0.134% 99.223% ( 22, 34 ] 19 0.016% 99.239% ( 34, 51 ] 9 0.008% 99.247% ( 76, 110 ] 21 0.018% 99.265% ( 110, 170 ] 107 0.092% 99.357% ( 170, 250 ] 206 0.177% 99.535% ( 250, 380 ] 165 0.142% 99.677% ( 380, 580 ] 145 0.125% 99.802% ( 580, 870 ] 84 0.072% 99.874% ( 870, 1300 ] 31 0.027% 99.901% ( 1300, 1900 ] 16 0.014% 99.915% ( 1900, 2900 ] 10 0.009% 99.923% ( 2900, 4400 ] 73 0.063% 99.986% ( 4400, 6600 ] 13 0.011% 99.997% ( 6600, 9900 ] 3 0.003% 100.000% ** Level 4 read latency histogram (micros): Count: 290 Average: 104.2241 StdDev: 664.24 Min: 1 Median: 2.6111 Max: 7453 Percentiles: P50: 2.61 P75: 4.72 P99: 3050.00 P99.9: 7453.00 P99.99: 7453.00 ------------------------------------------------------ [ 0, 1 ] 42 14.483% 14.483% ### ( 1, 2 ] 81 27.931% 42.414% ###### ( 2, 3 ] 36 12.414% 54.828% ## ( 3, 4 ] 48 16.552% 71.379% ### ( 4, 6 ] 29 10.000% 81.379% ## ( 6, 10 ] 20 6.897% 88.276% # ( 10, 15 ] 9 3.103% 91.379% # ( 15, 22 ] 6 2.069% 93.448% ( 22, 34 ] 1 0.345% 93.793% ( 110, 170 ] 3 1.034% 94.828% ( 170, 250 ] 3 1.034% 95.862% ( 380, 580 ] 2 0.690% 96.552% ( 580, 870 ] 2 0.690% 97.241% ( 870, 1300 ] 2 0.690% 97.931% ( 1300, 1900 ] 1 0.345% 98.276% ( 1900, 2900 ] 2 0.690% 98.966% ( 2900, 4400 ] 1 0.345% 99.310% ( 6600, 9900 ] 2 0.690% 100.000% ** Level 5 read latency histogram (micros): Count: 80 Average: 738.0625 StdDev: 913.76 Min: 97 Median: 241.1111 Max: 3719 Percentiles: P50: 241.11 P75: 1156.67 P99: 3719.00 P99.9: 3719.00 P99.99: 3719.00 ------------------------------------------------------ ( 76, 110 ] 4 5.000% 5.000% # ( 110, 170 ] 20 25.000% 30.000% ##### ( 170, 250 ] 18 22.500% 52.500% ##### ( 250, 380 ] 4 5.000% 57.500% # ( 380, 580 ] 7 8.750% 66.250% ## ( 580, 870 ] 5 6.250% 72.500% # ( 870, 1300 ] 3 3.750% 76.250% # ( 1300, 1900 ] 10 12.500% 88.750% ### ( 1900, 2900 ] 4 5.000% 93.750% # ( 2900, 4400 ] 5 6.250% 100.000% # ** DB Stats ** Uptime(secs): 53.6 total, 5.5 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 110.10 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:23.274 H:M:S, 43.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 179.60 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 5 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 8.686 micros/op 115127 ops/sec 60.585 seconds 6974999 operations; 111.6 MB/s Microseconds per write: Count: 6974999 Average: 8.6860 StdDev: 388.38 Min: 1 Median: 3.6415 Max: 795492 Percentiles: P50: 3.64 P75: 5.36 P99: 18.46 P99.9: 1265.22 P99.99: 2809.75 ------------------------------------------------------ [ 0, 1 ] 1 0.000% 0.000% ( 1, 2 ] 183899 2.637% 2.637% # ( 2, 3 ] 1955802 28.040% 30.677% ###### ( 3, 4 ] 2101079 30.123% 60.800% ###### ( 4, 6 ] 1461614 20.955% 81.755% #### ( 6, 10 ] 1087806 15.596% 97.351% ### ( 10, 15 ] 86972 1.247% 98.597% ( 15, 22 ] 56732 0.813% 99.411% ( 22, 34 ] 16576 0.238% 99.648% ( 34, 51 ] 1334 0.019% 99.668% ( 51, 76 ] 127 0.002% 99.669% ( 76, 110 ] 37 0.001% 99.670% ( 110, 170 ] 5385 0.077% 99.747% ( 170, 250 ] 1537 0.022% 99.769% ( 250, 380 ] 119 0.002% 99.771% ( 380, 580 ] 22 0.000% 99.771% ( 580, 870 ] 38 0.001% 99.772% ( 870, 1300 ] 9731 0.140% 99.911% ( 1300, 1900 ] 22 0.000% 99.912% ( 1900, 2900 ] 6011 0.086% 99.998% ( 2900, 4400 ] 142 0.002% 100.000% ( 4400, 6600 ] 7 0.000% 100.000% ( 6600, 9900 ] 1 0.000% 100.000% ( 14000, 22000 ] 1 0.000% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% ( 570000, 860000 ] 2 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 95352 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 6974999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 7198198968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 437 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 24897687 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 19019826616 rocksdb.compact.write.bytes COUNT : 18917242964 rocksdb.flush.write.bytes COUNT : 7134618497 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 50 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 197 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 7076651008 rocksdb.memtable.garbage.bytes.at.flush COUNT : 2367488 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 19093052086 rocksdb.non.last.level.read.count COUNT : 4609132 rocksdb.block.checksum.compute.count COUNT : 4608308 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 2.661931 P95 : 8.124529 P99 : 16.013549 P100 : 795488.000000 COUNT : 6974999 SUM : 53277262 rocksdb.compaction.times.micros P50 : 1185217.391304 P95 : 7160000.000000 P99 : 25520000.000000 P100 : 25918247.000000 COUNT : 68 SUM : 139994606 rocksdb.compaction.times.cpu_micros P50 : 973333.333333 P95 : 7160000.000000 P99 : 24668622.000000 P100 : 24668622.000000 COUNT : 68 SUM : 118145248 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 880.750000 P95 : 3162.500000 P99 : 6468.000000 P100 : 7569.000000 COUNT : 106 SUM : 119179 rocksdb.compaction.outfile.sync.micros P50 : 1263.352273 P95 : 8043.750000 P99 : 12300.000000 P100 : 12300.000000 COUNT : 275 SUM : 620564 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 206.756757 P95 : 1540.000000 P99 : 4444.000000 P100 : 6423.000000 COUNT : 196 SUM : 88412 rocksdb.table.open.io.micros P50 : 1435.000000 P95 : 10750.750000 P99 : 15934.000000 P100 : 15934.000000 COUNT : 437 SUM : 1083287 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 6.151931 P95 : 10.849792 P99 : 20.395254 P100 : 33077.000000 COUNT : 4607097 SUM : 46920523 rocksdb.read.block.get.micros P50 : 456.404494 P95 : 4034.375000 P99 : 6695.333333 P100 : 9877.000000 COUNT : 874 SUM : 839950 rocksdb.write.raw.block.micros P50 : 0.818688 P95 : 1.972082 P99 : 5.977885 P100 : 32849.000000 COUNT : 6311967 SUM : 22809965 rocksdb.numfiles.in.singlecompaction P50 : 3.045455 P95 : 7.933333 P99 : 21.860000 P100 : 23.000000 COUNT : 102 SUM : 366 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.574783 P95 : 1296.889267 P99 : 2698.396667 P100 : 795220.000000 COUNT : 121562 SUM : 24913208 rocksdb.sst.read.micros P50 : 2.522278 P95 : 5.460485 P99 : 14.034810 P100 : 33069.000000 COUNT : 4609360 SUM : 29519344 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 6974999 SUM : 7198198968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 228888.888889 P95 : 528833.333333 P99 : 1019800.000000 P100 : 1060508.000000 COUNT : 106 SUM : 29499738 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 239 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0