[2024-07-12 00:30:59.582214] Starting SPDK v24.05.1-pre git sha1 5fa2f5086 / DPDK 22.11.4 initialization... [2024-07-12 00:30:59.582483] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2497197 ] Set seed to 1720737059575844 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-12 00:30:59.761297] app.c: 909:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-12 00:30:59.809745] app.c: 604:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-12 00:30:59.809801] app.c: 605:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2497197' to capture a snapshot of events at runtime. [2024-07-12 00:30:59.809818] app.c: 610:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-12 00:30:59.809835] app.c: 611:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-12 00:30:59.809847] app.c: 612:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2497197 for offline analysis/debug. [2024-07-12 00:30:59.809966] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Fri Jul 12 00:31:04 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/07/12-00:31:13 ... thread 0: (1000000,1000000) ops and (109571.2,109571.2) ops/second in (9.126488,9.126488) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/9 2.63 GB 69.2 0.6 0.0 0.6 1.5 0.9 0.0 1.6 56.5 150.6 10.04 6.12 15 0.669 575K 593 0.0 0.0 L1 13/13 1.61 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 937.87 MB 0.9 2.4 1.6 0.9 2.4 1.5 0.0 1.5 103.9 103.0 23.94 17.18 10 2.394 503K 0 0.0 0.0 L4 199/0 9.71 GB 1.0 2.3 2.0 0.3 2.3 2.0 0.0 1.1 75.8 75.5 31.58 19.04 26 1.215 2044K 1477 0.0 0.0 L5 1452/0 89.32 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1692/22 104.19 GB 0.0 5.3 3.6 1.7 6.2 4.5 2.0 6.7 83.1 97.1 65.56 42.33 51 1.286 3123K 2070 0.0 0.0 Int 0/0 0.00 KB 0.0 5.3 3.6 1.7 6.2 4.5 2.0 6.7 83.1 97.1 65.56 42.33 51 1.286 3123K 2070 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.3 3.6 1.7 5.3 3.6 0.0 0.0 93.3 92.7 58.42 38.73 38 1.537 3123K 2070 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 132.4 7.14 3.60 13 0.549 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.4 total, 9.1 interval Flush(GB): cumulative 0.923, interval 0.923 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.21 GB write, 674.51 MB/s write, 5.32 GB read, 577.59 MB/s read, 65.6 seconds Interval compaction: 6.21 GB write, 697.00 MB/s write, 5.32 GB read, 596.85 MB/s read, 65.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 20, pending-compaction-bytes-delays: 8, pending-compaction-bytes-stops: 0, total-delays: 8, total-stops: 20, interval: 28 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 350230 Average: 15.7267 StdDev: 318.79 Min: 1 Median: 2.6055 Max: 39765 Percentiles: P50: 2.61 P75: 3.95 P99: 156.64 P99.9: 1266.31 P99.99: 2806.07 ------------------------------------------------------ [ 0, 1 ] 8570 2.447% 2.447% ( 1, 2 ] 90115 25.730% 28.177% ##### ( 2, 3 ] 126227 36.041% 64.218% ####### ( 3, 4 ] 39918 11.398% 75.616% ## ( 4, 6 ] 50180 14.328% 89.944% ### ( 6, 10 ] 25352 7.239% 97.182% # ( 10, 15 ] 4470 1.276% 98.459% ( 15, 22 ] 780 0.223% 98.681% ( 22, 34 ] 131 0.037% 98.719% ( 34, 51 ] 36 0.010% 98.729% ( 51, 76 ] 24 0.007% 98.736% ( 76, 110 ] 55 0.016% 98.752% ( 110, 170 ] 1120 0.320% 99.071% ( 170, 250 ] 1289 0.368% 99.440% ( 250, 380 ] 553 0.158% 99.597% ( 380, 580 ] 455 0.130% 99.727% ( 580, 870 ] 356 0.102% 99.829% ( 870, 1300 ] 271 0.077% 99.906% ( 1300, 1900 ] 178 0.051% 99.957% ( 1900, 2900 ] 128 0.037% 99.994% ( 2900, 4400 ] 122 0.035% 100.029% ( 4400, 6600 ] 44 0.013% 100.041% ( 6600, 9900 ] 28 0.008% 100.049% ( 9900, 14000 ] 19 0.005% 100.055% ( 14000, 22000 ] 42 0.012% 100.067% ( 22000, 33000 ] 12 0.003% 100.070% ( 33000, 50000 ] 5 0.001% 100.071% ** Level 1 read latency histogram (micros): Count: 419 Average: 1035.6014 StdDev: 2558.61 Min: 89 Median: 249.6154 Max: 21940 Percentiles: P50: 249.62 P75: 730.06 P99: 15296.00 P99.9: 21329.60 P99.99: 21932.96 ------------------------------------------------------ ( 76, 110 ] 5 1.193% 1.193% ( 110, 170 ] 101 24.105% 25.298% ##### ( 170, 250 ] 104 24.821% 50.119% ##### ( 250, 380 ] 47 11.217% 61.337% ## ( 380, 580 ] 35 8.353% 69.690% ## ( 580, 870 ] 43 10.263% 79.952% ## ( 870, 1300 ] 20 4.773% 84.726% # ( 1300, 1900 ] 19 4.535% 89.260% # ( 1900, 2900 ] 18 4.296% 93.556% # ( 2900, 4400 ] 8 1.909% 95.465% ( 4400, 6600 ] 3 0.716% 96.181% ( 6600, 9900 ] 5 1.193% 97.375% ( 9900, 14000 ] 6 1.432% 98.807% ( 14000, 22000 ] 5 1.193% 100.000% ** Level 2 read latency histogram (micros): Count: 228603 Average: 19.1489 StdDev: 395.32 Min: 1 Median: 2.9286 Max: 40782 Percentiles: P50: 2.93 P75: 4.93 P99: 14.82 P99.9: 1645.63 P99.99: 3491.53 ------------------------------------------------------ [ 0, 1 ] 815 0.357% 0.357% ( 1, 2 ] 36188 15.830% 16.187% ### ( 2, 3 ] 83243 36.414% 52.600% ####### ( 3, 4 ] 29408 12.864% 65.465% ### ( 4, 6 ] 46829 20.485% 85.949% #### ( 6, 10 ] 24925 10.903% 96.853% ## ( 10, 15 ] 5088 2.226% 99.078% ( 15, 22 ] 1086 0.475% 99.553% ( 22, 34 ] 193 0.084% 99.638% ( 34, 51 ] 28 0.012% 99.650% ( 51, 76 ] 37 0.016% 99.666% ( 76, 110 ] 16 0.007% 99.673% ( 110, 170 ] 13 0.006% 99.679% ( 170, 250 ] 15 0.007% 99.685% ( 250, 380 ] 35 0.015% 99.701% ( 380, 580 ] 80 0.035% 99.736% ( 580, 870 ] 151 0.066% 99.802% ( 870, 1300 ] 157 0.069% 99.871% ( 1300, 1900 ] 117 0.051% 99.922% ( 1900, 2900 ] 110 0.048% 99.970% ( 2900, 4400 ] 117 0.051% 100.021% ( 4400, 6600 ] 55 0.024% 100.045% ( 6600, 9900 ] 39 0.017% 100.062% ( 9900, 14000 ] 27 0.012% 100.074% ( 14000, 22000 ] 57 0.025% 100.099% ( 22000, 33000 ] 11 0.005% 100.104% ( 33000, 50000 ] 2 0.001% 100.105% ** Level 3 read latency histogram (micros): Count: 621375 Average: 8.0217 StdDev: 231.70 Min: 1 Median: 2.6353 Max: 38077 Percentiles: P50: 2.64 P75: 3.94 P99: 9.88 P99.9: 14.30 P99.99: 14.86 ------------------------------------------------------ [ 0, 1 ] 7351 1.183% 1.183% ( 1, 2 ] 146252 23.537% 24.720% ##### ( 2, 3 ] 247269 39.794% 64.514% ######## ( 3, 4 ] 69410 11.170% 75.684% ## ( 4, 6 ] 102680 16.525% 92.209% ### ( 6, 10 ] 43541 7.007% 99.216% # ( 10, 15 ] 4948 0.796% 100.012% ( 15, 22 ] 575 0.093% 100.105% ( 22, 34 ] 81 0.013% 100.118% ( 34, 51 ] 52 0.008% 100.126% ( 51, 76 ] 31 0.005% 100.131% ( 76, 110 ] 34 0.005% 100.137% ( 110, 170 ] 79 0.013% 100.149% ( 170, 250 ] 114 0.018% 100.168% ( 250, 380 ] 87 0.014% 100.182% ( 380, 580 ] 92 0.015% 100.196% ( 580, 870 ] 126 0.020% 100.217% ( 870, 1300 ] 129 0.021% 100.238% ( 1300, 1900 ] 85 0.014% 100.251% ( 1900, 2900 ] 54 0.009% 100.260% ( 2900, 4400 ] 130 0.021% 100.281% ( 4400, 6600 ] 9 0.001% 100.282% ( 6600, 9900 ] 17 0.003% 100.285% ( 9900, 14000 ] 14 0.002% 100.287% ( 14000, 22000 ] 42 0.007% 100.294% ( 22000, 33000 ] 20 0.003% 100.297% ( 33000, 50000 ] 3 0.000% 100.298% ** Level 4 read latency histogram (micros): Count: 80656 Average: 129.3312 StdDev: 1143.16 Min: 1 Median: 2.6029 Max: 61483 Percentiles: P50: 2.60 P75: 3.97 P99: 2784.75 P99.9: 17031.27 P99.99: 26383.96 ------------------------------------------------------ [ 0, 1 ] 1387 1.720% 1.720% ( 1, 2 ] 22134 27.442% 29.162% ##### ( 2, 3 ] 27878 34.564% 63.726% ####### ( 3, 4 ] 9327 11.564% 75.290% ## ( 4, 6 ] 10544 13.073% 88.363% ### ( 6, 10 ] 3138 3.891% 92.254% # ( 10, 15 ] 759 0.941% 93.195% ( 15, 22 ] 136 0.169% 93.363% ( 22, 34 ] 23 0.029% 93.392% ( 34, 51 ] 4 0.005% 93.397% ( 51, 76 ] 5 0.006% 93.403% ( 76, 110 ] 43 0.053% 93.456% ( 110, 170 ] 576 0.714% 94.170% ( 170, 250 ] 618 0.766% 94.937% ( 250, 380 ] 416 0.516% 95.452% ( 380, 580 ] 513 0.636% 96.088% ( 580, 870 ] 674 0.836% 96.924% ( 870, 1300 ] 717 0.889% 97.813% ( 1300, 1900 ] 600 0.744% 98.557% ( 1900, 2900 ] 404 0.501% 99.058% ( 2900, 4400 ] 371 0.460% 99.518% ( 4400, 6600 ] 179 0.222% 99.740% ( 6600, 9900 ] 52 0.064% 99.804% ( 9900, 14000 ] 44 0.055% 99.859% ( 14000, 22000 ] 88 0.109% 99.968% ( 22000, 33000 ] 45 0.056% 100.024% ( 33000, 50000 ] 11 0.014% 100.037% ( 50000, 75000 ] 2 0.002% 100.040% ** Level 5 read latency histogram (micros): Count: 23011 Average: 1105.6045 StdDev: 2840.14 Min: 88 Median: 361.6791 Max: 74504 Percentiles: P50: 361.68 P75: 1082.22 P99: 9041.76 P99.9: 43067.15 P99.99: 68651.73 ------------------------------------------------------ ( 76, 110 ] 241 1.047% 1.047% ( 110, 170 ] 4387 19.065% 20.112% #### ( 170, 250 ] 4809 20.899% 41.011% #### ( 250, 380 ] 2409 10.469% 51.480% ## ( 380, 580 ] 2381 10.347% 61.827% ## ( 580, 870 ] 2060 8.952% 70.779% ## ( 870, 1300 ] 1970 8.561% 79.340% ## ( 1300, 1900 ] 1421 6.175% 85.516% # ( 1900, 2900 ] 1240 5.389% 90.904% # ( 2900, 4400 ] 1258 5.467% 96.371% # ( 4400, 6600 ] 506 2.199% 98.570% ( 6600, 9900 ] 135 0.587% 99.157% ( 9900, 14000 ] 23 0.100% 99.257% ( 14000, 22000 ] 65 0.282% 99.539% ( 22000, 33000 ] 68 0.296% 99.835% ( 33000, 50000 ] 27 0.117% 99.952% ( 50000, 75000 ] 13 0.056% 100.009% ** DB Stats ** Uptime(secs): 9.4 total, 9.1 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 104.33 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.660 H:M:S, 28.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 107.79 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.660 H:M:S, 29.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 2024/07/12-00:31:18 ... thread 0: (1000000,2000000) ops and (216409.8,145482.6) ops/second in (4.620863,13.747351) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/9 3.49 GB 157.4 1.3 0.0 1.3 3.1 1.8 0.0 1.7 73.5 174.9 18.00 13.44 32 0.562 1343K 1326 0.0 0.0 L1 13/13 1.61 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 937.87 MB 0.9 2.4 1.6 0.9 2.4 1.5 0.0 1.5 103.9 103.0 23.94 17.18 10 2.394 503K 0 0.0 0.0 L4 199/0 9.71 GB 1.0 2.3 2.0 0.3 2.3 2.0 0.0 1.1 75.8 75.5 31.58 19.04 26 1.215 2044K 1477 0.0 0.0 L5 1452/0 89.32 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1697/22 105.05 GB 0.0 6.1 3.6 2.5 7.8 5.4 2.0 4.4 84.4 108.8 73.52 49.65 68 1.081 3890K 2803 0.0 0.0 Int 0/0 0.00 KB 0.0 6.1 3.6 2.5 7.8 5.4 2.0 4.4 84.4 108.8 73.52 49.65 68 1.081 3890K 2803 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.1 3.6 2.5 6.0 3.6 0.0 0.0 98.2 97.7 63.16 43.37 41 1.541 3890K 2803 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 176.3 10.36 6.28 27 0.384 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 14.1 total, 13.7 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.81 GB write, 569.24 MB/s write, 6.06 GB read, 441.49 MB/s read, 73.5 seconds Interval compaction: 7.81 GB write, 581.84 MB/s write, 6.06 GB read, 451.26 MB/s read, 73.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 1, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 1, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 20, pending-compaction-bytes-delays: 8, pending-compaction-bytes-stops: 0, total-delays: 9, total-stops: 20, interval: 29 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 640614 Average: 16.5821 StdDev: 240.31 Min: 1 Median: 2.5491 Max: 39765 Percentiles: P50: 2.55 P75: 3.77 P99: 248.93 P99.9: 790.54 P99.99: 1291.35 ------------------------------------------------------ [ 0, 1 ] 16408 2.561% 2.561% # ( 1, 2 ] 187999 29.347% 31.908% ###### ( 2, 3 ] 211068 32.948% 64.856% ####### ( 3, 4 ] 83965 13.107% 77.963% ### ( 4, 6 ] 79987 12.486% 90.449% ## ( 6, 10 ] 32069 5.006% 95.455% # ( 10, 15 ] 5706 0.891% 96.345% ( 15, 22 ] 1080 0.169% 96.514% ( 22, 34 ] 177 0.028% 96.542% ( 34, 51 ] 43 0.007% 96.548% ( 51, 76 ] 25 0.004% 96.552% ( 76, 110 ] 303 0.047% 96.600% ( 110, 170 ] 7068 1.103% 97.703% ( 170, 250 ] 8423 1.315% 99.018% ( 250, 380 ] 3255 0.508% 99.526% ( 380, 580 ] 1781 0.278% 99.804% ( 580, 870 ] 849 0.133% 99.936% ( 870, 1300 ] 351 0.055% 99.991% ( 1300, 1900 ] 189 0.030% 100.021% ( 1900, 2900 ] 138 0.022% 100.042% ( 2900, 4400 ] 130 0.020% 100.062% ( 4400, 6600 ] 45 0.007% 100.069% ( 6600, 9900 ] 28 0.004% 100.074% ( 9900, 14000 ] 19 0.003% 100.077% ( 14000, 22000 ] 42 0.007% 100.083% ( 22000, 33000 ] 12 0.002% 100.085% ( 33000, 50000 ] 5 0.001% 100.086% ** Level 1 read latency histogram (micros): Count: 17882 Average: 46.0736 StdDev: 426.84 Min: 1 Median: 3.4603 Max: 21940 Percentiles: P50: 3.46 P75: 5.29 P99: 553.72 P99.9: 4253.00 P99.99: 15938.88 ------------------------------------------------------ [ 0, 1 ] 53 0.296% 0.296% ( 1, 2 ] 1963 10.978% 11.274% ## ( 2, 3 ] 5249 29.354% 40.627% ###### ( 3, 4 ] 3641 20.361% 60.989% #### ( 4, 6 ] 3889 21.748% 82.737% #### ( 6, 10 ] 849 4.748% 87.485% # ( 10, 15 ] 260 1.454% 88.939% ( 15, 22 ] 105 0.587% 89.526% ( 22, 34 ] 24 0.134% 89.660% ( 34, 51 ] 1 0.006% 89.666% ( 76, 110 ] 28 0.157% 89.822% ( 110, 170 ] 611 3.417% 93.239% # ( 170, 250 ] 699 3.909% 97.148% # ( 250, 380 ] 220 1.230% 98.378% ( 380, 580 ] 128 0.716% 99.094% ( 580, 870 ] 71 0.397% 99.491% ( 870, 1300 ] 27 0.151% 99.642% ( 1300, 1900 ] 20 0.112% 99.754% ( 1900, 2900 ] 18 0.101% 99.855% ( 2900, 4400 ] 9 0.050% 99.905% ( 4400, 6600 ] 3 0.017% 99.922% ( 6600, 9900 ] 5 0.028% 99.950% ( 9900, 14000 ] 6 0.034% 99.983% ( 14000, 22000 ] 5 0.028% 100.011% ** Level 2 read latency histogram (micros): Count: 228603 Average: 19.1489 StdDev: 395.32 Min: 1 Median: 2.9286 Max: 40782 Percentiles: P50: 2.93 P75: 4.93 P99: 14.82 P99.9: 1645.63 P99.99: 3491.53 ------------------------------------------------------ [ 0, 1 ] 815 0.357% 0.357% ( 1, 2 ] 36188 15.830% 16.187% ### ( 2, 3 ] 83243 36.414% 52.600% ####### ( 3, 4 ] 29408 12.864% 65.465% ### ( 4, 6 ] 46829 20.485% 85.949% #### ( 6, 10 ] 24925 10.903% 96.853% ## ( 10, 15 ] 5088 2.226% 99.078% ( 15, 22 ] 1086 0.475% 99.553% ( 22, 34 ] 193 0.084% 99.638% ( 34, 51 ] 28 0.012% 99.650% ( 51, 76 ] 37 0.016% 99.666% ( 76, 110 ] 16 0.007% 99.673% ( 110, 170 ] 13 0.006% 99.679% ( 170, 250 ] 15 0.007% 99.685% ( 250, 380 ] 35 0.015% 99.701% ( 380, 580 ] 80 0.035% 99.736% ( 580, 870 ] 151 0.066% 99.802% ( 870, 1300 ] 157 0.069% 99.871% ( 1300, 1900 ] 117 0.051% 99.922% ( 1900, 2900 ] 110 0.048% 99.970% ( 2900, 4400 ] 117 0.051% 100.021% ( 4400, 6600 ] 55 0.024% 100.045% ( 6600, 9900 ] 39 0.017% 100.062% ( 9900, 14000 ] 27 0.012% 100.074% ( 14000, 22000 ] 57 0.025% 100.099% ( 22000, 33000 ] 11 0.005% 100.104% ( 33000, 50000 ] 2 0.001% 100.105% ** Level 3 read latency histogram (micros): Count: 622170 Average: 8.2737 StdDev: 231.74 Min: 1 Median: 2.6368 Max: 38077 Percentiles: P50: 2.64 P75: 3.95 P99: 9.94 P99.9: 14.99 P99.99: 21.74 ------------------------------------------------------ [ 0, 1 ] 7351 1.182% 1.182% ( 1, 2 ] 146259 23.508% 24.689% ##### ( 2, 3 ] 247301 39.748% 64.438% ######## ( 3, 4 ] 69432 11.160% 75.597% ## ( 4, 6 ] 102718 16.510% 92.107% ### ( 6, 10 ] 43544 6.999% 99.106% # ( 10, 15 ] 4949 0.795% 99.901% ( 15, 22 ] 575 0.092% 99.993% ( 22, 34 ] 81 0.013% 100.006% ( 34, 51 ] 52 0.008% 100.015% ( 51, 76 ] 31 0.005% 100.020% ( 76, 110 ] 45 0.007% 100.027% ( 110, 170 ] 337 0.054% 100.081% ( 170, 250 ] 364 0.059% 100.140% ( 250, 380 ] 182 0.029% 100.169% ( 380, 580 ] 147 0.024% 100.193% ( 580, 870 ] 146 0.023% 100.216% ( 870, 1300 ] 130 0.021% 100.237% ( 1300, 1900 ] 85 0.014% 100.251% ( 1900, 2900 ] 56 0.009% 100.260% ( 2900, 4400 ] 130 0.021% 100.280% ( 4400, 6600 ] 9 0.001% 100.282% ( 6600, 9900 ] 17 0.003% 100.285% ( 9900, 14000 ] 14 0.002% 100.287% ( 14000, 22000 ] 42 0.007% 100.294% ( 22000, 33000 ] 20 0.003% 100.297% ( 33000, 50000 ] 3 0.000% 100.297% ** Level 4 read latency histogram (micros): Count: 88317 Average: 135.0413 StdDev: 1093.52 Min: 1 Median: 2.7290 Max: 61483 Percentiles: P50: 2.73 P75: 4.89 P99: 2601.30 P99.9: 16334.82 P99.99: 26196.70 ------------------------------------------------------ [ 0, 1 ] 1387 1.570% 1.570% ( 1, 2 ] 22207 25.145% 26.715% ##### ( 2, 3 ] 28210 31.942% 58.657% ###### ( 3, 4 ] 9581 10.848% 69.505% ## ( 4, 6 ] 10922 12.367% 81.872% ## ( 6, 10 ] 3180 3.601% 85.473% # ( 10, 15 ] 771 0.873% 86.346% ( 15, 22 ] 138 0.156% 86.502% ( 22, 34 ] 24 0.027% 86.529% ( 34, 51 ] 4 0.005% 86.534% ( 51, 76 ] 5 0.006% 86.539% ( 76, 110 ] 176 0.199% 86.739% ( 110, 170 ] 2955 3.346% 90.085% # ( 170, 250 ] 3191 3.613% 93.698% # ( 250, 380 ] 1249 1.414% 95.112% ( 380, 580 ] 976 1.105% 96.217% ( 580, 870 ] 837 0.948% 97.165% ( 870, 1300 ] 733 0.830% 97.995% ( 1300, 1900 ] 601 0.681% 98.675% ( 1900, 2900 ] 409 0.463% 99.138% ( 2900, 4400 ] 372 0.421% 99.560% ( 4400, 6600 ] 179 0.203% 99.762% ( 6600, 9900 ] 52 0.059% 99.821% ( 9900, 14000 ] 44 0.050% 99.871% ( 14000, 22000 ] 88 0.100% 99.971% ( 22000, 33000 ] 45 0.051% 100.022% ( 33000, 50000 ] 11 0.012% 100.034% ( 50000, 75000 ] 2 0.002% 100.036% ** Level 5 read latency histogram (micros): Count: 77017 Average: 488.0662 StdDev: 1607.90 Min: 86 Median: 209.8421 Max: 74504 Percentiles: P50: 209.84 P75: 338.77 P99: 4686.22 P99.9: 26850.19 P99.99: 58265.96 ------------------------------------------------------ ( 76, 110 ] 1216 1.579% 1.579% ( 110, 170 ] 24551 31.877% 33.456% ###### ( 170, 250 ] 25584 33.219% 66.675% ####### ( 250, 380 ] 9390 12.192% 78.867% ## ( 380, 580 ] 5986 7.772% 86.639% ## ( 580, 870 ] 3368 4.373% 91.012% # ( 870, 1300 ] 2124 2.758% 93.770% # ( 1300, 1900 ] 1443 1.874% 95.644% ( 1900, 2900 ] 1256 1.631% 97.275% ( 2900, 4400 ] 1263 1.640% 98.915% ( 4400, 6600 ] 506 0.657% 99.572% ( 6600, 9900 ] 135 0.175% 99.747% ( 9900, 14000 ] 23 0.030% 99.777% ( 14000, 22000 ] 65 0.084% 99.861% ( 22000, 33000 ] 68 0.088% 99.949% ( 33000, 50000 ] 27 0.035% 99.984% ( 50000, 75000 ] 13 0.017% 100.001% ** DB Stats ** Uptime(secs): 14.1 total, 4.6 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 140.06 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.729 H:M:S, 19.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 213.01 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.069 H:M:S, 1.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/12-00:32:01 ... thread 0: (1000000,3000000) ops and (22813.0,52099.5) ops/second in (43.834724,57.582075) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/8 2.69 GB 6.3 3.8 0.0 3.8 6.5 2.7 0.0 2.4 110.5 188.5 35.34 29.61 52 0.680 3964K 13K 0.0 0.0 L1 22/22 2.45 GB 0.0 3.3 1.7 1.6 3.3 1.7 0.0 1.9 62.9 62.2 53.98 26.77 1 53.981 3449K 36K 0.0 0.0 L3 24/2 1.18 GB 1.0 3.5 2.0 1.5 3.5 1.9 0.4 1.8 114.0 112.9 31.56 24.07 15 2.104 1107K 3102 0.0 0.0 L4 198/4 9.76 GB 1.0 3.1 2.6 0.5 3.1 2.6 0.0 1.2 84.4 83.8 37.45 23.74 33 1.135 2616K 1477 0.0 0.0 L5 1460/0 89.81 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1713/36 105.90 GB 0.0 13.7 6.3 7.4 16.3 8.9 2.9 6.0 88.8 105.6 158.33 104.19 101 1.568 11M 54K 0.0 0.0 Int 0/0 0.00 KB 0.0 13.7 6.3 7.4 16.3 8.9 2.9 6.0 88.8 105.6 158.33 104.19 101 1.568 11M 54K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.7 6.3 7.4 13.6 6.2 0.0 0.0 97.5 96.8 144.14 95.01 59 2.443 11M 54K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.7 2.7 0.0 0.0 0.0 195.3 14.19 9.19 42 0.338 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 57.9 total, 57.6 interval Flush(GB): cumulative 2.707, interval 2.707 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 16.33 GB write, 288.93 MB/s write, 13.73 GB read, 242.82 MB/s read, 158.3 seconds Interval compaction: 16.33 GB write, 290.46 MB/s write, 13.73 GB read, 244.10 MB/s read, 158.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 4, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 4, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 20, pending-compaction-bytes-delays: 28, pending-compaction-bytes-stops: 0, total-delays: 32, total-stops: 20, interval: 52 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1584864 Average: 35.1279 StdDev: 190.74 Min: 1 Median: 2.4175 Max: 49465 Percentiles: P50: 2.42 P75: 4.22 P99: 346.78 P99.9: 809.51 P99.99: 1235.16 ------------------------------------------------------ [ 0, 1 ] 63278 3.993% 3.993% # ( 1, 2 ] 559248 35.287% 39.279% ####### ( 2, 3 ] 406981 25.679% 64.959% ##### ( 3, 4 ] 145809 9.200% 74.159% ## ( 4, 6 ] 121223 7.649% 81.808% ## ( 6, 10 ] 39039 2.463% 84.271% ( 10, 15 ] 7179 0.453% 84.724% ( 15, 22 ] 1345 0.085% 84.809% ( 22, 34 ] 221 0.014% 84.823% ( 34, 51 ] 48 0.003% 84.826% ( 51, 76 ] 28 0.002% 84.827% ( 76, 110 ] 4204 0.265% 85.093% ( 110, 170 ] 109659 6.919% 92.012% # ( 170, 250 ] 98578 6.220% 98.232% # ( 250, 380 ] 16357 1.032% 99.264% ( 380, 580 ] 7717 0.487% 99.751% ( 580, 870 ] 2991 0.189% 99.939% ( 870, 1300 ] 945 0.060% 99.999% ( 1300, 1900 ] 346 0.022% 100.021% ( 1900, 2900 ] 250 0.016% 100.037% ( 2900, 4400 ] 211 0.013% 100.050% ( 4400, 6600 ] 76 0.005% 100.055% ( 6600, 9900 ] 33 0.002% 100.057% ( 9900, 14000 ] 23 0.001% 100.058% ( 14000, 22000 ] 42 0.003% 100.061% ( 22000, 33000 ] 14 0.001% 100.062% ( 33000, 50000 ] 9 0.001% 100.062% ** Level 1 read latency histogram (micros): Count: 543330 Average: 10.9260 StdDev: 88.99 Min: 0 Median: 2.9795 Max: 21940 Percentiles: P50: 2.98 P75: 4.50 P99: 209.40 P99.9: 518.87 P99.99: 911.12 ------------------------------------------------------ [ 0, 1 ] 17590 3.237% 3.237% # ( 1, 2 ] 115773 21.308% 24.545% #### ( 2, 3 ] 141194 25.987% 50.532% ##### ( 3, 4 ] 102271 18.823% 69.355% #### ( 4, 6 ] 122916 22.623% 91.978% ##### ( 6, 10 ] 17237 3.172% 95.150% # ( 10, 15 ] 4800 0.883% 96.034% ( 15, 22 ] 3572 0.657% 96.691% ( 22, 34 ] 176 0.032% 96.724% ( 34, 51 ] 4 0.001% 96.724% ( 76, 110 ] 445 0.082% 96.806% ( 110, 170 ] 8628 1.588% 98.394% ( 170, 250 ] 6682 1.230% 99.624% ( 250, 380 ] 1064 0.196% 99.820% ( 380, 580 ] 626 0.115% 99.935% ( 580, 870 ] 286 0.053% 99.988% ( 870, 1300 ] 122 0.022% 100.010% ( 1300, 1900 ] 43 0.008% 100.018% ( 1900, 2900 ] 24 0.004% 100.023% ( 2900, 4400 ] 16 0.003% 100.026% ( 4400, 6600 ] 6 0.001% 100.027% ( 6600, 9900 ] 5 0.001% 100.028% ( 9900, 14000 ] 6 0.001% 100.029% ( 14000, 22000 ] 5 0.001% 100.030% ** Level 2 read latency histogram (micros): Count: 228603 Average: 19.1489 StdDev: 395.32 Min: 1 Median: 2.9286 Max: 40782 Percentiles: P50: 2.93 P75: 4.93 P99: 14.82 P99.9: 1645.63 P99.99: 3491.53 ------------------------------------------------------ [ 0, 1 ] 815 0.357% 0.357% ( 1, 2 ] 36188 15.830% 16.187% ### ( 2, 3 ] 83243 36.414% 52.600% ####### ( 3, 4 ] 29408 12.864% 65.465% ### ( 4, 6 ] 46829 20.485% 85.949% #### ( 6, 10 ] 24925 10.903% 96.853% ## ( 10, 15 ] 5088 2.226% 99.078% ( 15, 22 ] 1086 0.475% 99.553% ( 22, 34 ] 193 0.084% 99.638% ( 34, 51 ] 28 0.012% 99.650% ( 51, 76 ] 37 0.016% 99.666% ( 76, 110 ] 16 0.007% 99.673% ( 110, 170 ] 13 0.006% 99.679% ( 170, 250 ] 15 0.007% 99.685% ( 250, 380 ] 35 0.015% 99.701% ( 380, 580 ] 80 0.035% 99.736% ( 580, 870 ] 151 0.066% 99.802% ( 870, 1300 ] 157 0.069% 99.871% ( 1300, 1900 ] 117 0.051% 99.922% ( 1900, 2900 ] 110 0.048% 99.970% ( 2900, 4400 ] 117 0.051% 100.021% ( 4400, 6600 ] 55 0.024% 100.045% ( 6600, 9900 ] 39 0.017% 100.062% ( 9900, 14000 ] 27 0.012% 100.074% ( 14000, 22000 ] 57 0.025% 100.099% ( 22000, 33000 ] 11 0.005% 100.104% ( 33000, 50000 ] 2 0.001% 100.105% ** Level 3 read latency histogram (micros): Count: 971024 Average: 8.6607 StdDev: 187.75 Min: 1 Median: 2.6639 Max: 38077 Percentiles: P50: 2.66 P75: 4.04 P99: 15.42 P99.9: 213.26 P99.99: 231.71 ------------------------------------------------------ [ 0, 1 ] 8658 0.892% 0.892% ( 1, 2 ] 216276 22.273% 23.165% #### ( 2, 3 ] 392490 40.420% 63.585% ######## ( 3, 4 ] 107247 11.045% 74.630% ## ( 4, 6 ] 161083 16.589% 91.219% ### ( 6, 10 ] 66872 6.887% 98.105% # ( 10, 15 ] 8614 0.887% 98.992% ( 15, 22 ] 1237 0.127% 99.120% ( 22, 34 ] 134 0.014% 99.134% ( 34, 51 ] 80 0.008% 99.142% ( 51, 76 ] 71 0.007% 99.149% ( 76, 110 ] 278 0.029% 99.178% ( 110, 170 ] 4965 0.511% 99.689% ( 170, 250 ] 3789 0.390% 100.079% ( 250, 380 ] 714 0.074% 100.153% ( 380, 580 ] 461 0.047% 100.200% ( 580, 870 ] 335 0.034% 100.235% ( 870, 1300 ] 188 0.019% 100.254% ( 1300, 1900 ] 107 0.011% 100.265% ( 1900, 2900 ] 72 0.007% 100.273% ( 2900, 4400 ] 141 0.015% 100.287% ( 4400, 6600 ] 11 0.001% 100.288% ( 6600, 9900 ] 17 0.002% 100.290% ( 9900, 14000 ] 14 0.001% 100.291% ( 14000, 22000 ] 42 0.004% 100.296% ( 22000, 33000 ] 20 0.002% 100.298% ( 33000, 50000 ] 3 0.000% 100.298% ** Level 4 read latency histogram (micros): Count: 249360 Average: 118.6470 StdDev: 718.30 Min: 1 Median: 4.8821 Max: 61483 Percentiles: P50: 4.88 P75: 152.26 P99: 1024.47 P99.9: 5126.30 P99.99: 11272.13 ------------------------------------------------------ [ 0, 1 ] 1846 0.740% 0.740% ( 1, 2 ] 37653 15.100% 15.840% ### ( 2, 3 ] 55598 22.296% 38.136% #### ( 3, 4 ] 18422 7.388% 45.524% # ( 4, 6 ] 25305 10.148% 55.672% ## ( 6, 10 ] 9551 3.830% 59.502% # ( 10, 15 ] 2163 0.867% 60.370% ( 15, 22 ] 350 0.140% 60.510% ( 22, 34 ] 66 0.026% 60.537% ( 34, 51 ] 10 0.004% 60.541% ( 51, 76 ] 21 0.008% 60.549% ( 76, 110 ] 2538 1.018% 61.567% ( 110, 170 ] 47555 19.071% 80.638% #### ( 170, 250 ] 34895 13.994% 94.631% ### ( 250, 380 ] 5907 2.369% 97.000% ( 380, 580 ] 3038 1.218% 98.219% ( 580, 870 ] 1596 0.640% 98.859% ( 870, 1300 ] 981 0.393% 99.252% ( 1300, 1900 ] 663 0.266% 99.518% ( 1900, 2900 ] 445 0.178% 99.696% ( 2900, 4400 ] 436 0.175% 99.871% ( 4400, 6600 ] 217 0.087% 99.958% ( 6600, 9900 ] 63 0.025% 99.984% ( 9900, 14000 ] 48 0.019% 100.003% ( 14000, 22000 ] 96 0.038% 100.041% ( 22000, 33000 ] 56 0.022% 100.064% ( 33000, 50000 ] 13 0.005% 100.069% ( 50000, 75000 ] 3 0.001% 100.070% ** Level 5 read latency histogram (micros): Count: 683026 Average: 222.2513 StdDev: 576.43 Min: 83 Median: 167.7208 Max: 74504 Percentiles: P50: 167.72 P75: 220.49 P99: 1151.41 P99.9: 5402.67 P99.99: 28523.21 ------------------------------------------------------ ( 76, 110 ] 16875 2.471% 2.471% ( 110, 170 ] 337457 49.406% 51.877% ########## ( 170, 250 ] 250227 36.635% 88.512% ####### ( 250, 380 ] 40895 5.987% 94.499% # ( 380, 580 ] 20307 2.973% 97.472% # ( 580, 870 ] 8192 1.199% 98.672% ( 870, 1300 ] 3427 0.502% 99.173% ( 1300, 1900 ] 1854 0.271% 99.445% ( 1900, 2900 ] 1438 0.211% 99.655% ( 2900, 4400 ] 1408 0.206% 99.861% ( 4400, 6600 ] 577 0.084% 99.946% ( 6600, 9900 ] 152 0.022% 99.968% ( 9900, 14000 ] 31 0.005% 99.973% ( 14000, 22000 ] 75 0.011% 99.984% ( 22000, 33000 ] 72 0.011% 99.994% ( 33000, 50000 ] 29 0.004% 99.999% ( 50000, 75000 ] 13 0.002% 100.000% ** DB Stats ** Uptime(secs): 57.9 total, 43.8 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 51.00 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:41.092 H:M:S, 71.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 22.45 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:38.364 H:M:S, 87.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 280.363 micros/op 14259 ops/sec 60.170 seconds 857996 operations; 13.8 MB/s (212999 of 212999 found) Microseconds per read: Count: 857996 Average: 280.3646 StdDev: 1308.04 Min: 2 Median: 212.0870 Max: 239666 Percentiles: P50: 212.09 P75: 252.51 P99: 929.32 P99.9: 13244.97 P99.99: 32473.27 ------------------------------------------------------ ( 1, 2 ] 3 0.000% 0.000% ( 2, 3 ] 37 0.004% 0.005% ( 3, 4 ] 56 0.007% 0.011% ( 4, 6 ] 75 0.009% 0.020% ( 6, 10 ] 229 0.027% 0.047% ( 10, 15 ] 2858 0.333% 0.380% ( 15, 22 ] 2494 0.291% 0.670% ( 22, 34 ] 5493 0.640% 1.311% ( 34, 51 ] 1887 0.220% 1.531% ( 51, 76 ] 210 0.024% 1.555% ( 76, 110 ] 532 0.062% 1.617% ( 110, 170 ] 179721 20.947% 22.564% #### ( 170, 250 ] 447460 52.152% 74.715% ########## ( 250, 380 ] 126303 14.721% 89.436% ### ( 380, 580 ] 62798 7.319% 96.755% # ( 580, 870 ] 18587 2.166% 98.922% ( 870, 1300 ] 4879 0.569% 99.490% ( 1300, 1900 ] 1257 0.147% 99.637% ( 1900, 2900 ] 652 0.076% 99.713% ( 2900, 4400 ] 517 0.060% 99.773% ( 4400, 6600 ] 302 0.035% 99.808% ( 6600, 9900 ] 345 0.040% 99.848% ( 9900, 14000 ] 543 0.063% 99.912% ( 14000, 22000 ] 537 0.063% 99.974% ( 22000, 33000 ] 142 0.017% 99.991% ( 33000, 50000 ] 33 0.004% 99.995% ( 50000, 75000 ] 10 0.001% 99.996% ( 75000, 110000 ] 4 0.000% 99.996% ( 110000, 170000 ] 16 0.002% 99.998% ( 170000, 250000 ] 16 0.002% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 12488012 rocksdb.bloom.filter.full.positive COUNT : 979318 rocksdb.bloom.filter.full.true.positive COUNT : 857637 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 359 rocksdb.memtable.miss COUNT : 857637 rocksdb.l0.hit COUNT : 35008 rocksdb.l1.hit COUNT : 14548 rocksdb.l2andup.hit COUNT : 808081 rocksdb.compaction.key.drop.new COUNT : 106876 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3446736 rocksdb.number.keys.read COUNT : 857996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3557031552 rocksdb.bytes.read COUNT : 857996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1905 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 41497332 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 15652424988 rocksdb.compact.write.bytes COUNT : 15531743039 rocksdb.flush.write.bytes COUNT : 3501128489 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 25 rocksdb.number.superversion_acquires COUNT : 742 rocksdb.number.superversion_releases COUNT : 710 rocksdb.number.superversion_cleanups COUNT : 183 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 151 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3472826368 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1294336 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 19101185737 rocksdb.non.last.level.read.count COUNT : 4473346 rocksdb.block.checksum.compute.count COUNT : 4778801 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 211.431510 P95 : 531.183234 P99 : 927.325077 P100 : 239663.000000 COUNT : 857996 SUM : 239835140 rocksdb.db.write.micros P50 : 2.247106 P95 : 6.219068 P99 : 15.260995 P100 : 555392.000000 COUNT : 3446736 SUM : 53883626 rocksdb.compaction.times.micros P50 : 1443478.260870 P95 : 3915000.000000 P99 : 53981267.000000 P100 : 53981267.000000 COUNT : 62 SUM : 148861276 rocksdb.compaction.times.cpu_micros P50 : 894000.000000 P95 : 2883333.333333 P99 : 26180000.000000 P100 : 26774555.000000 COUNT : 62 SUM : 99002070 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 1128.000000 P95 : 8250.000000 P99 : 27500.000000 P100 : 29722.000000 COUNT : 50 SUM : 112626 rocksdb.compaction.outfile.sync.micros P50 : 2975.000000 P95 : 20400.000000 P99 : 34700.000000 P100 : 48867.000000 COUNT : 180 SUM : 990830 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 309.090909 P95 : 5683.333333 P99 : 9570.000000 P100 : 10258.000000 COUNT : 130 SUM : 159238 rocksdb.table.open.io.micros P50 : 8360.193322 P95 : 20928.176796 P99 : 116157.894737 P100 : 173422.000000 COUNT : 1905 SUM : 20311558 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 5.385351 P95 : 17.078625 P99 : 239.636630 P100 : 51884.000000 COUNT : 3789121 SUM : 74079254 rocksdb.read.block.get.micros P50 : 171.491410 P95 : 379.039476 P99 : 852.578904 P100 : 74622.000000 COUNT : 988039 SUM : 205843540 rocksdb.write.raw.block.micros P50 : 0.605570 P95 : 1.855661 P99 : 4.169928 P100 : 91439.000000 COUNT : 4610904 SUM : 22522668 rocksdb.numfiles.in.singlecompaction P50 : 2.107143 P95 : 3.990385 P99 : 23.800000 P100 : 30.000000 COUNT : 85 SUM : 286 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.570973 P95 : 1983.880258 P99 : 2722.012568 P100 : 555241.000000 COUNT : 211438 SUM : 41514941 rocksdb.sst.read.micros P50 : 2.964183 P95 : 233.217114 P99 : 498.604720 P100 : 74504.000000 COUNT : 4473786 SUM : 267216294 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 857996 SUM : 857996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3446736 SUM : 3557031552 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 244074.074074 P95 : 831000.000000 P99 : 1300666.000000 P100 : 1300666.000000 COUNT : 50 SUM : 16535968 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 187 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0