[2024-11-05 18:59:33.186952] Starting SPDK v25.01-pre git sha1 8053cd6b8 / DPDK 24.03.0 initialization... [2024-11-05 18:59:33.187068] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2109522 ] Set seed to 1730829573184560 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-05 18:59:33.301665] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-05 18:59:33.401041] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-05 18:59:33.401088] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2109522' to capture a snapshot of events at runtime. [2024-11-05 18:59:33.401099] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-05 18:59:33.401125] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-05 18:59:33.401133] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2109522 for offline analysis/debug. [2024-11-05 18:59:33.402720] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Tue Nov 5 18:59:39 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/05-18:59:44 ... thread 0: (1000000,1000000) ops and (204618.3,204618.3) ops/second in (4.887149,4.887149) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/17 3.05 GB 6.3 0.7 0.0 0.7 1.6 0.9 0.0 1.9 138.7 299.9 5.44 5.00 17 0.320 766K 3647 0.0 0.0 L1 16/16 1.58 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 13/0 812.49 MB 0.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 122/0 9.53 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 120/0 9.84 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 289/33 24.79 GB 0.0 0.7 0.0 0.7 1.6 0.9 0.0 1.9 138.7 299.9 5.44 5.00 17 0.320 766K 3647 0.0 0.0 Int 0/0 0.00 KB 0.0 0.7 0.0 0.7 1.6 0.9 0.0 1.9 138.7 299.9 5.44 5.00 17 0.320 766K 3647 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.7 0.0 0.7 0.7 -0.0 0.0 0.0 276.8 275.4 2.73 2.68 3 0.909 766K 3647 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 324.5 2.71 2.32 14 0.194 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 5.0 total, 4.9 interval Flush(GB): cumulative 0.860, interval 0.860 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.59 GB write, 326.12 MB/s write, 0.74 GB read, 150.86 MB/s read, 5.4 seconds Interval compaction: 1.59 GB write, 333.71 MB/s write, 0.74 GB read, 154.37 MB/s read, 5.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 3, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 3, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 0, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 3, total-stops: 0, interval: 3 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 254241 Average: 32.6348 StdDev: 145.07 Min: 0 Median: 1.4780 Max: 17491 Percentiles: P50: 1.48 P75: 1.95 P99: 359.95 P99.9: 669.22 P99.99: 1144.47 ------------------------------------------------------ [ 0, 1 ] 62939 24.756% 24.756% ##### ( 1, 2 ] 134272 52.813% 77.569% ########### ( 2, 3 ] 14298 5.624% 83.192% # ( 3, 4 ] 3976 1.564% 84.756% ( 4, 6 ] 2829 1.113% 85.869% ( 6, 10 ] 975 0.383% 86.252% ( 10, 15 ] 155 0.061% 86.313% ( 15, 22 ] 26 0.010% 86.324% ( 22, 34 ] 3 0.001% 86.325% ( 34, 51 ] 4 0.002% 86.326% ( 51, 76 ] 196 0.077% 86.403% ( 76, 110 ] 1402 0.551% 86.955% ( 110, 170 ] 10546 4.148% 91.103% # ( 170, 250 ] 15189 5.974% 97.077% # ( 250, 380 ] 5781 2.274% 99.351% ( 380, 580 ] 1326 0.522% 99.873% ( 580, 870 ] 230 0.090% 99.963% ( 870, 1300 ] 109 0.043% 100.006% ( 1300, 1900 ] 83 0.033% 100.039% ( 1900, 2900 ] 46 0.018% 100.057% ( 2900, 4400 ] 7 0.003% 100.059% ( 4400, 6600 ] 3 0.001% 100.061% ( 6600, 9900 ] 1 0.000% 100.061% ( 9900, 14000 ] 11 0.004% 100.065% ( 14000, 22000 ] 5 0.002% 100.067% ** Level 1 read latency histogram (micros): Count: 5004 Average: 234.2792 StdDev: 396.63 Min: 48 Median: 199.2113 Max: 15204 Percentiles: P50: 199.21 P75: 242.82 P99: 988.49 P99.9: 3899.00 P99.99: 15204.00 ------------------------------------------------------ ( 34, 51 ] 1 0.020% 0.020% ( 51, 76 ] 9 0.180% 0.200% ( 76, 110 ] 81 1.619% 1.819% ( 110, 170 ] 1573 31.435% 33.253% ###### ( 170, 250 ] 2295 45.863% 79.117% ######### ( 250, 380 ] 818 16.347% 95.464% ### ( 380, 580 ] 148 2.958% 98.421% # ( 580, 870 ] 24 0.480% 98.901% ( 870, 1300 ] 18 0.360% 99.261% ( 1300, 1900 ] 15 0.300% 99.560% ( 1900, 2900 ] 13 0.260% 99.820% ( 2900, 4400 ] 6 0.120% 99.940% ( 9900, 14000 ] 1 0.020% 99.960% ( 14000, 22000 ] 2 0.040% 100.000% ** Level 3 read latency histogram (micros): Count: 2625 Average: 229.0019 StdDev: 209.55 Min: 52 Median: 197.6281 Max: 3849 Percentiles: P50: 197.63 P75: 243.24 P99: 964.06 P99.9: 3415.63 P99.99: 3849.00 ------------------------------------------------------ ( 51, 76 ] 9 0.343% 0.343% ( 76, 110 ] 39 1.486% 1.829% ( 110, 170 ] 867 33.029% 34.857% ####### ( 170, 250 ] 1151 43.848% 78.705% ######### ( 250, 380 ] 415 15.810% 94.514% ### ( 380, 580 ] 93 3.543% 98.057% # ( 580, 870 ] 23 0.876% 98.933% ( 870, 1300 ] 8 0.305% 99.238% ( 1300, 1900 ] 10 0.381% 99.619% ( 1900, 2900 ] 6 0.229% 99.848% ( 2900, 4400 ] 4 0.152% 100.000% ** Level 4 read latency histogram (micros): Count: 25020 Average: 229.8217 StdDev: 205.00 Min: 45 Median: 199.7304 Max: 4572 Percentiles: P50: 199.73 P75: 244.26 P99: 1062.79 P99.9: 3121.11 P99.99: 4372.11 ------------------------------------------------------ ( 34, 51 ] 2 0.008% 0.008% ( 51, 76 ] 42 0.168% 0.176% ( 76, 110 ] 523 2.090% 2.266% ( 110, 170 ] 7767 31.043% 33.309% ###### ( 170, 250 ] 11237 44.912% 78.221% ######### ( 250, 380 ] 4099 16.383% 94.604% ### ( 380, 580 ] 868 3.469% 98.074% # ( 580, 870 ] 191 0.763% 98.837% ( 870, 1300 ] 91 0.364% 99.201% ( 1300, 1900 ] 101 0.404% 99.604% ( 1900, 2900 ] 70 0.280% 99.884% ( 2900, 4400 ] 27 0.108% 99.992% ( 4400, 6600 ] 2 0.008% 100.000% ** Level 5 read latency histogram (micros): Count: 25612 Average: 227.5871 StdDev: 202.49 Min: 48 Median: 198.7927 Max: 5540 Percentiles: P50: 198.79 P75: 243.57 P99: 970.39 P99.9: 2986.75 P99.99: 4721.79 ------------------------------------------------------ ( 34, 51 ] 4 0.016% 0.016% ( 51, 76 ] 38 0.148% 0.164% ( 76, 110 ] 530 2.069% 2.233% ( 110, 170 ] 8117 31.692% 33.926% ###### ( 170, 250 ] 11439 44.663% 78.588% ######### ( 250, 380 ] 4149 16.199% 94.788% ### ( 380, 580 ] 903 3.526% 98.313% # ( 580, 870 ] 153 0.597% 98.911% ( 870, 1300 ] 98 0.383% 99.293% ( 1300, 1900 ] 87 0.340% 99.633% ( 1900, 2900 ] 67 0.262% 99.895% ( 2900, 4400 ] 24 0.094% 99.988% ( 4400, 6600 ] 3 0.012% 100.000% ** DB Stats ** Uptime(secs): 5.0 total, 4.9 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 196.65 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.387 H:M:S, 27.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 201.21 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.387 H:M:S, 28.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/05-18:59:53 ... thread 0: (1000000,2000000) ops and (110570.1,143562.8) ops/second in (9.044040,13.931189) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/17 4.00 GB 160.5 3.1 0.0 3.1 4.9 1.8 0.0 2.6 182.0 289.6 17.25 16.28 38 0.454 3187K 31K 0.0 0.0 L1 16/16 1.58 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 13/0 812.49 MB 0.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 122/0 9.53 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 120/0 9.84 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 290/33 25.75 GB 0.0 3.1 0.0 3.1 4.9 1.8 0.0 2.6 182.0 289.6 17.25 16.28 38 0.454 3187K 31K 0.0 0.0 Int 0/0 0.00 KB 0.0 3.1 0.0 3.1 4.9 1.8 0.0 2.6 182.0 289.6 17.25 16.28 38 0.454 3187K 31K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.1 0.0 3.1 3.0 -0.0 0.0 0.0 274.5 271.7 11.43 11.29 8 1.429 3187K 31K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 324.8 5.81 4.98 30 0.194 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 14.0 total, 13.9 interval Flush(GB): cumulative 1.843, interval 1.843 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 4.88 GB write, 355.53 MB/s write, 3.06 GB read, 223.39 MB/s read, 17.2 seconds Interval compaction: 4.88 GB write, 358.43 MB/s write, 3.06 GB read, 225.22 MB/s read, 17.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 16, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 16, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 0, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 16, total-stops: 0, interval: 16 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 931051 Average: 26.8980 StdDev: 96.09 Min: 0 Median: 1.5087 Max: 17491 Percentiles: P50: 1.51 P75: 1.96 P99: 336.07 P99.9: 524.27 P99.99: 572.41 ------------------------------------------------------ [ 0, 1 ] 205966 22.122% 22.122% #### ( 1, 2 ] 510199 54.798% 76.920% ########### ( 2, 3 ] 70399 7.561% 84.481% ## ( 3, 4 ] 17054 1.832% 86.313% ( 4, 6 ] 11817 1.269% 87.582% ( 6, 10 ] 3122 0.335% 87.918% ( 10, 15 ] 638 0.069% 87.986% ( 15, 22 ] 86 0.009% 87.995% ( 22, 34 ] 7 0.001% 87.996% ( 34, 51 ] 8 0.001% 87.997% ( 51, 76 ] 820 0.088% 88.085% ( 76, 110 ] 4991 0.536% 88.621% ( 110, 170 ] 36034 3.870% 92.491% # ( 170, 250 ] 49104 5.274% 97.765% # ( 250, 380 ] 17365 1.865% 99.630% ( 380, 580 ] 3481 0.374% 100.004% ( 580, 870 ] 380 0.041% 100.045% ( 870, 1300 ] 128 0.014% 100.059% ( 1300, 1900 ] 91 0.010% 100.069% ( 1900, 2900 ] 60 0.006% 100.075% ( 2900, 4400 ] 8 0.001% 100.076% ( 4400, 6600 ] 3 0.000% 100.076% ( 6600, 9900 ] 1 0.000% 100.076% ( 9900, 14000 ] 11 0.001% 100.078% ( 14000, 22000 ] 5 0.001% 100.078% ** Level 1 read latency histogram (micros): Count: 14918 Average: 214.3319 StdDev: 238.72 Min: 48 Median: 196.5668 Max: 15204 Percentiles: P50: 196.57 P75: 239.55 P99: 553.27 P99.9: 2477.29 P99.99: 15204.00 ------------------------------------------------------ ( 34, 51 ] 1 0.007% 0.007% ( 51, 76 ] 9 0.060% 0.067% ( 76, 110 ] 292 1.957% 2.024% ( 110, 170 ] 4852 32.524% 34.549% ####### ( 170, 250 ] 6941 46.528% 81.077% ######### ( 250, 380 ] 2355 15.786% 96.863% ### ( 380, 580 ] 368 2.467% 99.330% ( 580, 870 ] 39 0.261% 99.591% ( 870, 1300 ] 20 0.134% 99.725% ( 1300, 1900 ] 18 0.121% 99.846% ( 1900, 2900 ] 14 0.094% 99.940% ( 2900, 4400 ] 6 0.040% 99.980% ( 9900, 14000 ] 1 0.007% 99.987% ( 14000, 22000 ] 2 0.013% 100.000% ** Level 3 read latency histogram (micros): Count: 7707 Average: 212.7501 StdDev: 137.07 Min: 52 Median: 195.5101 Max: 3849 Percentiles: P50: 195.51 P75: 239.93 P99: 569.64 P99.9: 2282.17 P99.99: 3849.00 ------------------------------------------------------ ( 51, 76 ] 9 0.117% 0.117% ( 76, 110 ] 157 2.037% 2.154% ( 110, 170 ] 2581 33.489% 35.643% ####### ( 170, 250 ] 3470 45.024% 80.667% ######### ( 250, 380 ] 1192 15.466% 96.133% ### ( 380, 580 ] 233 3.023% 99.157% # ( 580, 870 ] 36 0.467% 99.624% ( 870, 1300 ] 8 0.104% 99.728% ( 1300, 1900 ] 11 0.143% 99.870% ( 1900, 2900 ] 6 0.078% 99.948% ( 2900, 4400 ] 4 0.052% 100.000% ** Level 4 read latency histogram (micros): Count: 74231 Average: 213.1661 StdDev: 134.66 Min: 45 Median: 196.3945 Max: 4572 Percentiles: P50: 196.39 P75: 240.13 P99: 564.66 P99.9: 2310.24 P99.99: 4109.47 ------------------------------------------------------ ( 34, 51 ] 2 0.003% 0.003% ( 51, 76 ] 42 0.057% 0.059% ( 76, 110 ] 1484 1.999% 2.058% ( 110, 170 ] 24389 32.856% 34.914% ####### ( 170, 250 ] 33945 45.729% 80.643% ######### ( 250, 380 ] 11590 15.613% 96.256% ### ( 380, 580 ] 2207 2.973% 99.229% # ( 580, 870 ] 264 0.356% 99.585% ( 870, 1300 ] 98 0.132% 99.717% ( 1300, 1900 ] 106 0.143% 99.860% ( 1900, 2900 ] 75 0.101% 99.961% ( 2900, 4400 ] 28 0.038% 99.999% ( 4400, 6600 ] 2 0.003% 100.001% ** Level 5 read latency histogram (micros): Count: 76110 Average: 212.3340 StdDev: 132.34 Min: 48 Median: 196.3060 Max: 5540 Percentiles: P50: 196.31 P75: 239.93 P99: 559.62 P99.9: 2222.39 P99.99: 4174.31 ------------------------------------------------------ ( 34, 51 ] 4 0.005% 0.005% ( 51, 76 ] 38 0.050% 0.055% ( 76, 110 ] 1439 1.891% 1.946% ( 110, 170 ] 25100 32.979% 34.924% ####### ( 170, 250 ] 34894 45.847% 80.771% ######### ( 250, 380 ] 11846 15.564% 96.336% ### ( 380, 580 ] 2258 2.967% 99.302% # ( 580, 870 ] 233 0.306% 99.608% ( 870, 1300 ] 108 0.142% 99.750% ( 1300, 1900 ] 91 0.120% 99.870% ( 1900, 2900 ] 71 0.093% 99.963% ( 2900, 4400 ] 24 0.032% 99.995% ( 4400, 6600 ] 3 0.004% 99.999% ** DB Stats ** Uptime(secs): 14.0 total, 9.0 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 140.11 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.748 H:M:S, 48.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 108.82 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.361 H:M:S, 59.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 265.138 micros/op 15020 ops/sec 20.172 seconds 302996 operations; 14.6 MB/s (74999 of 74999 found) Microseconds per read: Count: 302996 Average: 265.1452 StdDev: 340.00 Min: 2 Median: 229.9530 Max: 28706 Percentiles: P50: 229.95 P75: 321.19 P99: 802.79 P99.9: 3346.03 P99.99: 18357.98 ------------------------------------------------------ ( 1, 2 ] 22 0.007% 0.007% ( 2, 3 ] 84 0.028% 0.035% ( 3, 4 ] 118 0.039% 0.074% ( 4, 6 ] 211 0.070% 0.144% ( 6, 10 ] 362 0.119% 0.263% ( 10, 15 ] 3172 1.047% 1.310% ( 15, 22 ] 3566 1.177% 2.487% ( 22, 34 ] 5327 1.758% 4.245% ( 34, 51 ] 927 0.306% 4.551% ( 51, 76 ] 115 0.038% 4.589% ( 76, 110 ] 243 0.080% 4.669% ( 110, 170 ] 35855 11.833% 16.503% ## ( 170, 250 ] 135434 44.698% 61.201% ######### ( 250, 380 ] 76350 25.198% 86.399% ##### ( 380, 580 ] 32641 10.773% 97.172% ## ( 580, 870 ] 7210 2.380% 99.551% ( 870, 1300 ] 864 0.285% 99.837% ( 1300, 1900 ] 89 0.029% 99.866% ( 1900, 2900 ] 81 0.027% 99.893% ( 2900, 4400 ] 74 0.024% 99.917% ( 4400, 6600 ] 98 0.032% 99.950% ( 6600, 9900 ] 83 0.027% 99.977% ( 9900, 14000 ] 19 0.006% 99.983% ( 14000, 22000 ] 38 0.013% 99.996% ( 22000, 33000 ] 13 0.004% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 5743682 rocksdb.bloom.filter.full.positive COUNT : 357977 rocksdb.bloom.filter.full.true.positive COUNT : 302162 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 834 rocksdb.memtable.miss COUNT : 302162 rocksdb.l0.hit COUNT : 52785 rocksdb.l1.hit COUNT : 20287 rocksdb.l2andup.hit COUNT : 229090 rocksdb.compaction.key.drop.new COUNT : 51115 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 2555638 rocksdb.number.keys.read COUNT : 302996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 2637418416 rocksdb.bytes.read COUNT : 302996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 334 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 11010145 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 4450296294 rocksdb.compact.write.bytes COUNT : 4396591270 rocksdb.flush.write.bytes COUNT : 2572924047 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 19 rocksdb.number.superversion_acquires COUNT : 363 rocksdb.number.superversion_releases COUNT : 349 rocksdb.number.superversion_cleanups COUNT : 89 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 2555470848 rocksdb.memtable.garbage.bytes.at.flush COUNT : 3989504 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 6103975790 rocksdb.non.last.level.read.count COUNT : 1440957 rocksdb.block.checksum.compute.count COUNT : 1440308 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 229.208860 P95 : 538.977967 P99 : 801.983441 P100 : 28704.000000 COUNT : 302996 SUM : 80052827 rocksdb.db.write.micros P50 : 1.424762 P95 : 3.940411 P99 : 8.795999 P100 : 2392.000000 COUNT : 2555638 SUM : 16932116 rocksdb.compaction.times.micros P50 : 1067777.777778 P95 : 2948930.000000 P99 : 2948930.000000 P100 : 2948930.000000 COUNT : 11 SUM : 14286514 rocksdb.compaction.times.cpu_micros P50 : 1067777.777778 P95 : 2918772.000000 P99 : 2918772.000000 P100 : 2918772.000000 COUNT : 11 SUM : 14100668 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 707.941176 P95 : 1442.000000 P99 : 1442.000000 P100 : 1442.000000 COUNT : 39 SUM : 30891 rocksdb.compaction.outfile.sync.micros P50 : 1132.777778 P95 : 2575.000000 P99 : 2744.000000 P100 : 2744.000000 COUNT : 13 SUM : 17224 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 157.307692 P95 : 247.555556 P99 : 255.000000 P100 : 255.000000 COUNT : 51 SUM : 8505 rocksdb.table.open.io.micros P50 : 4458.407080 P95 : 9622.500000 P99 : 35420.000000 P100 : 35420.000000 COUNT : 334 SUM : 1764670 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.398177 P95 : 170.393986 P99 : 302.427054 P100 : 4653.000000 COUNT : 1078553 SUM : 21807375 rocksdb.read.block.get.micros P50 : 191.459556 P95 : 360.382900 P99 : 533.330260 P100 : 17680.000000 COUNT : 361700 SUM : 70812049 rocksdb.write.raw.block.micros P50 : 0.523922 P95 : 0.995452 P99 : 3.540801 P100 : 3396.000000 COUNT : 1688936 SUM : 3444093 rocksdb.numfiles.in.singlecompaction P50 : 4.000000 P95 : 26.200000 P99 : 29.000000 P100 : 29.000000 COUNT : 13 SUM : 77 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.573427 P95 : 1259.453242 P99 : 2220.000000 P100 : 2220.000000 COUNT : 61601 SUM : 11016845 rocksdb.sst.read.micros P50 : 1.737641 P95 : 249.822818 P99 : 376.429755 P100 : 17491.000000 COUNT : 1441425 SUM : 88831853 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 302996 SUM : 302996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 2555638 SUM : 2637418416 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 211052.631579 P95 : 248000.000000 P99 : 329300.000000 P100 : 336736.000000 COUNT : 39 SUM : 7533365 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 44 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0