[2024-07-15 18:34:49.779051] Starting SPDK v24.09-pre git sha1 a95bbf233 / DPDK 24.03.0 initialization... [2024-07-15 18:34:49.779384] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3806184 ] Set seed to 1721061289771919 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-15 18:34:49.932113] app.c: 909:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-15 18:34:50.114047] app.c: 603:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-15 18:34:50.114097] app.c: 604:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3806184' to capture a snapshot of events at runtime. [2024-07-15 18:34:50.114109] app.c: 609:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-15 18:34:50.114122] app.c: 610:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-15 18:34:50.114131] app.c: 611:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3806184 for offline analysis/debug. [2024-07-15 18:34:50.114248] reactor.c: 941:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Jul 15 18:34:52 2024 CPU: 144 * Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz CPUCache: 55296 KB 2024/07/15-18:34:57 ... thread 0: (1000000,1000000) ops and (186685.9,186685.9) ops/second in (5.356590,5.356590) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/14 1.25 GB 6.3 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 171.0 4.78 1.89 10 0.478 0 0 0.0 0.0 L1 4/4 263.02 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/1 128.78 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 23/6 1.27 GB 1.0 2.2 1.8 0.3 2.1 1.8 0.0 1.2 128.6 126.2 17.19 9.40 8 2.149 1302K 0 0.0 0.0 L4 151/14 10.85 GB 1.0 3.1 1.6 1.5 2.9 1.4 0.5 1.8 148.9 137.9 21.60 12.47 14 1.543 665K 0 0.0 0.0 L5 141/0 10.43 GB 0.1 0.0 0.0 0.0 0.0 0.0 1.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 335/39 24.18 GB 0.0 5.3 3.4 1.9 5.8 3.9 1.8 7.3 124.6 136.9 43.57 23.77 32 1.362 1968K 0 0.0 0.0 Int 0/0 0.00 KB 0.0 5.3 3.4 1.9 5.8 3.9 1.8 7.3 124.6 136.9 43.57 23.77 32 1.362 1968K 0 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.3 3.4 1.9 5.0 3.1 0.0 0.0 139.9 132.7 38.79 21.88 22 1.763 1968K 0 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 171.0 4.78 1.89 10 0.478 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 5.4 total, 5.4 interval Flush(GB): cumulative 0.798, interval 0.798 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.83 GB write, 1100.53 MB/s write, 5.30 GB read, 1001.02 MB/s read, 43.6 seconds Interval compaction: 5.83 GB write, 1113.40 MB/s write, 5.30 GB read, 1012.73 MB/s read, 43.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 20, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 20, interval: 20 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 117918 Average: 63.6680 StdDev: 539.59 Min: 1 Median: 1.7196 Max: 57042 Percentiles: P50: 1.72 P75: 2.79 P99: 1471.15 P99.9: 4616.81 P99.99: 23208.10 ------------------------------------------------------ [ 0, 1 ] 17323 14.691% 14.691% ### ( 1, 2 ] 57857 49.065% 63.756% ########## ( 2, 3 ] 16776 14.227% 77.983% ### ( 3, 4 ] 10031 8.507% 86.490% ## ( 4, 6 ] 8173 6.931% 93.421% # ( 6, 10 ] 1941 1.646% 95.067% ( 10, 15 ] 110 0.093% 95.160% ( 15, 22 ] 14 0.012% 95.172% ( 22, 34 ] 2 0.002% 95.174% ( 34, 51 ] 18 0.015% 95.189% ( 51, 76 ] 10 0.008% 95.198% ( 110, 170 ] 15 0.013% 95.210% ( 170, 250 ] 53 0.045% 95.255% ( 250, 380 ] 318 0.270% 95.525% ( 380, 580 ] 1020 0.865% 96.390% ( 580, 870 ] 1830 1.552% 97.942% ( 870, 1300 ] 1131 0.959% 98.901% ( 1300, 1900 ] 413 0.350% 99.251% ( 1900, 2900 ] 416 0.353% 99.604% ( 2900, 4400 ] 342 0.290% 99.894% ( 4400, 6600 ] 82 0.070% 99.964% ( 6600, 9900 ] 23 0.020% 99.983% ( 9900, 14000 ] 3 0.003% 99.986% ( 14000, 22000 ] 5 0.004% 99.990% ( 22000, 33000 ] 11 0.009% 99.999% ( 33000, 50000 ] 5 0.004% 100.003% ( 50000, 75000 ] 1 0.001% 100.004% ** Level 1 read latency histogram (micros): Count: 23509 Average: 21.1549 StdDev: 199.33 Min: 1 Median: 1.4630 Max: 5546 Percentiles: P50: 1.46 P75: 2.16 P99: 678.24 P99.9: 3604.14 P99.99: 5546.00 ------------------------------------------------------ [ 0, 1 ] 7088 30.150% 30.150% ###### ( 1, 2 ] 10082 42.886% 73.036% ######### ( 2, 3 ] 2922 12.429% 85.465% ## ( 3, 4 ] 1806 7.682% 93.147% ## ( 4, 6 ] 1047 4.454% 97.601% # ( 6, 10 ] 176 0.749% 98.350% ( 10, 15 ] 16 0.068% 98.418% ( 22, 34 ] 1 0.004% 98.422% ( 34, 51 ] 5 0.021% 98.443% ( 76, 110 ] 2 0.009% 98.452% ( 110, 170 ] 3 0.013% 98.464% ( 170, 250 ] 7 0.030% 98.494% ( 250, 380 ] 21 0.089% 98.584% ( 380, 580 ] 63 0.268% 98.852% ( 580, 870 ] 106 0.451% 99.302% ( 870, 1300 ] 70 0.298% 99.600% ( 1300, 1900 ] 25 0.106% 99.706% ( 1900, 2900 ] 31 0.132% 99.838% ( 2900, 4400 ] 33 0.140% 99.979% ( 4400, 6600 ] 6 0.026% 100.004% ** Level 2 read latency histogram (micros): Count: 376450 Average: 13.6833 StdDev: 253.35 Min: 0 Median: 1.6403 Max: 43135 Percentiles: P50: 1.64 P75: 2.58 P99: 9.57 P99.9: 2761.27 P99.99: 5278.85 ------------------------------------------------------ [ 0, 1 ] 74924 19.903% 19.903% #### ( 1, 2 ] 176958 47.007% 66.910% ######### ( 2, 3 ] 52844 14.037% 80.947% ### ( 3, 4 ] 35429 9.411% 90.359% ## ( 4, 6 ] 27214 7.229% 97.588% # ( 6, 10 ] 5962 1.584% 99.171% ( 10, 15 ] 508 0.135% 99.306% ( 15, 22 ] 132 0.035% 99.341% ( 22, 34 ] 58 0.015% 99.357% ( 34, 51 ] 103 0.027% 99.384% ( 51, 76 ] 32 0.009% 99.393% ( 76, 110 ] 19 0.005% 99.398% ( 110, 170 ] 24 0.006% 99.404% ( 170, 250 ] 43 0.011% 99.416% ( 250, 380 ] 149 0.040% 99.455% ( 380, 580 ] 382 0.101% 99.557% ( 580, 870 ] 458 0.122% 99.678% ( 870, 1300 ] 342 0.091% 99.769% ( 1300, 1900 ] 229 0.061% 99.830% ( 1900, 2900 ] 306 0.081% 99.911% ( 2900, 4400 ] 262 0.070% 99.981% ( 4400, 6600 ] 86 0.023% 100.004% ( 6600, 9900 ] 29 0.008% 100.011% ( 9900, 14000 ] 7 0.002% 100.013% ( 14000, 22000 ] 15 0.004% 100.017% ( 22000, 33000 ] 8 0.002% 100.019% ( 33000, 50000 ] 2 0.001% 100.020% ** Level 3 read latency histogram (micros): Count: 456986 Average: 14.4684 StdDev: 249.96 Min: 0 Median: 1.6137 Max: 55981 Percentiles: P50: 1.61 P75: 2.50 P99: 9.88 P99.9: 2584.82 P99.99: 4319.72 ------------------------------------------------------ [ 0, 1 ] 97953 21.435% 21.435% #### ( 1, 2 ] 212706 46.545% 67.980% ######### ( 2, 3 ] 63788 13.958% 81.938% ### ( 3, 4 ] 42304 9.257% 91.196% ## ( 4, 6 ] 30198 6.608% 97.804% # ( 6, 10 ] 5643 1.235% 99.038% ( 10, 15 ] 304 0.067% 99.105% ( 15, 22 ] 70 0.015% 99.120% ( 22, 34 ] 43 0.009% 99.130% ( 34, 51 ] 120 0.026% 99.156% ( 51, 76 ] 38 0.008% 99.164% ( 76, 110 ] 27 0.006% 99.170% ( 110, 170 ] 28 0.006% 99.176% ( 170, 250 ] 67 0.015% 99.191% ( 250, 380 ] 231 0.051% 99.242% ( 380, 580 ] 725 0.159% 99.400% ( 580, 870 ] 1060 0.232% 99.632% ( 870, 1300 ] 677 0.148% 99.780% ( 1300, 1900 ] 296 0.065% 99.845% ( 1900, 2900 ] 368 0.081% 99.926% ( 2900, 4400 ] 312 0.068% 99.994% ( 4400, 6600 ] 62 0.014% 100.007% ( 6600, 9900 ] 15 0.003% 100.011% ( 9900, 14000 ] 7 0.002% 100.012% ( 14000, 22000 ] 19 0.004% 100.016% ( 22000, 33000 ] 8 0.002% 100.018% ( 33000, 50000 ] 2 0.000% 100.019% ( 50000, 75000 ] 1 0.000% 100.019% ** Level 4 read latency histogram (micros): Count: 581432 Average: 24.8872 StdDev: 328.13 Min: 0 Median: 1.5575 Max: 59977 Percentiles: P50: 1.56 P75: 2.39 P99: 685.30 P99.9: 3322.83 P99.99: 4373.61 ------------------------------------------------------ [ 0, 1 ] 145325 24.994% 24.994% ##### ( 1, 2 ] 260816 44.858% 69.852% ######### ( 2, 3 ] 76655 13.184% 83.036% ### ( 3, 4 ] 50108 8.618% 91.654% ## ( 4, 6 ] 32778 5.637% 97.291% # ( 6, 10 ] 5656 0.973% 98.264% ( 10, 15 ] 405 0.070% 98.334% ( 15, 22 ] 92 0.016% 98.349% ( 22, 34 ] 53 0.009% 98.359% ( 34, 51 ] 141 0.024% 98.383% ( 51, 76 ] 47 0.008% 98.391% ( 76, 110 ] 24 0.004% 98.395% ( 110, 170 ] 53 0.009% 98.404% ( 170, 250 ] 117 0.020% 98.424% ( 250, 380 ] 577 0.099% 98.523% ( 380, 580 ] 1785 0.307% 98.830% ( 580, 870 ] 2720 0.468% 99.298% ( 870, 1300 ] 1674 0.288% 99.586% ( 1300, 1900 ] 706 0.121% 99.708% ( 1900, 2900 ] 910 0.157% 99.864% ( 2900, 4400 ] 747 0.128% 99.993% ( 4400, 6600 ] 185 0.032% 100.024% ( 6600, 9900 ] 40 0.007% 100.031% ( 9900, 14000 ] 12 0.002% 100.033% ( 14000, 22000 ] 23 0.004% 100.037% ( 22000, 33000 ] 13 0.002% 100.040% ( 33000, 50000 ] 9 0.002% 100.041% ( 50000, 75000 ] 1 0.000% 100.041% ** Level 5 read latency histogram (micros): Count: 5361 Average: 1350.6523 StdDev: 2748.76 Min: 123 Median: 794.9008 Max: 71775 Percentiles: P50: 794.90 P75: 1294.00 P99: 6354.16 P99.9: 44982.88 P99.99: 70532.50 ------------------------------------------------------ ( 110, 170 ] 9 0.168% 0.168% ( 170, 250 ] 83 1.548% 1.716% ( 250, 380 ] 368 6.864% 8.580% # ( 380, 580 ] 1063 19.828% 28.409% #### ( 580, 870 ] 1562 29.136% 57.545% ###### ( 870, 1300 ] 949 17.702% 75.247% #### ( 1300, 1900 ] 392 7.312% 82.559% # ( 1900, 2900 ] 426 7.946% 90.506% ## ( 2900, 4400 ] 379 7.070% 97.575% # ( 4400, 6600 ] 86 1.604% 99.179% ( 6600, 9900 ] 14 0.261% 99.440% ( 9900, 14000 ] 2 0.037% 99.478% ( 14000, 22000 ] 5 0.093% 99.571% ( 22000, 33000 ] 12 0.224% 99.795% ( 33000, 50000 ] 8 0.149% 99.944% ( 50000, 75000 ] 3 0.056% 100.000% ** DB Stats ** Uptime(secs): 5.4 total, 5.4 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 181.52 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.635 H:M:S, 30.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 183.62 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.635 H:M:S, 30.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 8 num-running-flushes: 1 2024/07/15-18:35:02 ... thread 0: (1000000,2000000) ops and (224753.0,203958.5) ops/second in (4.449328,9.805918) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/9 1.84 GB 125.1 1.2 0.0 1.2 3.0 1.8 0.0 1.6 75.8 195.1 15.75 8.44 30 0.525 1213K 6946 0.0 0.0 L1 9/0 720.69 MB 72.1 0.7 0.5 0.3 0.7 0.4 0.0 1.5 83.9 82.8 8.71 3.85 1 8.706 741K 9951 0.0 0.0 L3 18/0 999.87 MB 1.0 2.4 1.9 0.4 2.3 1.9 0.0 1.2 132.6 130.2 18.43 10.14 9 2.047 1538K 2792 0.0 0.0 L4 141/0 9.68 GB 1.0 4.8 1.9 3.0 4.3 1.4 0.7 2.3 173.2 154.6 28.64 17.12 19 1.507 966K 0 0.0 0.0 L5 158/0 11.72 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 340/9 24.92 GB 0.0 9.1 4.3 4.8 10.4 5.5 3.2 5.6 130.4 148.5 71.52 39.56 59 1.212 4459K 19K 0.0 0.0 Int 0/0 0.00 KB 0.0 9.1 4.3 4.8 10.4 5.5 3.2 5.6 130.4 148.5 71.52 39.56 59 1.212 4459K 19K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.1 4.3 4.8 8.5 3.7 0.0 0.0 147.5 138.2 63.23 34.68 33 1.916 4459K 19K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 227.6 8.29 4.87 26 0.319 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.9 total, 9.8 interval Flush(GB): cumulative 1.842, interval 1.842 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.37 GB write, 1076.18 MB/s write, 9.11 GB read, 945.12 MB/s read, 71.5 seconds Interval compaction: 10.37 GB write, 1083.05 MB/s write, 9.11 GB read, 951.15 MB/s read, 71.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 6, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 6, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 24, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 6, total-stops: 24, interval: 30 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 475276 Average: 37.3483 StdDev: 283.77 Min: 0 Median: 1.2351 Max: 57042 Percentiles: P50: 1.24 P75: 1.88 P99: 678.33 P99.9: 2298.03 P99.99: 3587.74 ------------------------------------------------------ [ 0, 1 ] 194492 40.922% 40.922% ######## ( 1, 2 ] 183542 38.618% 79.540% ######## ( 2, 3 ] 31026 6.528% 86.068% # ( 3, 4 ] 16286 3.427% 89.495% # ( 4, 6 ] 11637 2.448% 91.943% ( 6, 10 ] 2842 0.598% 92.541% ( 10, 15 ] 1137 0.239% 92.780% ( 15, 22 ] 86 0.018% 92.798% ( 22, 34 ] 63 0.013% 92.812% ( 34, 51 ] 25 0.005% 92.817% ( 51, 76 ] 10 0.002% 92.819% ( 76, 110 ] 97 0.020% 92.839% ( 110, 170 ] 2373 0.499% 93.339% ( 170, 250 ] 6418 1.350% 94.689% ( 250, 380 ] 11944 2.513% 97.202% # ( 380, 580 ] 7367 1.550% 98.752% ( 580, 870 ] 3475 0.731% 99.483% ( 870, 1300 ] 1351 0.284% 99.768% ( 1300, 1900 ] 453 0.095% 99.863% ( 1900, 2900 ] 444 0.093% 99.956% ( 2900, 4400 ] 350 0.074% 100.030% ( 4400, 6600 ] 82 0.017% 100.047% ( 6600, 9900 ] 23 0.005% 100.052% ( 9900, 14000 ] 3 0.001% 100.053% ( 14000, 22000 ] 5 0.001% 100.054% ( 22000, 33000 ] 11 0.002% 100.056% ( 33000, 50000 ] 5 0.001% 100.057% ( 50000, 75000 ] 1 0.000% 100.057% ** Level 1 read latency histogram (micros): Count: 67924 Average: 14.9380 StdDev: 125.41 Min: 0 Median: 2.2060 Max: 5546 Percentiles: P50: 2.21 P75: 3.80 P99: 357.12 P99.9: 1934.71 P99.99: 4318.53 ------------------------------------------------------ [ 0, 1 ] 12244 18.026% 18.026% #### ( 1, 2 ] 19818 29.177% 47.203% ###### ( 2, 3 ] 9223 13.578% 60.781% ### ( 3, 4 ] 12119 17.842% 78.623% #### ( 4, 6 ] 8617 12.686% 91.309% ### ( 6, 10 ] 2333 3.435% 94.744% # ( 10, 15 ] 1419 2.089% 96.833% ( 15, 22 ] 443 0.652% 97.485% ( 22, 34 ] 245 0.361% 97.846% ( 34, 51 ] 40 0.059% 97.905% ( 51, 76 ] 6 0.009% 97.914% ( 76, 110 ] 9 0.013% 97.927% ( 110, 170 ] 113 0.166% 98.093% ( 170, 250 ] 226 0.333% 98.426% ( 250, 380 ] 473 0.696% 99.123% ( 380, 580 ] 270 0.398% 99.520% ( 580, 870 ] 149 0.219% 99.739% ( 870, 1300 ] 80 0.118% 99.857% ( 1300, 1900 ] 28 0.041% 99.898% ( 1900, 2900 ] 31 0.046% 99.944% ( 2900, 4400 ] 33 0.049% 99.993% ( 4400, 6600 ] 6 0.009% 100.001% ** Level 2 read latency histogram (micros): Count: 376454 Average: 13.6887 StdDev: 253.36 Min: 0 Median: 1.6403 Max: 43135 Percentiles: P50: 1.64 P75: 2.58 P99: 9.57 P99.9: 2761.26 P99.99: 5278.84 ------------------------------------------------------ [ 0, 1 ] 74924 19.903% 19.903% #### ( 1, 2 ] 176958 47.007% 66.909% ######### ( 2, 3 ] 52844 14.037% 80.946% ### ( 3, 4 ] 35429 9.411% 90.358% ## ( 4, 6 ] 27214 7.229% 97.587% # ( 6, 10 ] 5962 1.584% 99.170% ( 10, 15 ] 508 0.135% 99.305% ( 15, 22 ] 132 0.035% 99.340% ( 22, 34 ] 58 0.015% 99.356% ( 34, 51 ] 103 0.027% 99.383% ( 51, 76 ] 32 0.009% 99.392% ( 76, 110 ] 19 0.005% 99.397% ( 110, 170 ] 24 0.006% 99.403% ( 170, 250 ] 43 0.011% 99.415% ( 250, 380 ] 150 0.040% 99.454% ( 380, 580 ] 383 0.102% 99.556% ( 580, 870 ] 460 0.122% 99.678% ( 870, 1300 ] 342 0.091% 99.769% ( 1300, 1900 ] 229 0.061% 99.830% ( 1900, 2900 ] 306 0.081% 99.911% ( 2900, 4400 ] 262 0.070% 99.981% ( 4400, 6600 ] 86 0.023% 100.004% ( 6600, 9900 ] 29 0.008% 100.011% ( 9900, 14000 ] 7 0.002% 100.013% ( 14000, 22000 ] 15 0.004% 100.017% ( 22000, 33000 ] 8 0.002% 100.019% ( 33000, 50000 ] 2 0.001% 100.020% ** Level 3 read latency histogram (micros): Count: 506338 Average: 15.5528 StdDev: 239.38 Min: 0 Median: 1.5884 Max: 55981 Percentiles: P50: 1.59 P75: 2.46 P99: 332.51 P99.9: 2435.67 P99.99: 4262.78 ------------------------------------------------------ [ 0, 1 ] 116474 23.003% 23.003% ##### ( 1, 2 ] 232305 45.879% 68.883% ######### ( 2, 3 ] 67948 13.419% 82.302% ### ( 3, 4 ] 44514 8.791% 91.093% ## ( 4, 6 ] 31368 6.195% 97.289% # ( 6, 10 ] 5800 1.145% 98.434% ( 10, 15 ] 342 0.068% 98.502% ( 15, 22 ] 73 0.014% 98.516% ( 22, 34 ] 46 0.009% 98.525% ( 34, 51 ] 120 0.024% 98.549% ( 51, 76 ] 39 0.008% 98.556% ( 76, 110 ] 33 0.007% 98.563% ( 110, 170 ] 308 0.061% 98.624% ( 170, 250 ] 825 0.163% 98.787% ( 250, 380 ] 1701 0.336% 99.123% ( 380, 580 ] 1461 0.289% 99.411% ( 580, 870 ] 1275 0.252% 99.663% ( 870, 1300 ] 703 0.139% 99.802% ( 1300, 1900 ] 299 0.059% 99.861% ( 1900, 2900 ] 369 0.073% 99.934% ( 2900, 4400 ] 313 0.062% 99.996% ( 4400, 6600 ] 62 0.012% 100.008% ( 6600, 9900 ] 15 0.003% 100.011% ( 9900, 14000 ] 7 0.001% 100.012% ( 14000, 22000 ] 19 0.004% 100.016% ( 22000, 33000 ] 8 0.002% 100.018% ( 33000, 50000 ] 2 0.000% 100.018% ( 50000, 75000 ] 1 0.000% 100.018% ** Level 4 read latency histogram (micros): Count: 780430 Average: 27.1379 StdDev: 288.46 Min: 0 Median: 1.4993 Max: 59977 Percentiles: P50: 1.50 P75: 2.33 P99: 584.83 P99.9: 2812.67 P99.99: 4148.91 ------------------------------------------------------ [ 0, 1 ] 226236 28.989% 28.989% ###### ( 1, 2 ] 328393 42.078% 71.067% ######## ( 2, 3 ] 92971 11.913% 82.980% ## ( 3, 4 ] 58752 7.528% 90.508% ## ( 4, 6 ] 37625 4.821% 95.329% # ( 6, 10 ] 6545 0.839% 96.168% ( 10, 15 ] 737 0.094% 96.262% ( 15, 22 ] 132 0.017% 96.279% ( 22, 34 ] 75 0.010% 96.289% ( 34, 51 ] 142 0.018% 96.307% ( 51, 76 ] 50 0.006% 96.313% ( 76, 110 ] 107 0.014% 96.327% ( 110, 170 ] 1894 0.243% 96.570% ( 170, 250 ] 4481 0.574% 97.144% ( 250, 380 ] 8634 1.106% 98.250% ( 380, 580 ] 5790 0.742% 98.992% ( 580, 870 ] 3701 0.474% 99.466% ( 870, 1300 ] 1822 0.233% 99.700% ( 1300, 1900 ] 722 0.093% 99.792% ( 1900, 2900 ] 921 0.118% 99.910% ( 2900, 4400 ] 747 0.096% 100.006% ( 4400, 6600 ] 185 0.024% 100.030% ( 6600, 9900 ] 40 0.005% 100.035% ( 9900, 14000 ] 12 0.002% 100.036% ( 14000, 22000 ] 23 0.003% 100.039% ( 22000, 33000 ] 13 0.002% 100.041% ( 33000, 50000 ] 9 0.001% 100.042% ( 50000, 75000 ] 1 0.000% 100.042% ** Level 5 read latency histogram (micros): Count: 19839 Average: 604.0349 StdDev: 1505.82 Min: 92 Median: 354.8077 Max: 71775 Percentiles: P50: 354.81 P75: 563.92 P99: 4131.45 P99.9: 24897.58 P99.99: 58467.50 ------------------------------------------------------ ( 76, 110 ] 73 0.368% 0.368% ( 110, 170 ] 1323 6.669% 7.037% # ( 170, 250 ] 3346 16.866% 23.902% ### ( 250, 380 ] 6422 32.371% 56.273% ###### ( 380, 580 ] 4040 20.364% 76.637% #### ( 580, 870 ] 2239 11.286% 87.923% ## ( 870, 1300 ] 1035 5.217% 93.140% # ( 1300, 1900 ] 407 2.052% 95.191% ( 1900, 2900 ] 442 2.228% 97.419% ( 2900, 4400 ] 382 1.926% 99.345% ( 4400, 6600 ] 86 0.433% 99.778% ( 6600, 9900 ] 14 0.071% 99.849% ( 9900, 14000 ] 2 0.010% 99.859% ( 14000, 22000 ] 5 0.025% 99.884% ( 22000, 33000 ] 12 0.060% 99.945% ( 33000, 50000 ] 8 0.040% 99.985% ( 50000, 75000 ] 3 0.015% 100.000% ** DB Stats ** Uptime(secs): 9.9 total, 4.4 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 199.41 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.724 H:M:S, 27.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 221.20 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.090 H:M:S, 24.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/15-18:35:05 ... thread 0: (1000000,3000000) ops and (271482.1,222396.8) ops/second in (3.683484,13.489402) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/5 2.75 GB 244.0 2.2 0.0 2.2 5.0 2.8 0.0 1.8 100.6 225.9 22.50 14.76 48 0.469 2299K 12K 0.0 0.0 L1 9/0 720.69 MB 72.1 0.7 0.5 0.3 0.7 0.4 0.0 1.5 83.9 82.8 8.71 3.85 1 8.706 741K 9951 0.0 0.0 L3 18/0 999.87 MB 1.0 2.4 1.9 0.4 2.3 1.9 0.0 1.2 132.6 130.2 18.43 10.14 9 2.047 1538K 2792 0.0 0.0 L4 141/0 9.68 GB 1.0 4.8 1.9 3.0 4.3 1.4 0.7 2.3 173.2 154.6 28.64 17.12 19 1.507 966K 0 0.0 0.0 L5 158/0 11.72 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 342/5 25.83 GB 0.0 10.2 4.3 5.9 12.3 6.5 3.2 4.5 132.9 161.4 78.26 45.88 77 1.016 5545K 25K 0.0 0.0 Int 0/0 0.00 KB 0.0 10.2 4.3 5.9 12.3 6.5 3.2 4.5 132.9 161.4 78.26 45.88 77 1.016 5545K 25K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.2 4.3 5.9 9.6 3.7 0.0 0.0 156.0 147.0 66.65 38.02 37 1.801 5545K 25K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 243.6 11.62 7.86 40 0.290 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 13.6 total, 13.5 interval Flush(GB): cumulative 2.764, interval 2.764 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.33 GB write, 931.80 MB/s write, 10.15 GB read, 767.16 MB/s read, 78.3 seconds Interval compaction: 12.33 GB write, 936.13 MB/s write, 10.15 GB read, 770.73 MB/s read, 78.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 6, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 6, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 6, total-stops: 26, interval: 32 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 740212 Average: 30.7828 StdDev: 231.67 Min: 0 Median: 1.2505 Max: 57042 Percentiles: P50: 1.25 P75: 1.84 P99: 549.94 P99.9: 1628.67 P99.99: 2968.09 ------------------------------------------------------ [ 0, 1 ] 292036 39.453% 39.453% ######## ( 1, 2 ] 311707 42.111% 81.564% ######## ( 2, 3 ] 43296 5.849% 87.413% # ( 3, 4 ] 21208 2.865% 90.278% # ( 4, 6 ] 14505 1.960% 92.237% ( 6, 10 ] 4037 0.545% 92.783% ( 10, 15 ] 2789 0.377% 93.160% ( 15, 22 ] 240 0.032% 93.192% ( 22, 34 ] 141 0.019% 93.211% ( 34, 51 ] 40 0.005% 93.216% ( 51, 76 ] 13 0.002% 93.218% ( 76, 110 ] 211 0.029% 93.247% ( 110, 170 ] 4410 0.596% 93.842% ( 170, 250 ] 10578 1.429% 95.271% ( 250, 380 ] 19671 2.657% 97.929% # ( 380, 580 ] 9330 1.260% 99.189% ( 580, 870 ] 3648 0.493% 99.682% ( 870, 1300 ] 1362 0.184% 99.866% ( 1300, 1900 ] 456 0.062% 99.928% ( 1900, 2900 ] 444 0.060% 99.988% ( 2900, 4400 ] 352 0.048% 100.035% ( 4400, 6600 ] 82 0.011% 100.046% ( 6600, 9900 ] 23 0.003% 100.050% ( 9900, 14000 ] 3 0.000% 100.050% ( 14000, 22000 ] 5 0.001% 100.051% ( 22000, 33000 ] 11 0.001% 100.052% ( 33000, 50000 ] 5 0.001% 100.053% ( 50000, 75000 ] 1 0.000% 100.053% ** Level 1 read latency histogram (micros): Count: 69545 Average: 20.8769 StdDev: 130.85 Min: 0 Median: 2.2935 Max: 5546 Percentiles: P50: 2.29 P75: 3.90 P99: 426.79 P99.9: 1888.32 P99.99: 4311.16 ------------------------------------------------------ [ 0, 1 ] 12245 17.607% 17.607% #### ( 1, 2 ] 19819 28.498% 46.105% ###### ( 2, 3 ] 9229 13.271% 59.376% ### ( 3, 4 ] 12126 17.436% 76.812% ### ( 4, 6 ] 8631 12.411% 89.223% ## ( 6, 10 ] 2341 3.366% 92.589% # ( 10, 15 ] 1437 2.066% 94.655% ( 15, 22 ] 444 0.638% 95.294% ( 22, 34 ] 246 0.354% 95.647% ( 34, 51 ] 40 0.058% 95.705% ( 51, 76 ] 6 0.009% 95.714% ( 76, 110 ] 22 0.032% 95.745% ( 110, 170 ] 336 0.483% 96.228% ( 170, 250 ] 610 0.877% 97.105% ( 250, 380 ] 1209 1.738% 98.844% ( 380, 580 ] 464 0.667% 99.511% ( 580, 870 ] 162 0.233% 99.744% ( 870, 1300 ] 81 0.116% 99.861% ( 1300, 1900 ] 28 0.040% 99.901% ( 1900, 2900 ] 31 0.045% 99.945% ( 2900, 4400 ] 33 0.047% 99.993% ( 4400, 6600 ] 6 0.009% 100.001% ** Level 2 read latency histogram (micros): Count: 376454 Average: 13.6887 StdDev: 253.36 Min: 0 Median: 1.6403 Max: 43135 Percentiles: P50: 1.64 P75: 2.58 P99: 9.57 P99.9: 2761.26 P99.99: 5278.84 ------------------------------------------------------ [ 0, 1 ] 74924 19.903% 19.903% #### ( 1, 2 ] 176958 47.007% 66.909% ######### ( 2, 3 ] 52844 14.037% 80.946% ### ( 3, 4 ] 35429 9.411% 90.358% ## ( 4, 6 ] 27214 7.229% 97.587% # ( 6, 10 ] 5962 1.584% 99.170% ( 10, 15 ] 508 0.135% 99.305% ( 15, 22 ] 132 0.035% 99.340% ( 22, 34 ] 58 0.015% 99.356% ( 34, 51 ] 103 0.027% 99.383% ( 51, 76 ] 32 0.009% 99.392% ( 76, 110 ] 19 0.005% 99.397% ( 110, 170 ] 24 0.006% 99.403% ( 170, 250 ] 43 0.011% 99.415% ( 250, 380 ] 150 0.040% 99.454% ( 380, 580 ] 383 0.102% 99.556% ( 580, 870 ] 460 0.122% 99.678% ( 870, 1300 ] 342 0.091% 99.769% ( 1300, 1900 ] 229 0.061% 99.830% ( 1900, 2900 ] 306 0.081% 99.911% ( 2900, 4400 ] 262 0.070% 99.981% ( 4400, 6600 ] 86 0.023% 100.004% ( 6600, 9900 ] 29 0.008% 100.011% ( 9900, 14000 ] 7 0.002% 100.013% ( 14000, 22000 ] 15 0.004% 100.017% ( 22000, 33000 ] 8 0.002% 100.019% ( 33000, 50000 ] 2 0.001% 100.020% ** Level 3 read latency histogram (micros): Count: 509360 Average: 17.1216 StdDev: 239.67 Min: 0 Median: 1.5949 Max: 55981 Percentiles: P50: 1.59 P75: 2.49 P99: 369.75 P99.9: 2427.48 P99.99: 4261.33 ------------------------------------------------------ [ 0, 1 ] 116474 22.867% 22.867% ##### ( 1, 2 ] 232305 45.607% 68.474% ######### ( 2, 3 ] 67948 13.340% 81.814% ### ( 3, 4 ] 44514 8.739% 90.553% ## ( 4, 6 ] 31368 6.158% 96.711% # ( 6, 10 ] 5800 1.139% 97.850% ( 10, 15 ] 342 0.067% 97.917% ( 15, 22 ] 73 0.014% 97.932% ( 22, 34 ] 46 0.009% 97.941% ( 34, 51 ] 120 0.024% 97.964% ( 51, 76 ] 39 0.008% 97.972% ( 76, 110 ] 69 0.014% 97.985% ( 110, 170 ] 729 0.143% 98.128% ( 170, 250 ] 1570 0.308% 98.437% ( 250, 380 ] 3115 0.612% 99.048% ( 380, 580 ] 1837 0.361% 99.409% ( 580, 870 ] 1303 0.256% 99.665% ( 870, 1300 ] 704 0.138% 99.803% ( 1300, 1900 ] 300 0.059% 99.862% ( 1900, 2900 ] 369 0.072% 99.934% ( 2900, 4400 ] 313 0.061% 99.996% ( 4400, 6600 ] 62 0.012% 100.008% ( 6600, 9900 ] 15 0.003% 100.011% ( 9900, 14000 ] 7 0.001% 100.012% ( 14000, 22000 ] 19 0.004% 100.016% ( 22000, 33000 ] 8 0.002% 100.017% ( 33000, 50000 ] 2 0.000% 100.018% ( 50000, 75000 ] 1 0.000% 100.018% ** Level 4 read latency histogram (micros): Count: 798298 Average: 32.7496 StdDev: 288.08 Min: 0 Median: 1.5265 Max: 59977 Percentiles: P50: 1.53 P75: 2.47 P99: 584.65 P99.9: 2794.47 P99.99: 4145.66 ------------------------------------------------------ [ 0, 1 ] 226236 28.340% 28.340% ###### ( 1, 2 ] 328409 41.139% 69.478% ######## ( 2, 3 ] 92995 11.649% 81.128% ## ( 3, 4 ] 58768 7.362% 88.489% # ( 4, 6 ] 37662 4.718% 93.207% # ( 6, 10 ] 6569 0.823% 94.030% ( 10, 15 ] 786 0.098% 94.128% ( 15, 22 ] 135 0.017% 94.145% ( 22, 34 ] 77 0.010% 94.155% ( 34, 51 ] 142 0.018% 94.173% ( 51, 76 ] 50 0.006% 94.179% ( 76, 110 ] 277 0.035% 94.214% ( 110, 170 ] 4318 0.541% 94.755% ( 170, 250 ] 8899 1.115% 95.869% ( 250, 380 ] 17013 2.131% 98.000% ( 380, 580 ] 7917 0.992% 98.992% ( 580, 870 ] 3870 0.485% 99.477% ( 870, 1300 ] 1827 0.229% 99.706% ( 1300, 1900 ] 725 0.091% 99.797% ( 1900, 2900 ] 922 0.115% 99.912% ( 2900, 4400 ] 748 0.094% 100.006% ( 4400, 6600 ] 185 0.023% 100.029% ( 6600, 9900 ] 40 0.005% 100.034% ( 9900, 14000 ] 12 0.002% 100.036% ( 14000, 22000 ] 23 0.003% 100.038% ( 22000, 33000 ] 13 0.002% 100.040% ( 33000, 50000 ] 9 0.001% 100.041% ( 50000, 75000 ] 1 0.000% 100.041% ** Level 5 read latency histogram (micros): Count: 32626 Average: 477.4871 StdDev: 1186.63 Min: 91 Median: 318.0043 Max: 71775 Percentiles: P50: 318.00 P75: 458.56 P99: 3627.44 P99.9: 9045.30 P99.99: 47316.97 ------------------------------------------------------ ( 76, 110 ] 203 0.622% 0.622% ( 110, 170 ] 3037 9.309% 9.931% ## ( 170, 250 ] 6520 19.984% 29.915% #### ( 250, 380 ] 12527 38.396% 68.311% ######## ( 380, 580 ] 5556 17.029% 85.340% ### ( 580, 870 ] 2380 7.295% 92.635% # ( 870, 1300 ] 1041 3.191% 95.825% # ( 1300, 1900 ] 407 1.247% 97.073% ( 1900, 2900 ] 443 1.358% 98.431% ( 2900, 4400 ] 383 1.174% 99.605% ( 4400, 6600 ] 86 0.264% 99.868% ( 6600, 9900 ] 14 0.043% 99.911% ( 9900, 14000 ] 2 0.006% 99.917% ( 14000, 22000 ] 5 0.015% 99.933% ( 22000, 33000 ] 12 0.037% 99.969% ( 33000, 50000 ] 8 0.025% 99.994% ( 50000, 75000 ] 3 0.009% 100.003% ** DB Stats ** Uptime(secs): 13.6 total, 3.7 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 217.83 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.891 H:M:S, 21.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 267.19 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.166 H:M:S, 4.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/07/15-18:35:10 ... thread 0: (1000000,4000000) ops and (209460.7,219015.3) ops/second in (4.774165,18.263567) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/8 3.73 GB 275.2 3.3 0.0 3.3 7.0 3.7 0.0 1.9 84.1 178.7 40.37 23.41 68 0.594 3448K 18K 0.0 0.0 L1 9/0 720.69 MB 72.1 0.7 0.5 0.3 0.7 0.4 0.0 1.5 83.9 82.8 8.71 3.85 1 8.706 741K 9951 0.0 0.0 L3 18/0 999.87 MB 1.0 2.4 1.9 0.4 2.3 1.9 0.0 1.2 132.6 130.2 18.43 10.14 9 2.047 1538K 2792 0.0 0.0 L4 141/0 9.68 GB 1.0 4.8 1.9 3.0 4.3 1.4 0.7 2.3 173.2 154.6 28.64 17.12 19 1.507 966K 0 0.0 0.0 L5 158/0 11.72 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 345/8 26.81 GB 0.0 11.3 4.3 7.0 14.4 7.4 3.2 3.8 119.9 153.5 96.14 54.53 97 0.991 6695K 31K 0.0 0.0 Int 0/0 0.00 KB 0.0 11.3 4.3 7.0 14.4 7.4 3.2 3.8 119.9 153.5 96.14 54.53 97 0.991 6695K 31K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.3 4.3 7.0 10.7 3.7 0.0 0.0 141.9 134.4 81.28 43.81 41 1.982 6695K 31K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.7 3.7 0.0 0.0 0.0 258.1 14.87 10.72 56 0.265 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 18.3 total, 18.3 interval Flush(GB): cumulative 3.747, interval 3.747 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.42 GB write, 805.42 MB/s write, 11.26 GB read, 629.11 MB/s read, 96.1 seconds Interval compaction: 14.42 GB write, 808.19 MB/s write, 11.26 GB read, 631.27 MB/s read, 96.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 10, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 10, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 10, total-stops: 26, interval: 36 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1202478 Average: 24.0035 StdDev: 185.55 Min: 0 Median: 1.2693 Max: 57042 Percentiles: P50: 1.27 P75: 1.81 P99: 471.12 P99.9: 1120.48 P99.99: 2009.56 ------------------------------------------------------ [ 0, 1 ] 452651 37.643% 37.643% ######## ( 1, 2 ] 551732 45.883% 83.526% ######### ( 2, 3 ] 66583 5.537% 89.063% # ( 3, 4 ] 31014 2.579% 91.642% # ( 4, 6 ] 19855 1.651% 93.294% ( 6, 10 ] 5673 0.472% 93.765% ( 10, 15 ] 5172 0.430% 94.195% ( 15, 22 ] 418 0.035% 94.230% ( 22, 34 ] 255 0.021% 94.251% ( 34, 51 ] 50 0.004% 94.256% ( 51, 76 ] 15 0.001% 94.257% ( 76, 110 ] 438 0.036% 94.293% ( 110, 170 ] 7402 0.616% 94.909% ( 170, 250 ] 15386 1.280% 96.188% ( 250, 380 ] 28667 2.384% 98.572% ( 380, 580 ] 11296 0.939% 99.512% ( 580, 870 ] 3871 0.322% 99.834% ( 870, 1300 ] 1376 0.114% 99.948% ( 1300, 1900 ] 459 0.038% 99.986% ( 1900, 2900 ] 445 0.037% 100.023% ( 2900, 4400 ] 353 0.029% 100.053% ( 4400, 6600 ] 82 0.007% 100.059% ( 6600, 9900 ] 23 0.002% 100.061% ( 9900, 14000 ] 3 0.000% 100.062% ( 14000, 22000 ] 5 0.000% 100.062% ( 22000, 33000 ] 11 0.001% 100.063% ( 33000, 50000 ] 5 0.000% 100.063% ( 50000, 75000 ] 1 0.000% 100.063% ** Level 1 read latency histogram (micros): Count: 71459 Average: 27.7795 StdDev: 136.59 Min: 0 Median: 2.3972 Max: 5546 Percentiles: P50: 2.40 P75: 4.04 P99: 474.66 P99.9: 1847.31 P99.99: 4302.46 ------------------------------------------------------ [ 0, 1 ] 12245 17.136% 17.136% ### ( 1, 2 ] 19819 27.735% 44.870% ###### ( 2, 3 ] 9229 12.915% 57.786% ### ( 3, 4 ] 12126 16.969% 74.755% ### ( 4, 6 ] 8631 12.078% 86.833% ## ( 6, 10 ] 2341 3.276% 90.109% # ( 10, 15 ] 1437 2.011% 92.120% ( 15, 22 ] 444 0.621% 92.741% ( 22, 34 ] 246 0.344% 93.086% ( 34, 51 ] 40 0.056% 93.142% ( 51, 76 ] 6 0.008% 93.150% ( 76, 110 ] 42 0.059% 93.209% ( 110, 170 ] 606 0.848% 94.057% ( 170, 250 ] 1076 1.506% 95.562% ( 250, 380 ] 2136 2.989% 98.552% # ( 380, 580 ] 677 0.947% 99.499% ( 580, 870 ] 180 0.252% 99.751% ( 870, 1300 ] 81 0.113% 99.864% ( 1300, 1900 ] 28 0.039% 99.903% ( 1900, 2900 ] 31 0.043% 99.947% ( 2900, 4400 ] 33 0.046% 99.993% ( 4400, 6600 ] 6 0.008% 100.001% ** Level 2 read latency histogram (micros): Count: 376454 Average: 13.6887 StdDev: 253.36 Min: 0 Median: 1.6403 Max: 43135 Percentiles: P50: 1.64 P75: 2.58 P99: 9.57 P99.9: 2761.26 P99.99: 5278.84 ------------------------------------------------------ [ 0, 1 ] 74924 19.903% 19.903% #### ( 1, 2 ] 176958 47.007% 66.909% ######### ( 2, 3 ] 52844 14.037% 80.946% ### ( 3, 4 ] 35429 9.411% 90.358% ## ( 4, 6 ] 27214 7.229% 97.587% # ( 6, 10 ] 5962 1.584% 99.170% ( 10, 15 ] 508 0.135% 99.305% ( 15, 22 ] 132 0.035% 99.340% ( 22, 34 ] 58 0.015% 99.356% ( 34, 51 ] 103 0.027% 99.383% ( 51, 76 ] 32 0.009% 99.392% ( 76, 110 ] 19 0.005% 99.397% ( 110, 170 ] 24 0.006% 99.403% ( 170, 250 ] 43 0.011% 99.415% ( 250, 380 ] 150 0.040% 99.454% ( 380, 580 ] 383 0.102% 99.556% ( 580, 870 ] 460 0.122% 99.678% ( 870, 1300 ] 342 0.091% 99.769% ( 1300, 1900 ] 229 0.061% 99.830% ( 1900, 2900 ] 306 0.081% 99.911% ( 2900, 4400 ] 262 0.070% 99.981% ( 4400, 6600 ] 86 0.023% 100.004% ( 6600, 9900 ] 29 0.008% 100.011% ( 9900, 14000 ] 7 0.002% 100.013% ( 14000, 22000 ] 15 0.004% 100.017% ( 22000, 33000 ] 8 0.002% 100.019% ( 33000, 50000 ] 2 0.001% 100.020% ** Level 3 read latency histogram (micros): Count: 513097 Average: 19.0444 StdDev: 240.02 Min: 0 Median: 1.6030 Max: 55981 Percentiles: P50: 1.60 P75: 2.53 P99: 394.94 P99.9: 2417.35 P99.99: 4259.54 ------------------------------------------------------ [ 0, 1 ] 116474 22.700% 22.700% ##### ( 1, 2 ] 232305 45.275% 67.975% ######### ( 2, 3 ] 67948 13.243% 81.218% ### ( 3, 4 ] 44514 8.676% 89.894% ## ( 4, 6 ] 31368 6.113% 96.007% # ( 6, 10 ] 5800 1.130% 97.137% ( 10, 15 ] 342 0.067% 97.204% ( 15, 22 ] 73 0.014% 97.218% ( 22, 34 ] 46 0.009% 97.227% ( 34, 51 ] 120 0.023% 97.251% ( 51, 76 ] 39 0.008% 97.258% ( 76, 110 ] 106 0.021% 97.279% ( 110, 170 ] 1219 0.238% 97.516% ( 170, 250 ] 2476 0.483% 97.999% ( 250, 380 ] 4969 0.968% 98.967% ( 380, 580 ] 2236 0.436% 99.403% ( 580, 870 ] 1349 0.263% 99.666% ( 870, 1300 ] 708 0.138% 99.804% ( 1300, 1900 ] 301 0.059% 99.863% ( 1900, 2900 ] 369 0.072% 99.935% ( 2900, 4400 ] 313 0.061% 99.996% ( 4400, 6600 ] 62 0.012% 100.008% ( 6600, 9900 ] 15 0.003% 100.011% ( 9900, 14000 ] 7 0.001% 100.012% ( 14000, 22000 ] 19 0.004% 100.016% ( 22000, 33000 ] 8 0.002% 100.017% ( 33000, 50000 ] 2 0.000% 100.018% ( 50000, 75000 ] 1 0.000% 100.018% ** Level 4 read latency histogram (micros): Count: 820141 Average: 39.3424 StdDev: 287.55 Min: 0 Median: 1.5598 Max: 59977 Percentiles: P50: 1.56 P75: 2.65 P99: 590.18 P99.9: 2769.84 P99.99: 4137.62 ------------------------------------------------------ [ 0, 1 ] 226236 27.585% 27.585% ###### ( 1, 2 ] 328409 40.043% 67.628% ######## ( 2, 3 ] 92995 11.339% 78.967% ## ( 3, 4 ] 58768 7.166% 86.133% # ( 4, 6 ] 37662 4.592% 90.725% # ( 6, 10 ] 6569 0.801% 91.526% ( 10, 15 ] 786 0.096% 91.621% ( 15, 22 ] 135 0.016% 91.638% ( 22, 34 ] 77 0.009% 91.647% ( 34, 51 ] 142 0.017% 91.665% ( 51, 76 ] 50 0.006% 91.671% ( 76, 110 ] 470 0.057% 91.728% ( 110, 170 ] 7414 0.904% 92.632% ( 170, 250 ] 14172 1.728% 94.360% ( 250, 380 ] 27592 3.364% 97.724% # ( 380, 580 ] 10317 1.258% 98.982% ( 580, 870 ] 4147 0.506% 99.488% ( 870, 1300 ] 1850 0.226% 99.713% ( 1300, 1900 ] 727 0.089% 99.802% ( 1900, 2900 ] 923 0.113% 99.915% ( 2900, 4400 ] 749 0.091% 100.006% ( 4400, 6600 ] 185 0.023% 100.029% ( 6600, 9900 ] 40 0.005% 100.033% ( 9900, 14000 ] 12 0.001% 100.035% ( 14000, 22000 ] 23 0.003% 100.038% ( 22000, 33000 ] 13 0.002% 100.039% ( 33000, 50000 ] 9 0.001% 100.040% ( 50000, 75000 ] 1 0.000% 100.040% ** Level 5 read latency histogram (micros): Count: 48414 Average: 413.2243 StdDev: 980.30 Min: 91 Median: 303.5358 Max: 71775 Percentiles: P50: 303.54 P75: 385.60 P99: 3009.11 P99.9: 6461.50 P99.99: 43962.02 ------------------------------------------------------ ( 76, 110 ] 348 0.719% 0.719% ( 110, 170 ] 5204 10.749% 11.468% ## ( 170, 250 ] 10324 21.324% 32.792% #### ( 250, 380 ] 20230 41.785% 74.578% ######## ( 380, 580 ] 7309 15.097% 89.674% ### ( 580, 870 ] 2580 5.329% 95.004% # ( 870, 1300 ] 1055 2.179% 97.183% ( 1300, 1900 ] 409 0.845% 98.027% ( 1900, 2900 ] 443 0.915% 98.942% ( 2900, 4400 ] 383 0.791% 99.734% ( 4400, 6600 ] 86 0.178% 99.911% ( 6600, 9900 ] 14 0.029% 99.940% ( 9900, 14000 ] 2 0.004% 99.944% ( 14000, 22000 ] 5 0.010% 99.955% ( 22000, 33000 ] 12 0.025% 99.979% ( 33000, 50000 ] 8 0.017% 99.996% ( 50000, 75000 ] 3 0.006% 100.002% ** DB Stats ** Uptime(secs): 18.3 total, 4.8 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 214.78 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.679 H:M:S, 20.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 206.15 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.788 H:M:S, 16.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 484.474 micros/op 8209 ops/sec 20.343 seconds 166996 operations; 8.0 MB/s (41999 of 41999 found) Microseconds per read: Count: 166996 Average: 484.5070 StdDev: 1286.13 Min: 2 Median: 378.7938 Max: 172567 Percentiles: P50: 378.79 P75: 527.98 P99: 2981.10 P99.9: 15244.80 P99.99: 42728.16 ------------------------------------------------------ ( 1, 2 ] 3 0.002% 0.002% ( 2, 3 ] 12 0.007% 0.009% ( 3, 4 ] 23 0.014% 0.023% ( 4, 6 ] 77 0.046% 0.069% ( 6, 10 ] 226 0.135% 0.204% ( 10, 15 ] 653 0.391% 0.595% ( 15, 22 ] 908 0.544% 1.139% ( 22, 34 ] 2352 1.408% 2.547% ( 34, 51 ] 2054 1.230% 3.777% ( 51, 76 ] 2632 1.576% 5.353% ( 76, 110 ] 1021 0.611% 5.965% ( 110, 170 ] 4964 2.973% 8.937% # ( 170, 250 ] 26395 15.806% 24.743% ### ( 250, 380 ] 42573 25.493% 50.237% ##### ( 380, 580 ] 55893 33.470% 83.706% ####### ( 580, 870 ] 16960 10.156% 93.862% ## ( 870, 1300 ] 6062 3.630% 97.492% # ( 1300, 1900 ] 1445 0.865% 98.357% ( 1900, 2900 ] 1022 0.612% 98.969% ( 2900, 4400 ] 944 0.565% 99.535% ( 4400, 6600 ] 320 0.192% 99.726% ( 6600, 9900 ] 187 0.112% 99.838% ( 9900, 14000 ] 89 0.053% 99.892% ( 14000, 22000 ] 90 0.054% 99.946% ( 22000, 33000 ] 64 0.038% 99.984% ( 33000, 50000 ] 18 0.011% 99.995% ( 50000, 75000 ] 3 0.002% 99.996% ( 75000, 110000 ] 2 0.001% 99.998% ( 110000, 170000 ] 2 0.001% 99.999% ( 170000, 250000 ] 2 0.001% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 2739772 rocksdb.bloom.filter.full.positive COUNT : 193188 rocksdb.bloom.filter.full.true.positive COUNT : 166239 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 757 rocksdb.memtable.miss COUNT : 166239 rocksdb.l0.hit COUNT : 20386 rocksdb.l1.hit COUNT : 4398 rocksdb.l2andup.hit COUNT : 141455 rocksdb.compaction.key.drop.new COUNT : 630043 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 4504390 rocksdb.number.keys.read COUNT : 166996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 4648530480 rocksdb.bytes.read COUNT : 166996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 507 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 3795718 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 13532711773 rocksdb.compact.write.bytes COUNT : 12878349816 rocksdb.flush.write.bytes COUNT : 4550743918 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 27 rocksdb.number.superversion_acquires COUNT : 766 rocksdb.number.superversion_releases COUNT : 740 rocksdb.number.superversion_cleanups COUNT : 191 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 163 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 4521235456 rocksdb.memtable.garbage.bytes.at.flush COUNT : 8353792 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 13480402461 rocksdb.non.last.level.read.count COUNT : 3225200 rocksdb.block.checksum.compute.count COUNT : 3473133 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 374.776893 P95 : 1000.521870 P99 : 2979.597031 P100 : 172566.000000 COUNT : 166996 SUM : 80464987 rocksdb.db.write.micros P50 : 1.559893 P95 : 3.948611 P99 : 6.075981 P100 : 480709.000000 COUNT : 4504390 SUM : 14871437 rocksdb.compaction.times.micros P50 : 1418750.000000 P95 : 6060000.000000 P99 : 12152000.000000 P100 : 12167797.000000 COUNT : 44 SUM : 85482631 rocksdb.compaction.times.cpu_micros P50 : 908571.428571 P95 : 2800000.000000 P99 : 3853913.000000 P100 : 3853913.000000 COUNT : 44 SUM : 47913008 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 652.500000 P95 : 6160.000000 P99 : 11163.000000 P100 : 11163.000000 COUNT : 64 SUM : 85128 rocksdb.compaction.outfile.sync.micros P50 : 1881.250000 P95 : 8882.500000 P99 : 20520.000000 P100 : 42174.000000 COUNT : 137 SUM : 432112 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 200.909091 P95 : 3455.000000 P99 : 46090.000000 P100 : 55847.000000 COUNT : 123 SUM : 145943 rocksdb.table.open.io.micros P50 : 5325.742574 P95 : 17106.666667 P99 : 29076.666667 P100 : 140915.000000 COUNT : 507 SUM : 3380257 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 2.779011 P95 : 7.674265 P99 : 317.810088 P100 : 57045.000000 COUNT : 3276861 SUM : 48191099 rocksdb.read.block.get.micros P50 : 286.195853 P95 : 763.850085 P99 : 2516.139613 P100 : 71930.000000 COUNT : 195841 SUM : 68562781 rocksdb.write.raw.block.micros P50 : 0.526089 P95 : 0.999569 P99 : 4.577600 P100 : 73287.000000 COUNT : 4230698 SUM : 10756315 rocksdb.numfiles.in.singlecompaction P50 : 3.200000 P95 : 5.750000 P99 : 23.000000 P100 : 23.000000 COUNT : 60 SUM : 232 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.575566 P95 : 1099.059946 P99 : 1282.542234 P100 : 480522.000000 COUNT : 15660 SUM : 3798465 rocksdb.sst.read.micros P50 : 1.478716 P95 : 241.601811 P99 : 542.546918 P100 : 71775.000000 COUNT : 3225308 SUM : 105486207 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 166996 SUM : 166996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 4504390 SUM : 4648530480 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 222571.428571 P95 : 628000.000000 P99 : 749623.000000 P100 : 749623.000000 COUNT : 64 SUM : 16353311 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 163 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0