[2024-07-15 08:55:03.435597] Starting SPDK v24.01.1-pre git sha1 4b94202c6 / DPDK 23.11.0 initialization... [2024-07-15 08:55:03.435697] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3019140 ] Set seed to 1721026503433284 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-15 08:55:03.536886] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-15 08:55:03.760498] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-15 08:55:03.760546] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3019140' to capture a snapshot of events at runtime. [2024-07-15 08:55:03.760557] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3019140 for offline analysis/debug. [2024-07-15 08:55:03.760585] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Jul 15 08:55:12 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/07/15-08:55:20 ... thread 0: (1000000,1000000) ops and (133600.8,133600.8) ops/second in (7.484987,7.484987) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 4/0 629.49 MB 62.9 0.3 0.0 0.3 1.1 0.8 0.0 1.4 36.4 131.0 8.65 3.76 10 0.865 319K 386 0.0 0.0 L1 2/1 175.67 MB 4.7 0.6 0.3 0.3 0.6 0.3 0.0 2.0 123.0 122.9 5.22 2.91 2 2.612 652K 685 0.0 0.0 L2 7/3 520.80 MB 2.4 0.2 0.2 0.0 0.2 0.2 0.0 1.0 146.9 146.9 1.63 1.13 1 1.631 243K 0 0.0 0.0 L3 48/9 3.35 GB 2.5 2.4 1.6 0.8 2.4 1.6 0.0 1.5 98.3 97.6 25.35 11.71 15 1.690 1601K 178 0.0 0.0 L4 183/14 11.95 GB 1.2 2.7 2.2 0.4 2.6 2.2 0.1 1.2 95.4 95.2 28.44 13.19 18 1.580 582K 0 0.0 0.0 L5 1076/0 88.46 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1320/27 105.06 GB 0.0 6.3 4.4 1.9 7.0 5.2 0.1 8.8 92.4 103.8 69.31 32.71 46 1.507 3399K 1249 0.0 0.0 Int 0/0 0.00 KB 0.0 6.3 4.4 1.9 7.0 5.2 0.1 8.8 92.4 103.8 69.31 32.71 46 1.507 3399K 1249 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.3 4.4 1.9 6.2 4.4 0.0 0.0 102.3 101.8 62.63 30.27 37 1.693 3399K 1249 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 122.7 6.67 2.44 9 0.741 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.6 total, 7.5 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.03 GB write, 941.00 MB/s write, 6.25 GB read, 837.54 MB/s read, 69.3 seconds Interval compaction: 7.03 GB write, 961.13 MB/s write, 6.25 GB read, 855.46 MB/s read, 69.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 33, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 33, interval: 33 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 162045 Average: 8.6139 StdDev: 158.46 Min: 1 Median: 1.8853 Max: 26532 Percentiles: P50: 1.89 P75: 2.89 P99: 9.82 P99.9: 1826.37 P99.99: 5059.62 ------------------------------------------------------ [ 0, 1 ] 6516 4.021% 4.021% # ( 1, 2 ] 84160 51.936% 55.957% ########## ( 2, 3 ] 34837 21.498% 77.456% #### ( 3, 4 ] 13876 8.563% 86.019% ## ( 4, 6 ] 17750 10.954% 96.972% ## ( 6, 10 ] 3438 2.122% 99.094% ( 10, 15 ] 873 0.539% 99.633% ( 15, 22 ] 111 0.068% 99.701% ( 22, 34 ] 10 0.006% 99.707% ( 34, 51 ] 1 0.001% 99.708% ( 51, 76 ] 5 0.003% 99.711% ( 76, 110 ] 1 0.001% 99.712% ( 110, 170 ] 2 0.001% 99.713% ( 170, 250 ] 11 0.007% 99.720% ( 250, 380 ] 27 0.017% 99.736% ( 380, 580 ] 49 0.030% 99.767% ( 580, 870 ] 58 0.036% 99.803% ( 870, 1300 ] 79 0.049% 99.851% ( 1300, 1900 ] 90 0.056% 99.907% ( 1900, 2900 ] 71 0.044% 99.951% ( 2900, 4400 ] 56 0.035% 99.985% ( 4400, 6600 ] 26 0.016% 100.001% ( 6600, 9900 ] 4 0.002% 100.004% ( 9900, 14000 ] 1 0.001% 100.004% ( 14000, 22000 ] 3 0.002% 100.006% ( 22000, 33000 ] 1 0.001% 100.007% ** Level 1 read latency histogram (micros): Count: 150513 Average: 10.4077 StdDev: 222.31 Min: 1 Median: 1.9544 Max: 52428 Percentiles: P50: 1.95 P75: 2.94 P99: 9.94 P99.9: 2618.92 P99.99: 5824.36 ------------------------------------------------------ [ 0, 1 ] 3549 2.358% 2.358% ( 1, 2 ] 75133 49.918% 52.276% ########## ( 2, 3 ] 36287 24.109% 76.385% ##### ( 3, 4 ] 14045 9.331% 85.716% ## ( 4, 6 ] 16981 11.282% 96.998% ## ( 6, 10 ] 3059 2.032% 99.031% ( 10, 15 ] 844 0.561% 99.591% ( 15, 22 ] 123 0.082% 99.673% ( 22, 34 ] 7 0.005% 99.678% ( 34, 51 ] 2 0.001% 99.679% ( 51, 76 ] 2 0.001% 99.680% ( 76, 110 ] 2 0.001% 99.682% ( 110, 170 ] 2 0.001% 99.683% ( 170, 250 ] 4 0.003% 99.686% ( 250, 380 ] 14 0.009% 99.695% ( 380, 580 ] 30 0.020% 99.715% ( 580, 870 ] 40 0.027% 99.742% ( 870, 1300 ] 74 0.049% 99.791% ( 1300, 1900 ] 89 0.059% 99.850% ( 1900, 2900 ] 105 0.070% 99.920% ( 2900, 4400 ] 93 0.062% 99.981% ( 4400, 6600 ] 20 0.013% 99.995% ( 6600, 9900 ] 8 0.005% 100.000% ( 22000, 33000 ] 2 0.001% 100.001% ( 50000, 75000 ] 1 0.001% 100.002% ** Level 2 read latency histogram (micros): Count: 406538 Average: 17.9685 StdDev: 215.41 Min: 1 Median: 2.0016 Max: 10455 Percentiles: P50: 2.00 P75: 3.19 P99: 13.93 P99.9: 2391.21 P99.99: 3042.30 ------------------------------------------------------ [ 0, 1 ] 11538 2.838% 2.838% # ( 1, 2 ] 191576 47.124% 49.962% ######### ( 2, 3 ] 94125 23.153% 73.115% ##### ( 3, 4 ] 40280 9.908% 83.023% ## ( 4, 6 ] 49018 12.057% 95.080% ## ( 6, 10 ] 12704 3.125% 98.205% # ( 10, 15 ] 4111 1.011% 99.216% ( 15, 22 ] 702 0.173% 99.389% ( 22, 34 ] 82 0.020% 99.409% ( 34, 51 ] 17 0.004% 99.413% ( 51, 76 ] 15 0.004% 99.417% ( 76, 110 ] 8 0.002% 99.419% ( 110, 170 ] 16 0.004% 99.423% ( 170, 250 ] 45 0.011% 99.434% ( 250, 380 ] 105 0.026% 99.460% ( 380, 580 ] 205 0.050% 99.510% ( 580, 870 ] 333 0.082% 99.592% ( 870, 1300 ] 440 0.108% 99.700% ( 1300, 1900 ] 502 0.123% 99.824% ( 1900, 2900 ] 630 0.155% 99.979% ( 2900, 4400 ] 478 0.118% 100.096% ( 4400, 6600 ] 179 0.044% 100.140% ( 6600, 9900 ] 36 0.009% 100.149% ( 9900, 14000 ] 4 0.001% 100.150% ** Level 3 read latency histogram (micros): Count: 843346 Average: 17.7217 StdDev: 225.83 Min: 1 Median: 2.1124 Max: 38953 Percentiles: P50: 2.11 P75: 3.14 P99: 13.00 P99.9: 1810.51 P99.99: 2297.48 ------------------------------------------------------ [ 0, 1 ] 13012 1.543% 1.543% ( 1, 2 ] 383418 45.464% 47.007% ######### ( 2, 3 ] 224654 26.638% 73.645% ##### ( 3, 4 ] 83560 9.908% 83.553% ## ( 4, 6 ] 102351 12.136% 95.690% ## ( 6, 10 ] 23449 2.780% 98.470% # ( 10, 15 ] 7462 0.885% 99.355% ( 15, 22 ] 998 0.118% 99.473% ( 22, 34 ] 67 0.008% 99.481% ( 34, 51 ] 28 0.003% 99.485% ( 51, 76 ] 31 0.004% 99.488% ( 76, 110 ] 19 0.002% 99.490% ( 110, 170 ] 21 0.002% 99.493% ( 170, 250 ] 68 0.008% 99.501% ( 250, 380 ] 189 0.022% 99.523% ( 380, 580 ] 332 0.039% 99.563% ( 580, 870 ] 661 0.078% 99.641% ( 870, 1300 ] 1036 0.123% 99.764% ( 1300, 1900 ] 1350 0.160% 99.924% ( 1900, 2900 ] 1403 0.166% 100.090% ( 2900, 4400 ] 892 0.106% 100.196% ( 4400, 6600 ] 252 0.030% 100.226% ( 6600, 9900 ] 36 0.004% 100.230% ( 9900, 14000 ] 6 0.001% 100.231% ( 14000, 22000 ] 7 0.001% 100.232% ( 22000, 33000 ] 4 0.000% 100.232% ( 33000, 50000 ] 4 0.000% 100.233% ** Level 4 read latency histogram (micros): Count: 168949 Average: 49.3184 StdDev: 410.07 Min: 1 Median: 2.0584 Max: 48561 Percentiles: P50: 2.06 P75: 3.14 P99: 1719.29 P99.9: 4350.69 P99.99: 6030.01 ------------------------------------------------------ [ 0, 1 ] 2970 1.758% 1.758% ( 1, 2 ] 79031 46.778% 48.536% ######### ( 2, 3 ] 42354 25.069% 73.605% ##### ( 3, 4 ] 16649 9.854% 83.460% ## ( 4, 6 ] 18939 11.210% 94.669% ## ( 6, 10 ] 3868 2.289% 96.959% ( 10, 15 ] 1044 0.618% 97.577% ( 15, 22 ] 156 0.092% 97.669% ( 22, 34 ] 14 0.008% 97.677% ( 34, 51 ] 3 0.002% 97.679% ( 51, 76 ] 2 0.001% 97.680% ( 76, 110 ] 6 0.004% 97.684% ( 110, 170 ] 16 0.009% 97.693% ( 170, 250 ] 60 0.036% 97.729% ( 250, 380 ] 125 0.074% 97.803% ( 380, 580 ] 253 0.150% 97.953% ( 580, 870 ] 452 0.268% 98.220% ( 870, 1300 ] 734 0.434% 98.655% ( 1300, 1900 ] 835 0.494% 99.149% ( 1900, 2900 ] 741 0.439% 99.587% ( 2900, 4400 ] 546 0.323% 99.911% ( 4400, 6600 ] 181 0.107% 100.018% ( 6600, 9900 ] 47 0.028% 100.046% ( 9900, 14000 ] 2 0.001% 100.047% ( 14000, 22000 ] 5 0.003% 100.050% ( 22000, 33000 ] 4 0.002% 100.052% ( 33000, 50000 ] 1 0.001% 100.053% ** Level 5 read latency histogram (micros): Count: 9260 Average: 2311.7226 StdDev: 1958.04 Min: 102 Median: 1772.2506 Max: 44011 Percentiles: P50: 1772.25 P75: 3285.28 P99: 8837.18 P99.9: 15973.33 P99.99: 44011.00 ------------------------------------------------------ ( 76, 110 ] 3 0.032% 0.032% ( 110, 170 ] 32 0.346% 0.378% ( 170, 250 ] 74 0.799% 1.177% ( 250, 380 ] 229 2.473% 3.650% ( 380, 580 ] 530 5.724% 9.374% # ( 580, 870 ] 1002 10.821% 20.194% ## ( 870, 1300 ] 1529 16.512% 36.706% ### ( 1300, 1900 ] 1564 16.890% 53.596% ### ( 1900, 2900 ] 1579 17.052% 70.648% ### ( 2900, 4400 ] 1569 16.944% 87.592% ### ( 4400, 6600 ] 912 9.849% 97.441% ## ( 6600, 9900 ] 213 2.300% 99.741% ( 9900, 14000 ] 14 0.151% 99.892% ( 14000, 22000 ] 3 0.032% 99.924% ( 22000, 33000 ] 4 0.043% 99.968% ( 33000, 50000 ] 3 0.032% 100.000% ** DB Stats ** Uptime(secs): 7.6 total, 7.5 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 128.70 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.499 H:M:S, 45.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 131.44 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.499 H:M:S, 46.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/07/15-08:55:25 ... thread 0: (1000000,2000000) ops and (198764.8,159794.7) ops/second in (5.031072,12.516059) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/6 1.60 GB 81.8 0.9 0.0 0.9 2.6 1.8 0.0 1.5 54.5 167.3 16.18 8.94 26 0.622 895K 963 0.0 0.0 L1 1/1 46.89 MB 0.0 0.6 0.3 0.3 0.6 0.3 0.0 2.0 123.0 122.9 5.22 2.91 2 2.612 652K 685 0.0 0.0 L2 2/2 166.77 MB 0.0 0.5 0.4 0.2 0.5 0.4 0.0 1.5 125.3 125.2 4.44 2.66 2 2.218 564K 614 0.0 0.0 L3 45/24 2.57 GB 1.2 4.1 2.0 2.1 4.0 2.0 0.1 2.0 116.4 115.7 35.80 19.56 20 1.790 2482K 178 0.0 0.0 L4 157/11 9.55 GB 0.9 4.8 3.5 1.3 4.7 3.4 0.1 1.4 103.3 102.6 47.26 23.15 32 1.477 1552K 0 0.0 0.0 L5 1120/0 92.09 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1337/44 106.01 GB 0.0 10.9 6.2 4.7 12.6 7.9 3.8 7.1 102.2 118.4 108.91 57.22 82 1.328 6147K 2440 0.0 0.0 Int 0/0 0.00 KB 0.0 10.9 6.2 4.7 12.6 7.9 3.8 7.1 102.2 118.4 108.91 57.22 82 1.328 6147K 2440 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.9 6.2 4.7 10.8 6.1 0.0 0.0 114.7 114.1 97.03 51.86 59 1.645 6147K 2440 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 153.7 11.88 5.36 23 0.517 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.7 total, 12.5 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.60 GB write, 1017.40 MB/s write, 10.87 GB read, 878.08 MB/s read, 108.9 seconds Interval compaction: 12.60 GB write, 1030.42 MB/s write, 10.87 GB read, 889.31 MB/s read, 108.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 45, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 45, interval: 45 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 318773 Average: 19.2406 StdDev: 216.96 Min: 1 Median: 1.8494 Max: 34279 Percentiles: P50: 1.85 P75: 2.82 P99: 506.86 P99.9: 2414.97 P99.99: 4218.43 ------------------------------------------------------ [ 0, 1 ] 14937 4.686% 4.686% # ( 1, 2 ] 170064 53.350% 58.035% ########### ( 2, 3 ] 65809 20.644% 78.680% #### ( 3, 4 ] 25427 7.977% 86.656% ## ( 4, 6 ] 29856 9.366% 96.022% ## ( 6, 10 ] 5097 1.599% 97.621% ( 10, 15 ] 1171 0.367% 97.989% ( 15, 22 ] 139 0.044% 98.032% ( 22, 34 ] 16 0.005% 98.037% ( 34, 51 ] 5 0.002% 98.039% ( 51, 76 ] 8 0.003% 98.041% ( 76, 110 ] 22 0.007% 98.048% ( 110, 170 ] 301 0.094% 98.143% ( 170, 250 ] 842 0.264% 98.407% ( 250, 380 ] 1054 0.331% 98.737% ( 380, 580 ] 1320 0.414% 99.151% ( 580, 870 ] 1150 0.361% 99.512% ( 870, 1300 ] 754 0.237% 99.749% ( 1300, 1900 ] 354 0.111% 99.860% ( 1900, 2900 ] 249 0.078% 99.938% ( 2900, 4400 ] 189 0.059% 99.997% ( 4400, 6600 ] 59 0.019% 100.016% ( 6600, 9900 ] 9 0.003% 100.019% ( 9900, 14000 ] 5 0.002% 100.020% ( 14000, 22000 ] 5 0.002% 100.022% ( 22000, 33000 ] 4 0.001% 100.023% ( 33000, 50000 ] 1 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 179172 Average: 10.5226 StdDev: 209.47 Min: 1 Median: 1.9827 Max: 52428 Percentiles: P50: 1.98 P75: 2.97 P99: 10.63 P99.9: 2506.62 P99.99: 5619.11 ------------------------------------------------------ [ 0, 1 ] 3763 2.100% 2.100% ( 1, 2 ] 87332 48.742% 50.842% ########## ( 2, 3 ] 44499 24.836% 75.678% ##### ( 3, 4 ] 17113 9.551% 85.229% ## ( 4, 6 ] 20563 11.477% 96.706% ## ( 6, 10 ] 3974 2.218% 98.924% ( 10, 15 ] 1077 0.601% 99.525% ( 15, 22 ] 164 0.092% 99.617% ( 22, 34 ] 8 0.004% 99.621% ( 34, 51 ] 3 0.002% 99.623% ( 51, 76 ] 3 0.002% 99.624% ( 76, 110 ] 5 0.003% 99.627% ( 110, 170 ] 3 0.002% 99.629% ( 170, 250 ] 7 0.004% 99.633% ( 250, 380 ] 24 0.013% 99.646% ( 380, 580 ] 59 0.033% 99.679% ( 580, 870 ] 78 0.044% 99.723% ( 870, 1300 ] 123 0.069% 99.791% ( 1300, 1900 ] 119 0.066% 99.858% ( 1900, 2900 ] 125 0.070% 99.927% ( 2900, 4400 ] 101 0.056% 99.984% ( 4400, 6600 ] 20 0.011% 99.995% ( 6600, 9900 ] 9 0.005% 100.000% ( 22000, 33000 ] 2 0.001% 100.001% ( 50000, 75000 ] 1 0.001% 100.002% ** Level 2 read latency histogram (micros): Count: 532216 Average: 15.3224 StdDev: 197.09 Min: 1 Median: 1.9937 Max: 22061 Percentiles: P50: 1.99 P75: 3.09 P99: 13.45 P99.9: 2218.89 P99.99: 2988.10 ------------------------------------------------------ [ 0, 1 ] 13471 2.531% 2.531% # ( 1, 2 ] 254226 47.767% 50.299% ########## ( 2, 3 ] 126591 23.786% 74.084% ##### ( 3, 4 ] 51592 9.694% 83.778% ## ( 4, 6 ] 61977 11.645% 95.423% ## ( 6, 10 ] 15711 2.952% 98.375% # ( 10, 15 ] 4821 0.906% 99.281% ( 15, 22 ] 785 0.147% 99.428% ( 22, 34 ] 87 0.016% 99.445% ( 34, 51 ] 21 0.004% 99.449% ( 51, 76 ] 25 0.005% 99.453% ( 76, 110 ] 10 0.002% 99.455% ( 110, 170 ] 37 0.007% 99.462% ( 170, 250 ] 130 0.024% 99.487% ( 250, 380 ] 206 0.039% 99.525% ( 380, 580 ] 323 0.061% 99.586% ( 580, 870 ] 426 0.080% 99.666% ( 870, 1300 ] 513 0.096% 99.763% ( 1300, 1900 ] 521 0.098% 99.860% ( 1900, 2900 ] 661 0.124% 99.985% ( 2900, 4400 ] 490 0.092% 100.077% ( 4400, 6600 ] 184 0.035% 100.111% ( 6600, 9900 ] 36 0.007% 100.118% ( 9900, 14000 ] 4 0.001% 100.119% ( 14000, 22000 ] 2 0.000% 100.119% ( 22000, 33000 ] 1 0.000% 100.119% ** Level 3 read latency histogram (micros): Count: 1553589 Average: 13.4218 StdDev: 185.56 Min: 1 Median: 2.0395 Max: 38953 Percentiles: P50: 2.04 P75: 3.00 P99: 12.04 P99.9: 1300.15 P99.99: 1811.07 ------------------------------------------------------ [ 0, 1 ] 22575 1.453% 1.453% ( 1, 2 ] 738159 47.513% 48.966% ########## ( 2, 3 ] 406426 26.160% 75.127% ##### ( 3, 4 ] 146914 9.456% 84.583% ## ( 4, 6 ] 177726 11.440% 96.023% ## ( 6, 10 ] 41334 2.661% 98.683% # ( 10, 15 ] 12050 0.776% 99.459% ( 15, 22 ] 1457 0.094% 99.553% ( 22, 34 ] 101 0.007% 99.559% ( 34, 51 ] 67 0.004% 99.564% ( 51, 76 ] 76 0.005% 99.568% ( 76, 110 ] 49 0.003% 99.572% ( 110, 170 ] 196 0.013% 99.584% ( 170, 250 ] 398 0.026% 99.610% ( 250, 380 ] 697 0.045% 99.655% ( 380, 580 ] 958 0.062% 99.716% ( 580, 870 ] 1311 0.084% 99.801% ( 870, 1300 ] 1541 0.099% 99.900% ( 1300, 1900 ] 1642 0.106% 100.006% ( 1900, 2900 ] 1635 0.105% 100.111% ( 2900, 4400 ] 1089 0.070% 100.181% ( 4400, 6600 ] 305 0.020% 100.201% ( 6600, 9900 ] 48 0.003% 100.204% ( 9900, 14000 ] 13 0.001% 100.205% ( 14000, 22000 ] 14 0.001% 100.205% ( 22000, 33000 ] 4 0.000% 100.206% ( 33000, 50000 ] 4 0.000% 100.206% ** Level 4 read latency histogram (micros): Count: 418040 Average: 32.0374 StdDev: 315.90 Min: 1 Median: 2.0333 Max: 48561 Percentiles: P50: 2.03 P75: 3.14 P99: 894.92 P99.9: 3429.10 P99.99: 4177.58 ------------------------------------------------------ [ 0, 1 ] 7144 1.709% 1.709% ( 1, 2 ] 198472 47.477% 49.186% ######### ( 2, 3 ] 102392 24.493% 73.679% ##### ( 3, 4 ] 40217 9.620% 83.299% ## ( 4, 6 ] 48078 11.501% 94.800% ## ( 6, 10 ] 10772 2.577% 97.377% # ( 10, 15 ] 2764 0.661% 98.038% ( 15, 22 ] 351 0.084% 98.122% ( 22, 34 ] 31 0.007% 98.130% ( 34, 51 ] 18 0.004% 98.134% ( 51, 76 ] 25 0.006% 98.140% ( 76, 110 ] 23 0.006% 98.145% ( 110, 170 ] 160 0.038% 98.184% ( 170, 250 ] 422 0.101% 98.285% ( 250, 380 ] 701 0.168% 98.452% ( 380, 580 ] 972 0.233% 98.685% ( 580, 870 ] 1238 0.296% 98.981% ( 870, 1300 ] 1391 0.333% 99.314% ( 1300, 1900 ] 1213 0.290% 99.604% ( 1900, 2900 ] 973 0.233% 99.837% ( 2900, 4400 ] 754 0.180% 100.017% ( 4400, 6600 ] 234 0.056% 100.073% ( 6600, 9900 ] 53 0.013% 100.086% ( 9900, 14000 ] 4 0.001% 100.087% ( 14000, 22000 ] 9 0.002% 100.089% ( 22000, 33000 ] 10 0.002% 100.091% ( 33000, 50000 ] 1 0.000% 100.091% ** Level 5 read latency histogram (micros): Count: 27397 Average: 1330.1017 StdDev: 1592.96 Min: 86 Median: 784.4298 Max: 44011 Percentiles: P50: 784.43 P75: 1660.12 P99: 6696.26 P99.9: 12939.55 P99.99: 38356.27 ------------------------------------------------------ ( 76, 110 ] 76 0.277% 0.277% ( 110, 170 ] 938 3.424% 3.701% # ( 170, 250 ] 2091 7.632% 11.333% ## ( 250, 380 ] 3112 11.359% 22.692% ## ( 380, 580 ] 4236 15.462% 38.154% ### ( 580, 870 ] 4604 16.805% 54.959% ### ( 870, 1300 ] 3902 14.242% 69.201% ### ( 1300, 1900 ] 2647 9.662% 78.863% ## ( 1900, 2900 ] 2299 8.391% 87.254% ## ( 2900, 4400 ] 2160 7.884% 95.138% ## ( 4400, 6600 ] 1051 3.836% 98.974% # ( 6600, 9900 ] 241 0.880% 99.854% ( 9900, 14000 ] 17 0.062% 99.916% ( 14000, 22000 ] 9 0.033% 99.949% ( 22000, 33000 ] 10 0.037% 99.985% ( 33000, 50000 ] 4 0.015% 100.000% ** DB Stats ** Uptime(secs): 12.7 total, 5.0 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 155.26 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.719 H:M:S, 37.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 195.62 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.220 H:M:S, 24.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/07/15-08:55:30 ... thread 0: (1000000,3000000) ops and (209851.3,173597.7) ops/second in (4.765280,17.281339) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/10 2.58 GB 157.3 1.6 0.0 1.6 4.4 2.8 0.0 1.6 69.6 190.1 23.52 14.55 43 0.547 1663K 1716 0.0 0.0 L1 1/1 46.89 MB 0.0 0.6 0.3 0.3 0.6 0.3 0.0 2.0 123.0 122.9 5.22 2.91 2 2.612 652K 685 0.0 0.0 L2 0/0 0.00 KB 0.0 0.5 0.4 0.2 0.5 0.4 0.0 1.5 125.3 125.2 4.44 2.66 2 2.218 564K 614 0.0 0.0 L3 18/0 987.14 MB 1.0 5.1 2.2 2.9 5.0 2.1 0.1 2.3 121.6 120.9 42.63 24.41 22 1.938 3431K 178 0.0 0.0 L4 172/0 9.60 GB 1.0 7.3 4.9 2.4 7.2 4.8 0.4 1.5 112.7 111.5 66.56 35.53 51 1.305 3284K 2527 0.0 0.0 L5 1140/0 93.75 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1348/11 106.94 GB 0.0 15.2 7.8 7.4 17.8 10.4 5.8 6.4 109.0 128.1 142.37 80.07 120 1.186 9595K 5720 0.0 0.0 Int 0/0 0.00 KB 0.0 15.2 7.8 7.4 17.8 10.4 5.8 6.4 109.0 128.1 142.37 80.07 120 1.186 9595K 5720 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 15.2 7.8 7.4 15.0 7.7 0.0 0.0 123.1 122.2 126.11 71.94 83 1.519 9595K 5720 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 174.3 16.26 8.12 37 0.439 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 17.4 total, 17.3 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 17.81 GB write, 1045.75 MB/s write, 15.16 GB read, 889.84 MB/s read, 142.4 seconds Interval compaction: 17.81 GB write, 1055.45 MB/s write, 15.16 GB read, 898.09 MB/s read, 142.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 1, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 1, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 59, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 1, total-stops: 59, interval: 60 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 553148 Average: 22.2872 StdDev: 205.11 Min: 1 Median: 1.7537 Max: 34279 Percentiles: P50: 1.75 P75: 2.67 P99: 524.80 P99.9: 2257.64 P99.99: 3999.45 ------------------------------------------------------ [ 0, 1 ] 48784 8.819% 8.819% ## ( 1, 2 ] 302223 54.637% 63.456% ########### ( 2, 3 ] 94650 17.111% 80.567% ### ( 3, 4 ] 36913 6.673% 87.241% # ( 4, 6 ] 40882 7.391% 94.631% # ( 6, 10 ] 6840 1.237% 95.868% ( 10, 15 ] 1544 0.279% 96.147% ( 15, 22 ] 216 0.039% 96.186% ( 22, 34 ] 30 0.005% 96.192% ( 34, 51 ] 17 0.003% 96.195% ( 51, 76 ] 16 0.003% 96.198% ( 76, 110 ] 412 0.074% 96.272% ( 110, 170 ] 5197 0.940% 97.212% ( 170, 250 ] 5094 0.921% 98.133% ( 250, 380 ] 2780 0.503% 98.635% ( 380, 580 ] 2788 0.504% 99.139% ( 580, 870 ] 2150 0.389% 99.528% ( 870, 1300 ] 1309 0.237% 99.764% ( 1300, 1900 ] 600 0.108% 99.873% ( 1900, 2900 ] 419 0.076% 99.949% ( 2900, 4400 ] 312 0.056% 100.005% ( 4400, 6600 ] 101 0.018% 100.023% ( 6600, 9900 ] 15 0.003% 100.026% ( 9900, 14000 ] 7 0.001% 100.027% ( 14000, 22000 ] 8 0.001% 100.029% ( 22000, 33000 ] 5 0.001% 100.030% ( 33000, 50000 ] 1 0.000% 100.030% ** Level 1 read latency histogram (micros): Count: 184943 Average: 10.7258 StdDev: 208.27 Min: 1 Median: 2.0062 Max: 52428 Percentiles: P50: 2.01 P75: 3.04 P99: 11.48 P99.9: 2514.62 P99.99: 5691.85 ------------------------------------------------------ [ 0, 1 ] 3798 2.054% 2.054% ( 1, 2 ] 88391 47.794% 49.847% ########## ( 2, 3 ] 45780 24.754% 74.601% ##### ( 3, 4 ] 18200 9.841% 84.442% ## ( 4, 6 ] 22189 11.998% 96.439% ## ( 6, 10 ] 4386 2.372% 98.811% ( 10, 15 ] 1181 0.639% 99.450% ( 15, 22 ] 209 0.113% 99.563% ( 22, 34 ] 17 0.009% 99.572% ( 34, 51 ] 5 0.003% 99.574% ( 51, 76 ] 3 0.002% 99.576% ( 76, 110 ] 7 0.004% 99.580% ( 110, 170 ] 34 0.018% 99.598% ( 170, 250 ] 27 0.015% 99.613% ( 250, 380 ] 34 0.018% 99.631% ( 380, 580 ] 73 0.039% 99.671% ( 580, 870 ] 90 0.049% 99.719% ( 870, 1300 ] 133 0.072% 99.791% ( 1300, 1900 ] 123 0.067% 99.858% ( 1900, 2900 ] 127 0.069% 99.926% ( 2900, 4400 ] 104 0.056% 99.983% ( 4400, 6600 ] 23 0.012% 99.995% ( 6600, 9900 ] 9 0.005% 100.000% ( 22000, 33000 ] 2 0.001% 100.001% ( 50000, 75000 ] 1 0.001% 100.002% ** Level 2 read latency histogram (micros): Count: 555672 Average: 14.9300 StdDev: 193.34 Min: 1 Median: 1.9951 Max: 22061 Percentiles: P50: 2.00 P75: 3.09 P99: 13.38 P99.9: 2192.22 P99.99: 2989.55 ------------------------------------------------------ [ 0, 1 ] 13854 2.493% 2.493% ( 1, 2 ] 265274 47.739% 50.233% ########## ( 2, 3 ] 132709 23.883% 74.115% ##### ( 3, 4 ] 53852 9.691% 83.806% ## ( 4, 6 ] 64749 11.652% 95.459% ## ( 6, 10 ] 16321 2.937% 98.396% # ( 10, 15 ] 4972 0.895% 99.291% ( 15, 22 ] 795 0.143% 99.434% ( 22, 34 ] 88 0.016% 99.450% ( 34, 51 ] 21 0.004% 99.453% ( 51, 76 ] 29 0.005% 99.459% ( 76, 110 ] 15 0.003% 99.461% ( 110, 170 ] 49 0.009% 99.470% ( 170, 250 ] 142 0.026% 99.496% ( 250, 380 ] 216 0.039% 99.535% ( 380, 580 ] 338 0.061% 99.595% ( 580, 870 ] 445 0.080% 99.676% ( 870, 1300 ] 528 0.095% 99.771% ( 1300, 1900 ] 525 0.094% 99.865% ( 1900, 2900 ] 665 0.120% 99.985% ( 2900, 4400 ] 493 0.089% 100.073% ( 4400, 6600 ] 184 0.033% 100.107% ( 6600, 9900 ] 36 0.006% 100.113% ( 9900, 14000 ] 4 0.001% 100.114% ( 14000, 22000 ] 2 0.000% 100.114% ( 22000, 33000 ] 1 0.000% 100.114% ** Level 3 read latency histogram (micros): Count: 1912036 Average: 12.6590 StdDev: 179.18 Min: 1 Median: 2.0256 Max: 38953 Percentiles: P50: 2.03 P75: 2.99 P99: 11.78 P99.9: 1179.67 P99.99: 1715.26 ------------------------------------------------------ [ 0, 1 ] 28867 1.510% 1.510% ( 1, 2 ] 914477 47.827% 49.337% ########## ( 2, 3 ] 495919 25.937% 75.274% ##### ( 3, 4 ] 179938 9.411% 84.685% ## ( 4, 6 ] 218788 11.443% 96.127% ## ( 6, 10 ] 49981 2.614% 98.741% # ( 10, 15 ] 13886 0.726% 99.468% ( 15, 22 ] 1631 0.085% 99.553% ( 22, 34 ] 122 0.006% 99.559% ( 34, 51 ] 101 0.005% 99.565% ( 51, 76 ] 115 0.006% 99.571% ( 76, 110 ] 92 0.005% 99.575% ( 110, 170 ] 452 0.024% 99.599% ( 170, 250 ] 650 0.034% 99.633% ( 250, 380 ] 897 0.047% 99.680% ( 380, 580 ] 1259 0.066% 99.746% ( 580, 870 ] 1657 0.087% 99.832% ( 870, 1300 ] 1794 0.094% 99.926% ( 1300, 1900 ] 1761 0.092% 100.018% ( 1900, 2900 ] 1767 0.092% 100.111% ( 2900, 4400 ] 1230 0.064% 100.175% ( 4400, 6600 ] 350 0.018% 100.193% ( 6600, 9900 ] 63 0.003% 100.197% ( 9900, 14000 ] 19 0.001% 100.198% ( 14000, 22000 ] 18 0.001% 100.199% ( 22000, 33000 ] 5 0.000% 100.199% ( 33000, 50000 ] 4 0.000% 100.199% ** Level 4 read latency histogram (micros): Count: 635239 Average: 27.4956 StdDev: 279.87 Min: 0 Median: 2.0343 Max: 48561 Percentiles: P50: 2.03 P75: 3.17 P99: 628.00 P99.9: 2908.08 P99.99: 3878.18 ------------------------------------------------------ [ 0, 1 ] 11497 1.810% 1.810% ( 1, 2 ] 300845 47.359% 49.169% ######### ( 2, 3 ] 153992 24.242% 73.411% ##### ( 3, 4 ] 60103 9.461% 82.872% ## ( 4, 6 ] 73382 11.552% 94.424% ## ( 6, 10 ] 16599 2.613% 97.037% # ( 10, 15 ] 4218 0.664% 97.701% ( 15, 22 ] 476 0.075% 97.776% ( 22, 34 ] 53 0.008% 97.784% ( 34, 51 ] 32 0.005% 97.789% ( 51, 76 ] 47 0.007% 97.797% ( 76, 110 ] 169 0.027% 97.823% ( 110, 170 ] 1919 0.302% 98.126% ( 170, 250 ] 2068 0.326% 98.451% ( 250, 380 ] 1458 0.230% 98.681% ( 380, 580 ] 1732 0.273% 98.953% ( 580, 870 ] 1792 0.282% 99.235% ( 870, 1300 ] 1732 0.273% 99.508% ( 1300, 1900 ] 1390 0.219% 99.727% ( 1900, 2900 ] 1095 0.172% 99.899% ( 2900, 4400 ] 884 0.139% 100.038% ( 4400, 6600 ] 282 0.044% 100.083% ( 6600, 9900 ] 67 0.011% 100.093% ( 9900, 14000 ] 6 0.001% 100.094% ( 14000, 22000 ] 13 0.002% 100.096% ( 22000, 33000 ] 11 0.002% 100.098% ( 33000, 50000 ] 1 0.000% 100.098% ** Level 5 read latency histogram (micros): Count: 61840 Average: 800.0307 StdDev: 1258.63 Min: 83 Median: 349.0286 Max: 44011 Percentiles: P50: 349.03 P75: 850.60 P99: 6000.45 P99.9: 9784.24 P99.99: 31152.00 ------------------------------------------------------ ( 76, 110 ] 1131 1.829% 1.829% ( 110, 170 ] 12970 20.973% 22.802% #### ( 170, 250 ] 11604 18.765% 41.567% #### ( 250, 380 ] 6846 11.071% 52.637% ## ( 380, 580 ] 7495 12.120% 64.757% ## ( 580, 870 ] 6788 10.977% 75.734% ## ( 870, 1300 ] 5181 8.378% 84.112% ## ( 1300, 1900 ] 3207 5.186% 89.298% # ( 1900, 2900 ] 2700 4.366% 93.664% # ( 2900, 4400 ] 2463 3.983% 97.647% # ( 4400, 6600 ] 1150 1.860% 99.507% ( 6600, 9900 ] 252 0.408% 99.914% ( 9900, 14000 ] 19 0.031% 99.945% ( 14000, 22000 ] 17 0.027% 99.973% ( 22000, 33000 ] 13 0.021% 99.994% ( 33000, 50000 ] 4 0.006% 100.000% ** DB Stats ** Uptime(secs): 17.4 total, 4.8 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 169.27 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.785 H:M:S, 33.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 206.53 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.066 H:M:S, 22.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/15-08:55:40 ... thread 0: (1000000,4000000) ops and (95417.0,144083.6) ops/second in (10.480318,27.761657) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/6 3.56 GB 282.3 4.0 0.0 4.0 7.7 3.7 0.0 2.1 114.6 221.9 35.71 26.13 64 0.558 4156K 11K 0.0 0.0 L1 1/1 46.89 MB 0.0 0.6 0.3 0.3 0.6 0.3 0.0 2.0 123.0 122.9 5.22 2.91 2 2.612 652K 685 0.0 0.0 L2 0/0 0.00 KB 0.0 0.5 0.4 0.2 0.5 0.4 0.0 1.5 125.3 125.2 4.44 2.66 2 2.218 564K 614 0.0 0.0 L3 18/0 987.14 MB 1.0 5.1 2.2 2.9 5.0 2.1 0.1 2.3 121.6 120.9 42.63 24.41 22 1.938 3431K 178 0.0 0.0 L4 172/0 9.60 GB 1.0 7.3 4.9 2.4 7.2 4.8 0.4 1.5 112.7 111.5 66.56 35.53 51 1.305 3284K 2527 0.0 0.0 L5 1140/0 93.75 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1346/7 107.92 GB 0.0 17.6 7.8 9.8 21.2 11.4 5.8 5.6 116.3 140.4 154.56 91.65 141 1.096 12M 15K 0.0 0.0 Int 0/0 0.00 KB 0.0 17.6 7.8 9.8 21.2 11.4 5.8 5.6 116.3 140.4 154.56 91.65 141 1.096 12M 15K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 17.6 7.8 9.8 17.4 7.6 0.0 0.0 133.0 132.1 135.18 80.88 88 1.536 12M 15K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 198.4 19.37 10.77 53 0.366 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 27.9 total, 27.8 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 21.19 GB write, 776.94 MB/s write, 17.56 GB read, 643.80 MB/s read, 154.6 seconds Interval compaction: 21.19 GB write, 781.42 MB/s write, 17.56 GB read, 647.52 MB/s read, 154.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 11, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 11, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 59, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 11, total-stops: 59, interval: 70 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1293799 Average: 20.3761 StdDev: 142.77 Min: 0 Median: 1.5867 Max: 34279 Percentiles: P50: 1.59 P75: 2.20 P99: 343.54 P99.9: 1189.35 P99.99: 2317.49 ------------------------------------------------------ [ 0, 1 ] 228309 17.646% 17.646% #### ( 1, 2 ] 713467 55.145% 72.792% ########### ( 2, 3 ] 142994 11.052% 83.844% ## ( 3, 4 ] 56642 4.378% 88.222% # ( 4, 6 ] 53561 4.140% 92.362% # ( 6, 10 ] 9379 0.725% 93.086% ( 10, 15 ] 2117 0.164% 93.250% ( 15, 22 ] 313 0.024% 93.274% ( 22, 34 ] 53 0.004% 93.278% ( 34, 51 ] 19 0.001% 93.280% ( 51, 76 ] 17 0.001% 93.281% ( 76, 110 ] 3476 0.269% 93.550% ( 110, 170 ] 39103 3.022% 96.572% # ( 170, 250 ] 26220 2.027% 98.599% ( 250, 380 ] 7214 0.558% 99.156% ( 380, 580 ] 5651 0.437% 99.593% ( 580, 870 ] 2926 0.226% 99.819% ( 870, 1300 ] 1406 0.109% 99.928% ( 1300, 1900 ] 616 0.048% 99.976% ( 1900, 2900 ] 447 0.035% 100.010% ( 2900, 4400 ] 320 0.025% 100.035% ( 4400, 6600 ] 101 0.008% 100.043% ( 6600, 9900 ] 15 0.001% 100.044% ( 9900, 14000 ] 7 0.001% 100.044% ( 14000, 22000 ] 8 0.001% 100.045% ( 22000, 33000 ] 5 0.000% 100.045% ( 33000, 50000 ] 1 0.000% 100.045% ** Level 1 read latency histogram (micros): Count: 187254 Average: 10.9489 StdDev: 207.15 Min: 1 Median: 2.0280 Max: 52428 Percentiles: P50: 2.03 P75: 3.10 P99: 12.81 P99.9: 2496.43 P99.99: 5669.74 ------------------------------------------------------ [ 0, 1 ] 3802 2.030% 2.030% ( 1, 2 ] 88532 47.279% 49.309% ######### ( 2, 3 ] 46206 24.676% 73.985% ##### ( 3, 4 ] 18807 10.044% 84.029% ## ( 4, 6 ] 22842 12.198% 96.227% ## ( 6, 10 ] 4511 2.409% 98.636% ( 10, 15 ] 1214 0.648% 99.284% ( 15, 22 ] 228 0.122% 99.406% ( 22, 34 ] 20 0.011% 99.417% ( 34, 51 ] 7 0.004% 99.421% ( 51, 76 ] 3 0.002% 99.422% ( 76, 110 ] 26 0.014% 99.436% ( 110, 170 ] 192 0.103% 99.539% ( 170, 250 ] 111 0.059% 99.598% ( 250, 380 ] 50 0.027% 99.625% ( 380, 580 ] 89 0.048% 99.672% ( 580, 870 ] 95 0.051% 99.723% ( 870, 1300 ] 133 0.071% 99.794% ( 1300, 1900 ] 123 0.066% 99.860% ( 1900, 2900 ] 127 0.068% 99.927% ( 2900, 4400 ] 104 0.056% 99.983% ( 4400, 6600 ] 23 0.012% 99.995% ( 6600, 9900 ] 9 0.005% 100.000% ( 22000, 33000 ] 2 0.001% 100.001% ( 50000, 75000 ] 1 0.001% 100.002% ** Level 2 read latency histogram (micros): Count: 555817 Average: 14.9748 StdDev: 193.34 Min: 1 Median: 1.9954 Max: 22061 Percentiles: P50: 2.00 P75: 3.09 P99: 13.52 P99.9: 2192.00 P99.99: 2989.51 ------------------------------------------------------ [ 0, 1 ] 13854 2.493% 2.493% ( 1, 2 ] 265274 47.727% 50.219% ########## ( 2, 3 ] 132709 23.876% 74.096% ##### ( 3, 4 ] 53852 9.689% 83.785% ## ( 4, 6 ] 64749 11.649% 95.434% ## ( 6, 10 ] 16321 2.936% 98.370% # ( 10, 15 ] 4972 0.895% 99.265% ( 15, 22 ] 795 0.143% 99.408% ( 22, 34 ] 88 0.016% 99.424% ( 34, 51 ] 21 0.004% 99.428% ( 51, 76 ] 29 0.005% 99.433% ( 76, 110 ] 21 0.004% 99.437% ( 110, 170 ] 117 0.021% 99.458% ( 170, 250 ] 197 0.035% 99.493% ( 250, 380 ] 225 0.040% 99.533% ( 380, 580 ] 344 0.062% 99.595% ( 580, 870 ] 446 0.080% 99.676% ( 870, 1300 ] 528 0.095% 99.771% ( 1300, 1900 ] 525 0.094% 99.865% ( 1900, 2900 ] 665 0.120% 99.985% ( 2900, 4400 ] 493 0.089% 100.073% ( 4400, 6600 ] 184 0.033% 100.107% ( 6600, 9900 ] 36 0.006% 100.113% ( 9900, 14000 ] 4 0.001% 100.114% ( 14000, 22000 ] 2 0.000% 100.114% ( 22000, 33000 ] 1 0.000% 100.114% ** Level 3 read latency histogram (micros): Count: 1915210 Average: 12.9402 StdDev: 179.23 Min: 1 Median: 2.0287 Max: 38953 Percentiles: P50: 2.03 P75: 2.99 P99: 12.87 P99.9: 1179.59 P99.99: 1715.60 ------------------------------------------------------ [ 0, 1 ] 28867 1.507% 1.507% ( 1, 2 ] 914486 47.749% 49.256% ########## ( 2, 3 ] 495949 25.895% 75.151% ##### ( 3, 4 ] 179959 9.396% 84.547% ## ( 4, 6 ] 218838 11.426% 95.974% ## ( 6, 10 ] 49983 2.610% 98.584% # ( 10, 15 ] 13887 0.725% 99.309% ( 15, 22 ] 1631 0.085% 99.394% ( 22, 34 ] 122 0.006% 99.400% ( 34, 51 ] 101 0.005% 99.405% ( 51, 76 ] 115 0.006% 99.411% ( 76, 110 ] 229 0.012% 99.423% ( 110, 170 ] 2071 0.108% 99.532% ( 170, 250 ] 1588 0.083% 99.614% ( 250, 380 ] 1096 0.057% 99.672% ( 380, 580 ] 1393 0.073% 99.744% ( 580, 870 ] 1686 0.088% 99.832% ( 870, 1300 ] 1797 0.094% 99.926% ( 1300, 1900 ] 1762 0.092% 100.018% ( 1900, 2900 ] 1768 0.092% 100.111% ( 2900, 4400 ] 1230 0.064% 100.175% ( 4400, 6600 ] 350 0.018% 100.193% ( 6600, 9900 ] 63 0.003% 100.196% ( 9900, 14000 ] 19 0.001% 100.197% ( 14000, 22000 ] 18 0.001% 100.198% ( 22000, 33000 ] 5 0.000% 100.199% ( 33000, 50000 ] 4 0.000% 100.199% ** Level 4 read latency histogram (micros): Count: 659848 Average: 32.8347 StdDev: 276.79 Min: 0 Median: 2.1129 Max: 48561 Percentiles: P50: 2.11 P75: 3.46 P99: 625.34 P99.9: 2881.03 P99.99: 3872.31 ------------------------------------------------------ [ 0, 1 ] 11497 1.742% 1.742% ( 1, 2 ] 300966 45.611% 47.354% ######### ( 2, 3 ] 154660 23.439% 70.793% ##### ( 3, 4 ] 60557 9.177% 79.970% ## ( 4, 6 ] 74366 11.270% 91.240% ## ( 6, 10 ] 16699 2.531% 93.771% # ( 10, 15 ] 4256 0.645% 94.416% ( 15, 22 ] 478 0.072% 94.488% ( 22, 34 ] 53 0.008% 94.496% ( 34, 51 ] 32 0.005% 94.501% ( 51, 76 ] 47 0.007% 94.508% ( 76, 110 ] 1260 0.191% 94.699% ( 110, 170 ] 13581 2.058% 96.757% ( 170, 250 ] 8900 1.349% 98.106% ( 250, 380 ] 2916 0.442% 98.548% ( 380, 580 ] 2666 0.404% 98.952% ( 580, 870 ] 2018 0.306% 99.258% ( 870, 1300 ] 1764 0.267% 99.525% ( 1300, 1900 ] 1394 0.211% 99.737% ( 1900, 2900 ] 1099 0.167% 99.903% ( 2900, 4400 ] 884 0.134% 100.037% ( 4400, 6600 ] 282 0.043% 100.080% ( 6600, 9900 ] 67 0.010% 100.090% ( 9900, 14000 ] 6 0.001% 100.091% ( 14000, 22000 ] 13 0.002% 100.093% ( 22000, 33000 ] 11 0.002% 100.095% ( 33000, 50000 ] 1 0.000% 100.095% ** Level 5 read latency histogram (micros): Count: 198368 Average: 379.4459 StdDev: 762.77 Min: 82 Median: 180.4112 Max: 44011 Percentiles: P50: 180.41 P75: 273.43 P99: 4078.16 P99.9: 7996.37 P99.99: 20665.04 ------------------------------------------------------ ( 76, 110 ] 7976 4.021% 4.021% # ( 110, 170 ] 84210 42.451% 46.472% ######## ( 170, 250 ] 53773 27.108% 73.580% ##### ( 250, 380 ] 15627 7.878% 81.458% ## ( 380, 580 ] 13224 6.666% 88.124% # ( 580, 870 ] 8296 4.182% 92.306% # ( 870, 1300 ] 5360 2.702% 95.008% # ( 1300, 1900 ] 3251 1.639% 96.647% ( 1900, 2900 ] 2732 1.377% 98.024% ( 2900, 4400 ] 2464 1.242% 99.267% ( 4400, 6600 ] 1150 0.580% 99.846% ( 6600, 9900 ] 252 0.127% 99.973% ( 9900, 14000 ] 19 0.010% 99.983% ( 14000, 22000 ] 17 0.009% 99.991% ( 22000, 33000 ] 13 0.007% 99.998% ( 33000, 50000 ] 4 0.002% 100.000% ** DB Stats ** Uptime(secs): 27.9 total, 10.5 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 140.98 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:12.550 H:M:S, 44.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 93.91 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:6.765 H:M:S, 64.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/07/15-08:55:45 ... thread 0: (1000000,5000000) ops and (199807.7,152595.0) ops/second in (5.004813,32.766470) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/6 4.54 GB 383.0 5.0 0.0 5.0 9.7 4.7 0.0 2.0 119.6 233.0 42.65 32.50 84 0.508 5180K 12K 0.0 0.0 L1 1/1 46.89 MB 0.0 0.6 0.3 0.3 0.6 0.3 0.0 2.0 123.0 122.9 5.22 2.91 2 2.612 652K 685 0.0 0.0 L2 0/0 0.00 KB 0.0 0.5 0.4 0.2 0.5 0.4 0.0 1.5 125.3 125.2 4.44 2.66 2 2.218 564K 614 0.0 0.0 L3 18/0 987.14 MB 1.0 5.1 2.2 2.9 5.0 2.1 0.1 2.3 121.6 120.9 42.63 24.41 22 1.938 3431K 178 0.0 0.0 L4 172/0 9.60 GB 1.0 7.3 4.9 2.4 7.2 4.8 0.4 1.5 112.7 111.5 66.56 35.53 51 1.305 3284K 2527 0.0 0.0 L5 1140/0 93.75 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1350/7 108.90 GB 0.0 18.5 7.8 10.8 23.2 12.4 5.8 4.9 117.6 146.8 161.50 98.01 161 1.003 13M 16K 0.0 0.0 Int 0/0 0.00 KB 0.0 18.5 7.8 10.8 23.2 12.4 5.8 4.9 117.6 146.8 161.50 98.01 161 1.003 13M 16K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 18.5 7.8 10.8 18.4 7.6 0.0 0.0 136.6 135.7 138.99 84.61 92 1.511 13M 16K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 215.5 22.51 13.40 69 0.326 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 32.9 total, 32.8 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 23.15 GB write, 720.04 MB/s write, 18.54 GB read, 576.56 MB/s read, 161.5 seconds Interval compaction: 23.15 GB write, 723.56 MB/s write, 18.54 GB read, 579.38 MB/s read, 161.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 17, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 17, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 59, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 17, total-stops: 59, interval: 76 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1611869 Average: 20.3280 StdDev: 131.61 Min: 0 Median: 1.5745 Max: 34279 Percentiles: P50: 1.57 P75: 2.14 P99: 334.22 P99.9: 1054.12 P99.99: 1871.43 ------------------------------------------------------ [ 0, 1 ] 294473 18.269% 18.269% #### ( 1, 2 ] 890248 55.231% 73.500% ########### ( 2, 3 ] 168815 10.473% 83.973% ## ( 3, 4 ] 68098 4.225% 88.198% # ( 4, 6 ] 60632 3.762% 91.959% # ( 6, 10 ] 10963 0.680% 92.640% ( 10, 15 ] 2582 0.160% 92.800% ( 15, 22 ] 453 0.028% 92.828% ( 22, 34 ] 90 0.006% 92.833% ( 34, 51 ] 22 0.001% 92.835% ( 51, 76 ] 19 0.001% 92.836% ( 76, 110 ] 4791 0.297% 93.133% ( 110, 170 ] 52693 3.269% 96.402% # ( 170, 250 ] 35708 2.215% 98.618% ( 250, 380 ] 9514 0.590% 99.208% ( 380, 580 ] 7111 0.441% 99.649% ( 580, 870 ] 3411 0.212% 99.861% ( 870, 1300 ] 1481 0.092% 99.953% ( 1300, 1900 ] 634 0.039% 99.992% ( 1900, 2900 ] 459 0.028% 100.020% ( 2900, 4400 ] 325 0.020% 100.041% ( 4400, 6600 ] 101 0.006% 100.047% ( 6600, 9900 ] 15 0.001% 100.048% ( 9900, 14000 ] 7 0.000% 100.048% ( 14000, 22000 ] 8 0.000% 100.049% ( 22000, 33000 ] 5 0.000% 100.049% ( 33000, 50000 ] 1 0.000% 100.049% ** Level 1 read latency histogram (micros): Count: 187400 Average: 11.0976 StdDev: 207.16 Min: 1 Median: 2.0296 Max: 52428 Percentiles: P50: 2.03 P75: 3.11 P99: 13.40 P99.9: 2495.28 P99.99: 5668.35 ------------------------------------------------------ [ 0, 1 ] 3802 2.029% 2.029% ( 1, 2 ] 88532 47.242% 49.271% ######### ( 2, 3 ] 46206 24.656% 73.927% ##### ( 3, 4 ] 18807 10.036% 83.963% ## ( 4, 6 ] 22842 12.189% 96.152% ## ( 6, 10 ] 4511 2.407% 98.559% ( 10, 15 ] 1214 0.648% 99.207% ( 15, 22 ] 228 0.122% 99.329% ( 22, 34 ] 20 0.011% 99.339% ( 34, 51 ] 7 0.004% 99.343% ( 51, 76 ] 3 0.002% 99.345% ( 76, 110 ] 34 0.018% 99.363% ( 110, 170 ] 253 0.135% 99.498% ( 170, 250 ] 163 0.087% 99.585% ( 250, 380 ] 65 0.035% 99.620% ( 380, 580 ] 95 0.051% 99.670% ( 580, 870 ] 99 0.053% 99.723% ( 870, 1300 ] 133 0.071% 99.794% ( 1300, 1900 ] 123 0.066% 99.860% ( 1900, 2900 ] 127 0.068% 99.927% ( 2900, 4400 ] 104 0.055% 99.983% ( 4400, 6600 ] 23 0.012% 99.995% ( 6600, 9900 ] 9 0.005% 100.000% ( 22000, 33000 ] 2 0.001% 100.001% ( 50000, 75000 ] 1 0.001% 100.002% ** Level 2 read latency histogram (micros): Count: 555891 Average: 14.9993 StdDev: 193.34 Min: 1 Median: 1.9955 Max: 22061 Percentiles: P50: 2.00 P75: 3.09 P99: 13.59 P99.9: 2191.89 P99.99: 2989.49 ------------------------------------------------------ [ 0, 1 ] 13854 2.492% 2.492% ( 1, 2 ] 265274 47.721% 50.213% ########## ( 2, 3 ] 132709 23.873% 74.086% ##### ( 3, 4 ] 53852 9.688% 83.773% ## ( 4, 6 ] 64749 11.648% 95.421% ## ( 6, 10 ] 16321 2.936% 98.357% # ( 10, 15 ] 4972 0.894% 99.252% ( 15, 22 ] 795 0.143% 99.395% ( 22, 34 ] 88 0.016% 99.410% ( 34, 51 ] 21 0.004% 99.414% ( 51, 76 ] 29 0.005% 99.419% ( 76, 110 ] 24 0.004% 99.424% ( 110, 170 ] 154 0.028% 99.452% ( 170, 250 ] 221 0.040% 99.491% ( 250, 380 ] 230 0.041% 99.533% ( 380, 580 ] 347 0.062% 99.595% ( 580, 870 ] 448 0.081% 99.676% ( 870, 1300 ] 528 0.095% 99.771% ( 1300, 1900 ] 525 0.094% 99.865% ( 1900, 2900 ] 665 0.120% 99.985% ( 2900, 4400 ] 493 0.089% 100.073% ( 4400, 6600 ] 184 0.033% 100.106% ( 6600, 9900 ] 36 0.006% 100.113% ( 9900, 14000 ] 4 0.001% 100.114% ( 14000, 22000 ] 2 0.000% 100.114% ( 22000, 33000 ] 1 0.000% 100.114% ** Level 3 read latency histogram (micros): Count: 1916685 Average: 13.0826 StdDev: 179.26 Min: 1 Median: 2.0302 Max: 38953 Percentiles: P50: 2.03 P75: 3.00 P99: 13.40 P99.9: 1179.37 P99.99: 1715.55 ------------------------------------------------------ [ 0, 1 ] 28867 1.506% 1.506% ( 1, 2 ] 914486 47.712% 49.218% ########## ( 2, 3 ] 495949 25.875% 75.093% ##### ( 3, 4 ] 179959 9.389% 84.482% ## ( 4, 6 ] 218838 11.418% 95.900% ## ( 6, 10 ] 49983 2.608% 98.508% # ( 10, 15 ] 13887 0.725% 99.232% ( 15, 22 ] 1631 0.085% 99.317% ( 22, 34 ] 122 0.006% 99.324% ( 34, 51 ] 101 0.005% 99.329% ( 51, 76 ] 115 0.006% 99.335% ( 76, 110 ] 299 0.016% 99.351% ( 110, 170 ] 2779 0.145% 99.496% ( 170, 250 ] 2061 0.108% 99.603% ( 250, 380 ] 1212 0.063% 99.666% ( 380, 580 ] 1475 0.077% 99.743% ( 580, 870 ] 1710 0.089% 99.832% ( 870, 1300 ] 1799 0.094% 99.926% ( 1300, 1900 ] 1762 0.092% 100.018% ( 1900, 2900 ] 1768 0.092% 100.111% ( 2900, 4400 ] 1230 0.064% 100.175% ( 4400, 6600 ] 350 0.018% 100.193% ( 6600, 9900 ] 63 0.003% 100.196% ( 9900, 14000 ] 19 0.001% 100.197% ( 14000, 22000 ] 18 0.001% 100.198% ( 22000, 33000 ] 5 0.000% 100.198% ( 33000, 50000 ] 4 0.000% 100.199% ** Level 4 read latency histogram (micros): Count: 671157 Average: 35.5859 StdDev: 275.79 Min: 0 Median: 2.1493 Max: 48561 Percentiles: P50: 2.15 P75: 3.60 P99: 635.95 P99.9: 2872.68 P99.99: 3871.59 ------------------------------------------------------ [ 0, 1 ] 11497 1.713% 1.713% ( 1, 2 ] 300976 44.844% 46.557% ######### ( 2, 3 ] 154719 23.053% 69.610% ##### ( 3, 4 ] 60587 9.027% 78.637% ## ( 4, 6 ] 74428 11.090% 89.727% ## ( 6, 10 ] 16710 2.490% 92.216% ( 10, 15 ] 4261 0.635% 92.851% ( 15, 22 ] 478 0.071% 92.923% ( 22, 34 ] 53 0.008% 92.930% ( 34, 51 ] 32 0.005% 92.935% ( 51, 76 ] 47 0.007% 92.942% ( 76, 110 ] 1795 0.267% 93.210% ( 110, 170 ] 19123 2.849% 96.059% # ( 170, 250 ] 12334 1.838% 97.897% ( 250, 380 ] 3773 0.562% 98.459% ( 380, 580 ] 3208 0.478% 98.937% ( 580, 870 ] 2200 0.328% 99.265% ( 870, 1300 ] 1794 0.267% 99.532% ( 1300, 1900 ] 1397 0.208% 99.740% ( 1900, 2900 ] 1104 0.164% 99.904% ( 2900, 4400 ] 886 0.132% 100.037% ( 4400, 6600 ] 282 0.042% 100.079% ( 6600, 9900 ] 67 0.010% 100.089% ( 9900, 14000 ] 6 0.001% 100.089% ( 14000, 22000 ] 13 0.002% 100.091% ( 22000, 33000 ] 11 0.002% 100.093% ( 33000, 50000 ] 1 0.000% 100.093% ** Level 5 read latency histogram (micros): Count: 260461 Average: 336.2254 StdDev: 672.74 Min: 81 Median: 174.9885 Max: 44011 Percentiles: P50: 174.99 P75: 246.00 P99: 3699.51 P99.9: 7157.06 P99.99: 16801.84 ------------------------------------------------------ ( 76, 110 ] 11174 4.290% 4.290% # ( 110, 170 ] 114482 43.954% 48.244% ######### ( 170, 250 ] 73360 28.165% 76.409% ###### ( 250, 380 ] 20415 7.838% 84.247% ## ( 380, 580 ] 16334 6.271% 90.518% # ( 580, 870 ] 9262 3.556% 94.074% # ( 870, 1300 ] 5483 2.105% 96.179% ( 1300, 1900 ] 3280 1.259% 97.439% ( 1900, 2900 ] 2752 1.057% 98.495% ( 2900, 4400 ] 2466 0.947% 99.442% ( 4400, 6600 ] 1150 0.442% 99.884% ( 6600, 9900 ] 252 0.097% 99.980% ( 9900, 14000 ] 19 0.007% 99.988% ( 14000, 22000 ] 17 0.007% 99.994% ( 22000, 33000 ] 13 0.005% 99.999% ( 33000, 50000 ] 4 0.002% 100.001% ** DB Stats ** Uptime(secs): 32.9 total, 5.0 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 149.44 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:13.963 H:M:S, 42.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 196.65 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.413 H:M:S, 28.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/07/15-08:55:53 ... thread 0: (1000000,6000000) ops and (135089.3,149369.0) ops/second in (7.402511,40.168981) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/4 4.66 GB 451.5 7.0 0.0 7.0 12.7 5.6 0.0 2.2 134.7 243.0 53.30 42.57 103 0.517 7289K 20K 0.0 0.0 L1 10/0 862.15 MB 86.2 1.5 1.1 0.4 1.5 1.1 0.0 1.3 44.5 44.3 33.92 9.68 3 11.305 1531K 3764 0.0 0.0 L2 0/0 0.00 KB 0.0 0.5 0.4 0.2 0.5 0.4 0.0 1.5 125.3 125.2 4.44 2.66 2 2.218 564K 614 0.0 0.0 L3 18/0 987.14 MB 1.0 5.1 2.2 2.9 5.0 2.1 0.1 2.3 121.6 120.9 42.63 24.41 22 1.938 3431K 178 0.0 0.0 L4 172/0 9.60 GB 1.0 7.3 4.9 2.4 7.2 4.8 0.4 1.5 112.7 111.5 66.56 35.53 51 1.305 3284K 2527 0.0 0.0 L5 1140/0 93.75 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1354/4 109.81 GB 0.0 21.4 8.6 12.8 26.9 14.1 5.8 4.8 109.2 137.4 200.84 114.85 181 1.110 16M 27K 0.0 0.0 Int 0/0 0.00 KB 0.0 21.4 8.6 12.8 26.9 14.1 5.8 4.8 109.2 137.4 200.84 114.85 181 1.110 16M 27K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 21.4 8.6 12.8 21.3 8.4 0.0 0.0 125.0 124.2 175.44 99.00 97 1.809 16M 27K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 228.2 25.40 15.84 84 0.302 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 40.3 total, 40.2 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 26.94 GB write, 684.01 MB/s write, 21.41 GB read, 543.71 MB/s read, 200.8 seconds Interval compaction: 26.94 GB write, 686.74 MB/s write, 21.41 GB read, 545.88 MB/s read, 200.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 23, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 23, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 59, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 23, total-stops: 59, interval: 82 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2228521 Average: 18.6594 StdDev: 116.08 Min: 0 Median: 1.5377 Max: 34279 Percentiles: P50: 1.54 P75: 1.98 P99: 302.76 P99.9: 856.21 P99.99: 1553.16 ------------------------------------------------------ [ 0, 1 ] 438524 19.678% 19.678% #### ( 1, 2 ] 1256798 56.396% 76.074% ########### ( 2, 3 ] 211503 9.491% 85.565% ## ( 3, 4 ] 81680 3.665% 89.230% # ( 4, 6 ] 68943 3.094% 92.323% # ( 6, 10 ] 12544 0.563% 92.886% ( 10, 15 ] 3259 0.146% 93.033% ( 15, 22 ] 540 0.024% 93.057% ( 22, 34 ] 93 0.004% 93.061% ( 34, 51 ] 23 0.001% 93.062% ( 51, 76 ] 25 0.001% 93.063% ( 76, 110 ] 6989 0.314% 93.377% ( 110, 170 ] 72483 3.253% 96.629% # ( 170, 250 ] 47825 2.146% 98.775% ( 250, 380 ] 12340 0.554% 99.329% ( 380, 580 ] 8928 0.401% 99.730% ( 580, 870 ] 3986 0.179% 99.909% ( 870, 1300 ] 1544 0.069% 99.978% ( 1300, 1900 ] 645 0.029% 100.007% ( 1900, 2900 ] 483 0.022% 100.028% ( 2900, 4400 ] 337 0.015% 100.044% ( 4400, 6600 ] 101 0.005% 100.048% ( 6600, 9900 ] 15 0.001% 100.049% ( 9900, 14000 ] 7 0.000% 100.049% ( 14000, 22000 ] 8 0.000% 100.049% ( 22000, 33000 ] 5 0.000% 100.050% ( 33000, 50000 ] 1 0.000% 100.050% ** Level 1 read latency histogram (micros): Count: 188289 Average: 11.2840 StdDev: 206.85 Min: 0 Median: 2.0372 Max: 52428 Percentiles: P50: 2.04 P75: 3.12 P99: 14.17 P99.9: 2488.28 P99.99: 5659.84 ------------------------------------------------------ [ 0, 1 ] 3826 2.032% 2.032% ( 1, 2 ] 88592 47.051% 49.083% ######### ( 2, 3 ] 46455 24.672% 73.755% ##### ( 3, 4 ] 18933 10.055% 83.811% ## ( 4, 6 ] 23037 12.235% 96.045% ## ( 6, 10 ] 4547 2.415% 98.460% ( 10, 15 ] 1218 0.647% 99.107% ( 15, 22 ] 228 0.121% 99.228% ( 22, 34 ] 20 0.011% 99.239% ( 34, 51 ] 7 0.004% 99.243% ( 51, 76 ] 3 0.002% 99.244% ( 76, 110 ] 43 0.023% 99.267% ( 110, 170 ] 352 0.187% 99.454% ( 170, 250 ] 214 0.114% 99.568% ( 250, 380 ] 84 0.045% 99.612% ( 380, 580 ] 104 0.055% 99.668% ( 580, 870 ] 105 0.056% 99.723% ( 870, 1300 ] 134 0.071% 99.794% ( 1300, 1900 ] 124 0.066% 99.860% ( 1900, 2900 ] 127 0.067% 99.928% ( 2900, 4400 ] 104 0.055% 99.983% ( 4400, 6600 ] 23 0.012% 99.995% ( 6600, 9900 ] 9 0.005% 100.000% ( 22000, 33000 ] 2 0.001% 100.001% ( 50000, 75000 ] 1 0.001% 100.002% ** Level 2 read latency histogram (micros): Count: 555994 Average: 15.0387 StdDev: 193.36 Min: 1 Median: 1.9957 Max: 22061 Percentiles: P50: 2.00 P75: 3.10 P99: 13.70 P99.9: 2191.74 P99.99: 2989.45 ------------------------------------------------------ [ 0, 1 ] 13854 2.492% 2.492% ( 1, 2 ] 265274 47.712% 50.203% ########## ( 2, 3 ] 132709 23.869% 74.072% ##### ( 3, 4 ] 53852 9.686% 83.758% ## ( 4, 6 ] 64749 11.646% 95.404% ## ( 6, 10 ] 16321 2.935% 98.339% # ( 10, 15 ] 4972 0.894% 99.233% ( 15, 22 ] 795 0.143% 99.376% ( 22, 34 ] 88 0.016% 99.392% ( 34, 51 ] 21 0.004% 99.396% ( 51, 76 ] 29 0.005% 99.401% ( 76, 110 ] 30 0.005% 99.406% ( 110, 170 ] 196 0.035% 99.442% ( 170, 250 ] 258 0.046% 99.488% ( 250, 380 ] 236 0.042% 99.531% ( 380, 580 ] 355 0.064% 99.594% ( 580, 870 ] 450 0.081% 99.675% ( 870, 1300 ] 529 0.095% 99.771% ( 1300, 1900 ] 526 0.095% 99.865% ( 1900, 2900 ] 665 0.120% 99.985% ( 2900, 4400 ] 493 0.089% 100.073% ( 4400, 6600 ] 184 0.033% 100.106% ( 6600, 9900 ] 36 0.006% 100.113% ( 9900, 14000 ] 4 0.001% 100.114% ( 14000, 22000 ] 2 0.000% 100.114% ( 22000, 33000 ] 1 0.000% 100.114% ** Level 3 read latency histogram (micros): Count: 1918840 Average: 13.2906 StdDev: 179.32 Min: 1 Median: 2.0324 Max: 38953 Percentiles: P50: 2.03 P75: 3.00 P99: 14.17 P99.9: 1179.43 P99.99: 1715.82 ------------------------------------------------------ [ 0, 1 ] 28867 1.504% 1.504% ( 1, 2 ] 914486 47.658% 49.163% ########## ( 2, 3 ] 495949 25.846% 75.009% ##### ( 3, 4 ] 179959 9.379% 84.387% ## ( 4, 6 ] 218838 11.405% 95.792% ## ( 6, 10 ] 49983 2.605% 98.397% # ( 10, 15 ] 13887 0.724% 99.121% ( 15, 22 ] 1631 0.085% 99.206% ( 22, 34 ] 122 0.006% 99.212% ( 34, 51 ] 101 0.005% 99.217% ( 51, 76 ] 115 0.006% 99.223% ( 76, 110 ] 419 0.022% 99.245% ( 110, 170 ] 3844 0.200% 99.446% ( 170, 250 ] 2708 0.141% 99.587% ( 250, 380 ] 1384 0.072% 99.659% ( 380, 580 ] 1588 0.083% 99.742% ( 580, 870 ] 1742 0.091% 99.832% ( 870, 1300 ] 1804 0.094% 99.926% ( 1300, 1900 ] 1762 0.092% 100.018% ( 1900, 2900 ] 1768 0.092% 100.110% ( 2900, 4400 ] 1231 0.064% 100.174% ( 4400, 6600 ] 350 0.018% 100.193% ( 6600, 9900 ] 63 0.003% 100.196% ( 9900, 14000 ] 19 0.001% 100.197% ( 14000, 22000 ] 18 0.001% 100.198% ( 22000, 33000 ] 5 0.000% 100.198% ( 33000, 50000 ] 4 0.000% 100.198% ** Level 4 read latency histogram (micros): Count: 687514 Average: 39.4790 StdDev: 274.28 Min: 0 Median: 2.2022 Max: 48561 Percentiles: P50: 2.20 P75: 3.80 P99: 650.18 P99.9: 2858.02 P99.99: 3868.82 ------------------------------------------------------ [ 0, 1 ] 11497 1.672% 1.672% ( 1, 2 ] 300976 43.777% 45.450% ######### ( 2, 3 ] 154719 22.504% 67.954% ##### ( 3, 4 ] 60587 8.812% 76.766% ## ( 4, 6 ] 74428 10.826% 87.592% ## ( 6, 10 ] 16710 2.430% 90.022% ( 10, 15 ] 4261 0.620% 90.642% ( 15, 22 ] 478 0.070% 90.712% ( 22, 34 ] 53 0.008% 90.719% ( 34, 51 ] 32 0.005% 90.724% ( 51, 76 ] 47 0.007% 90.731% ( 76, 110 ] 2685 0.391% 91.121% ( 110, 170 ] 27244 3.963% 95.084% # ( 170, 250 ] 17218 2.504% 97.589% # ( 250, 380 ] 5005 0.728% 98.317% ( 380, 580 ] 4096 0.596% 98.912% ( 580, 870 ] 2491 0.362% 99.275% ( 870, 1300 ] 1834 0.267% 99.541% ( 1300, 1900 ] 1404 0.204% 99.746% ( 1900, 2900 ] 1108 0.161% 99.907% ( 2900, 4400 ] 886 0.129% 100.036% ( 4400, 6600 ] 282 0.041% 100.077% ( 6600, 9900 ] 67 0.010% 100.086% ( 9900, 14000 ] 6 0.001% 100.087% ( 14000, 22000 ] 13 0.002% 100.089% ( 22000, 33000 ] 11 0.002% 100.091% ( 33000, 50000 ] 1 0.000% 100.091% ** Level 5 read latency histogram (micros): Count: 351035 Average: 300.9483 StdDev: 586.03 Min: 81 Median: 170.1272 Max: 44011 Percentiles: P50: 170.13 P75: 240.13 P99: 3151.10 P99.9: 6508.11 P99.99: 13330.30 ------------------------------------------------------ ( 76, 110 ] 16363 4.661% 4.661% # ( 110, 170 ] 158995 45.293% 49.955% ######### ( 170, 250 ] 100295 28.571% 78.526% ###### ( 250, 380 ] 27519 7.839% 86.365% ## ( 380, 580 ] 21318 6.073% 92.438% # ( 580, 870 ] 10850 3.091% 95.529% # ( 870, 1300 ] 5688 1.620% 97.149% ( 1300, 1900 ] 3302 0.941% 98.090% ( 1900, 2900 ] 2781 0.792% 98.882% ( 2900, 4400 ] 2471 0.704% 99.586% ( 4400, 6600 ] 1150 0.328% 99.914% ( 6600, 9900 ] 252 0.072% 99.985% ( 9900, 14000 ] 19 0.005% 99.991% ( 14000, 22000 ] 17 0.005% 99.996% ( 22000, 33000 ] 13 0.004% 99.999% ( 33000, 50000 ] 4 0.001% 100.001% ** DB Stats ** Uptime(secs): 40.3 total, 7.4 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 146.42 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:17.731 H:M:S, 44.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 132.95 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.768 H:M:S, 50.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 287.543 micros/op 13891 ops/sec 60.182 seconds 835996 operations; 13.5 MB/s (206999 of 206999 found) Microseconds per read: Count: 835996 Average: 287.5452 StdDev: 913.70 Min: 2 Median: 203.3574 Max: 130698 Percentiles: P50: 203.36 P75: 247.27 P99: 1489.38 P99.9: 16917.42 P99.99: 32565.53 ------------------------------------------------------ ( 1, 2 ] 8 0.001% 0.001% ( 2, 3 ] 44 0.005% 0.006% ( 3, 4 ] 50 0.006% 0.012% ( 4, 6 ] 86 0.010% 0.022% ( 6, 10 ] 240 0.029% 0.051% ( 10, 15 ] 3040 0.364% 0.415% ( 15, 22 ] 4466 0.534% 0.949% ( 22, 34 ] 9494 1.136% 2.085% ( 34, 51 ] 2981 0.357% 2.441% ( 51, 76 ] 392 0.047% 2.488% ( 76, 110 ] 2134 0.255% 2.743% ( 110, 170 ] 236290 28.264% 31.008% ###### ( 170, 250 ] 380780 45.548% 76.556% ######### ( 250, 380 ] 101646 12.159% 88.715% ## ( 380, 580 ] 54555 6.526% 95.240% # ( 580, 870 ] 21754 2.602% 97.843% # ( 870, 1300 ] 8571 1.025% 98.868% ( 1300, 1900 ] 3501 0.419% 99.287% ( 1900, 2900 ] 2326 0.278% 99.565% ( 2900, 4400 ] 1728 0.207% 99.772% ( 4400, 6600 ] 486 0.058% 99.830% ( 6600, 9900 ] 111 0.013% 99.843% ( 9900, 14000 ] 221 0.026% 99.869% ( 14000, 22000 ] 702 0.084% 99.953% ( 22000, 33000 ] 319 0.038% 99.992% ( 33000, 50000 ] 49 0.006% 99.997% ( 50000, 75000 ] 20 0.002% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 13787632 rocksdb.bloom.filter.full.positive COUNT : 969570 rocksdb.bloom.filter.full.true.positive COUNT : 835620 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 376 rocksdb.memtable.miss COUNT : 835620 rocksdb.l0.hit COUNT : 35760 rocksdb.l1.hit COUNT : 4015 rocksdb.l2andup.hit COUNT : 795845 rocksdb.compaction.key.drop.new COUNT : 159931 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 6194045 rocksdb.number.keys.read COUNT : 835996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 6392254440 rocksdb.bytes.read COUNT : 835996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1679 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 36661200 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 24344584682 rocksdb.compact.write.bytes COUNT : 24177493237 rocksdb.flush.write.bytes COUNT : 6341633013 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 20 rocksdb.number.superversion_acquires COUNT : 1198 rocksdb.number.superversion_releases COUNT : 1171 rocksdb.number.superversion_cleanups COUNT : 303 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 327 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 6290373632 rocksdb.memtable.garbage.bytes.at.flush COUNT : 2332672 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 28366681691 rocksdb.non.last.level.read.count COUNT : 6710961 rocksdb.block.checksum.compute.count COUNT : 6875872 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 202.546214 P95 : 572.135673 P99 : 1488.161373 P100 : 130696.000000 COUNT : 835996 SUM : 239695375 rocksdb.db.write.micros P50 : 1.542684 P95 : 3.997026 P99 : 8.470107 P100 : 751880.000000 COUNT : 6194045 SUM : 51874668 rocksdb.compaction.times.micros P50 : 1316666.666667 P95 : 3635000.000000 P99 : 22110000.000000 P100 : 28691106.000000 COUNT : 99 SUM : 177363702 rocksdb.compaction.times.cpu_micros P50 : 850333.333333 P95 : 2935000.000000 P99 : 6533000.000000 P100 : 6767412.000000 COUNT : 99 SUM : 100892898 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 832.173913 P95 : 28600.000000 P99 : 179600.000000 P100 : 197239.000000 COUNT : 88 SUM : 594556 rocksdb.compaction.outfile.sync.micros P50 : 13487.500000 P95 : 180909.090909 P99 : 341000.000000 P100 : 501772.000000 COUNT : 290 SUM : 11954396 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 238.750000 P95 : 4175.000000 P99 : 6305.750000 P100 : 7621.000000 COUNT : 207 SUM : 199263 rocksdb.table.open.io.micros P50 : 11505.226824 P95 : 21593.205945 P99 : 30056.101695 P100 : 71024.000000 COUNT : 1679 SUM : 18352523 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.884797 P95 : 10.799069 P99 : 223.430507 P100 : 52432.000000 COUNT : 5895231 SUM : 100102393 rocksdb.read.block.get.micros P50 : 157.587877 P95 : 474.642411 P99 : 1464.640000 P100 : 44016.000000 COUNT : 978960 SUM : 206291757 rocksdb.write.raw.block.micros P50 : 0.526991 P95 : 1.037194 P99 : 3.475634 P100 : 58661.000000 COUNT : 7391923 SUM : 22901629 rocksdb.numfiles.in.singlecompaction P50 : 2.800000 P95 : 5.666667 P99 : 14.000000 P100 : 21.000000 COUNT : 120 SUM : 419 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.567179 P95 : 1236.400441 P99 : 2639.636414 P100 : 751730.000000 COUNT : 187786 SUM : 36675095 rocksdb.sst.read.micros P50 : 1.965267 P95 : 198.832462 P99 : 520.027003 P100 : 52428.000000 COUNT : 6710991 SUM : 291320332 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 835996 SUM : 835996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 6194045 SUM : 6392254440 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 224153.846154 P95 : 900800.000000 P99 : 979072.000000 P100 : 979072.000000 COUNT : 88 SUM : 26213835 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 328 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0