[2024-07-16 01:04:28.730299] Starting SPDK v24.09-pre git sha1 fd0bbcfdd / DPDK 24.03.0 initialization... [2024-07-16 01:04:28.730620] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2603775 ] Set seed to 1721084668723047 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-16 01:04:28.876175] app.c: 914:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-16 01:04:29.103177] app.c: 607:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-16 01:04:29.103231] app.c: 608:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2603775' to capture a snapshot of events at runtime. [2024-07-16 01:04:29.103247] app.c: 613:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-16 01:04:29.103261] app.c: 614:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-16 01:04:29.103271] app.c: 615:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2603775 for offline analysis/debug. [2024-07-16 01:04:29.103370] reactor.c: 941:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Tue Jul 16 01:04:32 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/07/16-01:04:39 ... thread 0: (1000000,1000000) ops and (145128.8,145128.8) ops/second in (6.890431,6.890431) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/7 817.24 MB 25.1 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 132.7 6.16 2.55 10 0.616 0 0 0.0 0.0 L1 4/4 282.11 MB 0.0 0.3 0.2 0.1 0.3 0.2 0.0 1.3 101.7 101.1 2.79 1.53 1 2.791 288K 1571 0.0 0.0 L2 14/13 1.75 GB 1.3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 31/8 1.76 GB 1.3 2.5 1.8 0.7 2.5 1.7 0.0 1.4 101.6 97.9 25.64 14.18 11 2.331 251K 0 0.0 0.0 L4 140/23 9.90 GB 0.8 1.3 0.2 1.0 1.1 0.0 0.6 4.4 124.3 103.7 10.64 6.20 4 2.659 254K 0 0.0 0.0 L5 124/0 9.41 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 323/55 23.90 GB 0.0 4.1 2.3 1.8 4.6 2.8 1.0 5.8 93.1 104.2 45.22 24.45 26 1.739 793K 1571 0.0 0.0 Int 0/0 0.00 KB 0.0 4.1 2.3 1.8 4.6 2.8 1.0 5.8 93.1 104.2 45.22 24.45 26 1.739 793K 1571 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.1 2.3 1.8 3.8 2.0 0.0 0.0 107.8 99.7 39.07 21.90 16 2.442 793K 1571 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 132.7 6.16 2.55 10 0.616 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.0 total, 6.9 interval Flush(GB): cumulative 0.798, interval 0.798 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 4.60 GB write, 674.94 MB/s write, 4.11 GB read, 603.33 MB/s read, 45.2 seconds Interval compaction: 4.60 GB write, 683.38 MB/s write, 4.11 GB read, 610.87 MB/s read, 45.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 12, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 12, interval: 12 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 117007 Average: 49.0264 StdDev: 586.03 Min: 1 Median: 2.2972 Max: 72380 Percentiles: P50: 2.30 P75: 3.44 P99: 1050.24 P99.9: 4647.67 P99.99: 17649.65 ------------------------------------------------------ [ 0, 1 ] 2062 1.762% 1.762% ( 1, 2 ] 46202 39.487% 41.249% ######## ( 2, 3 ] 34449 29.442% 70.691% ###### ( 3, 4 ] 11565 9.884% 80.575% ## ( 4, 6 ] 13747 11.749% 92.324% ## ( 6, 10 ] 3753 3.208% 95.531% # ( 10, 15 ] 730 0.624% 96.155% ( 15, 22 ] 117 0.100% 96.255% ( 22, 34 ] 15 0.013% 96.268% ( 34, 51 ] 11 0.009% 96.277% ( 51, 76 ] 12 0.010% 96.287% ( 76, 110 ] 5 0.004% 96.292% ( 110, 170 ] 51 0.044% 96.335% ( 170, 250 ] 237 0.203% 96.538% ( 250, 380 ] 622 0.532% 97.069% ( 380, 580 ] 1107 0.946% 98.016% ( 580, 870 ] 939 0.803% 98.818% ( 870, 1300 ] 508 0.434% 99.252% ( 1300, 1900 ] 193 0.165% 99.417% ( 1900, 2900 ] 269 0.230% 99.647% ( 2900, 4400 ] 288 0.246% 99.893% ( 4400, 6600 ] 71 0.061% 99.954% ( 6600, 9900 ] 27 0.023% 99.977% ( 9900, 14000 ] 8 0.007% 99.984% ( 14000, 22000 ] 16 0.014% 99.997% ( 22000, 33000 ] 4 0.003% 100.001% ( 33000, 50000 ] 1 0.001% 100.002% ( 50000, 75000 ] 5 0.004% 100.006% ** Level 1 read latency histogram (micros): Count: 66973 Average: 20.7877 StdDev: 391.14 Min: 1 Median: 2.3734 Max: 44471 Percentiles: P50: 2.37 P75: 3.49 P99: 397.58 P99.9: 3338.34 P99.99: 18917.37 ------------------------------------------------------ [ 0, 1 ] 699 1.044% 1.044% ( 1, 2 ] 24967 37.279% 38.323% ####### ( 2, 3 ] 20944 31.272% 69.595% ###### ( 3, 4 ] 7409 11.063% 80.658% ## ( 4, 6 ] 9199 13.735% 94.393% ### ( 6, 10 ] 2369 3.537% 97.931% # ( 10, 15 ] 421 0.629% 98.559% ( 15, 22 ] 41 0.061% 98.620% ( 22, 34 ] 9 0.013% 98.634% ( 34, 51 ] 24 0.036% 98.670% ( 51, 76 ] 12 0.018% 98.688% ( 76, 110 ] 5 0.007% 98.695% ( 110, 170 ] 11 0.016% 98.711% ( 170, 250 ] 46 0.069% 98.780% ( 250, 380 ] 126 0.188% 98.968% ( 380, 580 ] 242 0.361% 99.330% ( 580, 870 ] 194 0.290% 99.619% ( 870, 1300 ] 89 0.133% 99.752% ( 1300, 1900 ] 35 0.052% 99.804% ( 1900, 2900 ] 50 0.075% 99.879% ( 2900, 4400 ] 48 0.072% 99.951% ( 4400, 6600 ] 9 0.013% 99.964% ( 6600, 9900 ] 11 0.016% 99.981% ( 9900, 14000 ] 2 0.003% 99.984% ( 14000, 22000 ] 7 0.010% 99.994% ( 22000, 33000 ] 1 0.001% 99.996% ( 33000, 50000 ] 3 0.004% 100.000% ** Level 2 read latency histogram (micros): Count: 614036 Average: 19.3860 StdDev: 388.53 Min: 1 Median: 2.3775 Max: 69191 Percentiles: P50: 2.38 P75: 3.44 P99: 14.04 P99.9: 1409.38 P99.99: 2519.29 ------------------------------------------------------ [ 0, 1 ] 4405 0.717% 0.717% ( 1, 2 ] 227886 37.113% 37.830% ####### ( 2, 3 ] 197944 32.237% 70.067% ###### ( 3, 4 ] 69462 11.312% 81.379% ## ( 4, 6 ] 84511 13.763% 95.142% ### ( 6, 10 ] 20055 3.266% 98.408% # ( 10, 15 ] 4498 0.733% 99.141% ( 15, 22 ] 591 0.096% 99.237% ( 22, 34 ] 104 0.017% 99.254% ( 34, 51 ] 239 0.039% 99.293% ( 51, 76 ] 179 0.029% 99.322% ( 76, 110 ] 56 0.009% 99.331% ( 110, 170 ] 90 0.015% 99.346% ( 170, 250 ] 235 0.038% 99.384% ( 250, 380 ] 725 0.118% 99.502% ( 380, 580 ] 1131 0.184% 99.687% ( 580, 870 ] 825 0.134% 99.821% ( 870, 1300 ] 431 0.070% 99.891% ( 1300, 1900 ] 307 0.050% 99.941% ( 1900, 2900 ] 487 0.079% 100.020% ( 2900, 4400 ] 521 0.085% 100.105% ( 4400, 6600 ] 197 0.032% 100.137% ( 6600, 9900 ] 86 0.014% 100.151% ( 9900, 14000 ] 33 0.005% 100.157% ( 14000, 22000 ] 73 0.012% 100.169% ( 22000, 33000 ] 22 0.004% 100.172% ( 33000, 50000 ] 11 0.002% 100.174% ( 50000, 75000 ] 3 0.000% 100.174% ** Level 3 read latency histogram (micros): Count: 283809 Average: 24.5341 StdDev: 354.45 Min: 1 Median: 2.3826 Max: 45515 Percentiles: P50: 2.38 P75: 3.47 P99: 488.18 P99.9: 2977.10 P99.99: 4193.42 ------------------------------------------------------ [ 0, 1 ] 2346 0.827% 0.827% ( 1, 2 ] 104813 36.931% 37.757% ####### ( 2, 3 ] 90808 31.996% 69.754% ###### ( 3, 4 ] 31889 11.236% 80.990% ## ( 4, 6 ] 38268 13.484% 94.473% ### ( 6, 10 ] 9115 3.212% 97.685% # ( 10, 15 ] 1861 0.656% 98.341% ( 15, 22 ] 238 0.084% 98.425% ( 22, 34 ] 50 0.018% 98.442% ( 34, 51 ] 104 0.037% 98.479% ( 51, 76 ] 80 0.028% 98.507% ( 76, 110 ] 31 0.011% 98.518% ( 110, 170 ] 51 0.018% 98.536% ( 170, 250 ] 210 0.074% 98.610% ( 250, 380 ] 587 0.207% 98.817% ( 380, 580 ] 963 0.339% 99.156% ( 580, 870 ] 961 0.339% 99.495% ( 870, 1300 ] 530 0.187% 99.681% ( 1300, 1900 ] 294 0.104% 99.785% ( 1900, 2900 ] 311 0.110% 99.895% ( 2900, 4400 ] 315 0.111% 100.006% ( 4400, 6600 ] 84 0.030% 100.035% ( 6600, 9900 ] 43 0.015% 100.050% ( 9900, 14000 ] 7 0.002% 100.053% ( 14000, 22000 ] 29 0.010% 100.063% ( 22000, 33000 ] 14 0.005% 100.068% ( 33000, 50000 ] 2 0.001% 100.069% ** Level 4 read latency histogram (micros): Count: 473121 Average: 35.8806 StdDev: 520.14 Min: 1 Median: 2.4148 Max: 69143 Percentiles: P50: 2.41 P75: 3.57 P99: 616.02 P99.9: 2741.38 P99.99: 3442.49 ------------------------------------------------------ [ 0, 1 ] 3263 0.690% 0.690% ( 1, 2 ] 171183 36.182% 36.871% ####### ( 2, 3 ] 149741 31.650% 68.521% ###### ( 3, 4 ] 53946 11.402% 79.923% ## ( 4, 6 ] 65463 13.836% 93.760% ### ( 6, 10 ] 15668 3.312% 97.071% # ( 10, 15 ] 3176 0.671% 97.742% ( 15, 22 ] 342 0.072% 97.815% ( 22, 34 ] 85 0.018% 97.833% ( 34, 51 ] 197 0.042% 97.874% ( 51, 76 ] 141 0.030% 97.904% ( 76, 110 ] 42 0.009% 97.913% ( 110, 170 ] 178 0.038% 97.951% ( 170, 250 ] 607 0.128% 98.079% ( 250, 380 ] 1631 0.345% 98.424% ( 380, 580 ] 2464 0.521% 98.944% ( 580, 870 ] 2132 0.451% 99.395% ( 870, 1300 ] 1162 0.246% 99.641% ( 1300, 1900 ] 597 0.126% 99.767% ( 1900, 2900 ] 751 0.159% 99.926% ( 2900, 4400 ] 848 0.179% 100.105% ( 4400, 6600 ] 251 0.053% 100.158% ( 6600, 9900 ] 110 0.023% 100.181% ( 9900, 14000 ] 29 0.006% 100.187% ( 14000, 22000 ] 45 0.010% 100.197% ( 22000, 33000 ] 29 0.006% 100.203% ( 33000, 50000 ] 14 0.003% 100.206% ( 50000, 75000 ] 9 0.002% 100.208% ** Level 5 read latency histogram (micros): Count: 7517 Average: 1237.3709 StdDev: 2660.78 Min: 93 Median: 606.4585 Max: 72038 Percentiles: P50: 606.46 P75: 1143.85 P99: 7823.47 P99.9: 41458.71 P99.99: 70301.87 ------------------------------------------------------ ( 76, 110 ] 7 0.093% 0.093% ( 110, 170 ] 121 1.610% 1.703% ( 170, 250 ] 469 6.239% 7.942% # ( 250, 380 ] 1235 16.429% 24.371% ### ( 380, 580 ] 1787 23.773% 48.144% ##### ( 580, 870 ] 1529 20.341% 68.485% #### ( 870, 1300 ] 769 10.230% 78.715% ## ( 1300, 1900 ] 368 4.896% 83.610% # ( 1900, 2900 ] 430 5.720% 89.331% # ( 2900, 4400 ] 544 7.237% 96.568% # ( 4400, 6600 ] 168 2.235% 98.803% ( 6600, 9900 ] 40 0.532% 99.335% ( 9900, 14000 ] 7 0.093% 99.428% ( 14000, 22000 ] 11 0.146% 99.574% ( 22000, 33000 ] 21 0.279% 99.854% ( 33000, 50000 ] 7 0.093% 99.947% ( 50000, 75000 ] 4 0.053% 100.000% ** DB Stats ** Uptime(secs): 7.0 total, 6.9 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 140.98 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.212 H:M:S, 31.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 142.73 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.212 H:M:S, 32.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/07/16-01:04:45 ... thread 0: (1000000,2000000) ops and (187836.4,163743.7) ops/second in (5.323781,12.214212) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/8 1.53 GB 93.8 0.9 0.0 0.9 2.6 1.8 0.0 1.5 45.0 138.0 19.56 9.41 27 0.724 894K 4945 0.0 0.0 L1 3/0 183.08 MB 18.3 0.8 0.5 0.3 0.8 0.5 0.0 1.7 95.4 94.7 8.57 4.42 2 4.284 829K 6190 0.0 0.0 L2 1/0 87.36 MB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 22/6 1.48 GB 1.0 5.2 3.5 1.6 5.0 3.4 0.3 1.4 110.2 107.3 48.10 28.22 15 3.207 2870K 17K 0.0 0.0 L4 150/18 10.01 GB 0.9 5.2 2.2 2.9 4.6 1.6 0.9 2.1 132.6 117.1 39.81 24.11 16 2.488 2296K 0 0.0 0.0 L5 148/0 11.16 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 336/32 24.44 GB 0.0 12.0 6.2 5.8 13.0 7.2 3.6 7.3 105.8 114.9 116.04 66.16 60 1.934 6891K 29K 0.0 0.0 Int 0/0 0.00 KB 0.0 12.0 6.2 5.8 13.0 7.2 3.6 7.3 105.8 114.9 116.04 66.16 60 1.934 6891K 29K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.0 6.2 5.8 11.2 5.5 0.0 0.0 117.5 110.2 104.49 60.67 36 2.902 6891K 29K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 157.8 11.56 5.49 24 0.482 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.3 total, 12.2 interval Flush(GB): cumulative 1.781, interval 1.781 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.02 GB write, 1083.91 MB/s write, 11.99 GB read, 997.64 MB/s read, 116.0 seconds Interval compaction: 13.02 GB write, 1091.55 MB/s write, 11.99 GB read, 1004.67 MB/s read, 116.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 17, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 17, interval: 17 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 417044 Average: 33.1538 StdDev: 371.55 Min: 1 Median: 2.1664 Max: 72380 Percentiles: P50: 2.17 P75: 3.15 P99: 744.71 P99.9: 3656.95 P99.99: 5287.42 ------------------------------------------------------ [ 0, 1 ] 9301 2.230% 2.230% ( 1, 2 ] 179522 43.046% 45.277% ######### ( 2, 3 ] 118383 28.386% 73.663% ###### ( 3, 4 ] 38090 9.133% 82.796% ## ( 4, 6 ] 43582 10.450% 93.246% ## ( 6, 10 ] 11625 2.787% 96.034% # ( 10, 15 ] 2210 0.530% 96.564% ( 15, 22 ] 294 0.070% 96.634% ( 22, 34 ] 25 0.006% 96.640% ( 34, 51 ] 29 0.007% 96.647% ( 51, 76 ] 29 0.007% 96.654% ( 76, 110 ] 17 0.004% 96.658% ( 110, 170 ] 408 0.098% 96.756% ( 170, 250 ] 2046 0.491% 97.247% ( 250, 380 ] 2712 0.650% 97.897% ( 380, 580 ] 3234 0.775% 98.672% ( 580, 870 ] 2406 0.577% 99.249% ( 870, 1300 ] 1251 0.300% 99.549% ( 1300, 1900 ] 505 0.121% 99.670% ( 1900, 2900 ] 636 0.153% 99.823% ( 2900, 4400 ] 638 0.153% 99.976% ( 4400, 6600 ] 147 0.035% 100.011% ( 6600, 9900 ] 35 0.008% 100.019% ( 9900, 14000 ] 14 0.003% 100.023% ( 14000, 22000 ] 24 0.006% 100.029% ( 22000, 33000 ] 6 0.001% 100.030% ( 33000, 50000 ] 2 0.000% 100.030% ( 50000, 75000 ] 5 0.001% 100.032% ** Level 1 read latency histogram (micros): Count: 118464 Average: 21.4906 StdDev: 320.34 Min: 1 Median: 2.3522 Max: 44471 Percentiles: P50: 2.35 P75: 3.42 P99: 476.72 P99.9: 3402.95 P99.99: 10214.88 ------------------------------------------------------ [ 0, 1 ] 1299 1.097% 1.097% ( 1, 2 ] 44699 37.732% 38.829% ######## ( 2, 3 ] 37579 31.722% 70.551% ###### ( 3, 4 ] 12619 10.652% 81.203% ## ( 4, 6 ] 15284 12.902% 94.105% ### ( 6, 10 ] 4067 3.433% 97.538% # ( 10, 15 ] 682 0.576% 98.113% ( 15, 22 ] 77 0.065% 98.178% ( 22, 34 ] 12 0.010% 98.188% ( 34, 51 ] 33 0.028% 98.216% ( 51, 76 ] 21 0.018% 98.234% ( 76, 110 ] 8 0.007% 98.241% ( 110, 170 ] 61 0.051% 98.292% ( 170, 250 ] 224 0.189% 98.481% ( 250, 380 ] 374 0.316% 98.797% ( 380, 580 ] 497 0.420% 99.217% ( 580, 870 ] 375 0.317% 99.533% ( 870, 1300 ] 210 0.177% 99.710% ( 1300, 1900 ] 82 0.069% 99.780% ( 1900, 2900 ] 108 0.091% 99.871% ( 2900, 4400 ] 103 0.087% 99.958% ( 4400, 6600 ] 24 0.020% 99.978% ( 6600, 9900 ] 14 0.012% 99.990% ( 9900, 14000 ] 2 0.002% 99.992% ( 14000, 22000 ] 7 0.006% 99.997% ( 22000, 33000 ] 1 0.001% 99.998% ( 33000, 50000 ] 3 0.003% 100.001% ** Level 2 read latency histogram (micros): Count: 717125 Average: 18.3457 StdDev: 366.52 Min: 1 Median: 2.3844 Max: 69191 Percentiles: P50: 2.38 P75: 3.44 P99: 14.05 P99.9: 1462.82 P99.99: 2604.36 ------------------------------------------------------ [ 0, 1 ] 4766 0.665% 0.665% ( 1, 2 ] 264062 36.822% 37.487% ####### ( 2, 3 ] 233472 32.557% 70.044% ####### ( 3, 4 ] 81016 11.297% 81.341% ## ( 4, 6 ] 98723 13.766% 95.107% ### ( 6, 10 ] 23697 3.304% 98.412% # ( 10, 15 ] 5203 0.726% 99.137% ( 15, 22 ] 678 0.095% 99.232% ( 22, 34 ] 121 0.017% 99.249% ( 34, 51 ] 272 0.038% 99.287% ( 51, 76 ] 211 0.029% 99.316% ( 76, 110 ] 62 0.009% 99.325% ( 110, 170 ] 103 0.014% 99.339% ( 170, 250 ] 305 0.043% 99.382% ( 250, 380 ] 832 0.116% 99.498% ( 380, 580 ] 1290 0.180% 99.678% ( 580, 870 ] 987 0.138% 99.815% ( 870, 1300 ] 512 0.071% 99.887% ( 1300, 1900 ] 357 0.050% 99.936% ( 1900, 2900 ] 547 0.076% 100.013% ( 2900, 4400 ] 576 0.080% 100.093% ( 4400, 6600 ] 213 0.030% 100.123% ( 6600, 9900 ] 88 0.012% 100.135% ( 9900, 14000 ] 34 0.005% 100.140% ( 14000, 22000 ] 74 0.010% 100.150% ( 22000, 33000 ] 22 0.003% 100.153% ( 33000, 50000 ] 12 0.002% 100.155% ( 50000, 75000 ] 3 0.000% 100.155% ** Level 3 read latency histogram (micros): Count: 995365 Average: 15.8466 StdDev: 232.51 Min: 1 Median: 2.3431 Max: 45515 Percentiles: P50: 2.34 P75: 3.28 P99: 42.09 P99.9: 1183.48 P99.99: 1832.95 ------------------------------------------------------ [ 0, 1 ] 5420 0.545% 0.545% ( 1, 2 ] 377698 37.946% 38.490% ######## ( 2, 3 ] 333867 33.542% 72.032% ####### ( 3, 4 ] 107043 10.754% 82.787% ## ( 4, 6 ] 123494 12.407% 95.193% ## ( 6, 10 ] 31004 3.115% 98.308% # ( 10, 15 ] 5850 0.588% 98.896% ( 15, 22 ] 710 0.071% 98.967% ( 22, 34 ] 127 0.013% 98.980% ( 34, 51 ] 419 0.042% 99.022% ( 51, 76 ] 227 0.023% 99.045% ( 76, 110 ] 86 0.009% 99.054% ( 110, 170 ] 200 0.020% 99.074% ( 170, 250 ] 997 0.100% 99.174% ( 250, 380 ] 1849 0.186% 99.360% ( 380, 580 ] 2402 0.241% 99.601% ( 580, 870 ] 2140 0.215% 99.816% ( 870, 1300 ] 1149 0.115% 99.931% ( 1300, 1900 ] 658 0.066% 99.997% ( 1900, 2900 ] 709 0.071% 100.069% ( 2900, 4400 ] 726 0.073% 100.142% ( 4400, 6600 ] 199 0.020% 100.162% ( 6600, 9900 ] 54 0.005% 100.167% ( 9900, 14000 ] 12 0.001% 100.168% ( 14000, 22000 ] 34 0.003% 100.172% ( 22000, 33000 ] 14 0.001% 100.173% ( 33000, 50000 ] 4 0.000% 100.174% ** Level 4 read latency histogram (micros): Count: 805225 Average: 35.3065 StdDev: 436.98 Min: 1 Median: 2.4260 Max: 69143 Percentiles: P50: 2.43 P75: 3.59 P99: 651.46 P99.9: 2629.72 P99.99: 3284.36 ------------------------------------------------------ [ 0, 1 ] 5119 0.636% 0.636% ( 1, 2 ] 288157 35.786% 36.422% ####### ( 2, 3 ] 256654 31.874% 68.295% ###### ( 3, 4 ] 91495 11.363% 79.658% ## ( 4, 6 ] 109006 13.537% 93.195% ### ( 6, 10 ] 26073 3.238% 96.433% # ( 10, 15 ] 5301 0.658% 97.091% ( 15, 22 ] 609 0.076% 97.167% ( 22, 34 ] 130 0.016% 97.183% ( 34, 51 ] 260 0.032% 97.216% ( 51, 76 ] 202 0.025% 97.241% ( 76, 110 ] 84 0.010% 97.251% ( 110, 170 ] 645 0.080% 97.331% ( 170, 250 ] 2801 0.348% 97.679% ( 250, 380 ] 4306 0.535% 98.214% ( 380, 580 ] 5322 0.661% 98.875% ( 580, 870 ] 4098 0.509% 99.384% ( 870, 1300 ] 2129 0.264% 99.648% ( 1300, 1900 ] 1068 0.133% 99.781% ( 1900, 2900 ] 1318 0.164% 99.944% ( 2900, 4400 ] 1438 0.179% 100.123% ( 4400, 6600 ] 392 0.049% 100.172% ( 6600, 9900 ] 134 0.017% 100.188% ( 9900, 14000 ] 34 0.004% 100.192% ( 14000, 22000 ] 48 0.006% 100.198% ( 22000, 33000 ] 32 0.004% 100.202% ( 33000, 50000 ] 16 0.002% 100.204% ( 50000, 75000 ] 9 0.001% 100.206% ** Level 5 read latency histogram (micros): Count: 16522 Average: 952.1508 StdDev: 1963.90 Min: 93 Median: 510.3985 Max: 72038 Percentiles: P50: 510.40 P75: 859.07 P99: 6120.60 P99.9: 30631.33 P99.99: 58423.75 ------------------------------------------------------ ( 76, 110 ] 15 0.091% 0.091% ( 110, 170 ] 428 2.590% 2.681% # ( 170, 250 ] 2123 12.850% 15.531% ### ( 250, 380 ] 3192 19.320% 34.851% #### ( 380, 580 ] 3839 23.236% 58.086% ##### ( 580, 870 ] 2904 17.577% 75.663% #### ( 870, 1300 ] 1458 8.825% 84.487% ## ( 1300, 1900 ] 660 3.995% 88.482% # ( 1900, 2900 ] 713 4.315% 92.797% # ( 2900, 4400 ] 823 4.981% 97.779% # ( 4400, 6600 ] 258 1.562% 99.340% ( 6600, 9900 ] 56 0.339% 99.679% ( 9900, 14000 ] 9 0.054% 99.734% ( 14000, 22000 ] 11 0.067% 99.800% ( 22000, 33000 ] 21 0.127% 99.927% ( 33000, 50000 ] 9 0.054% 99.982% ( 50000, 75000 ] 4 0.024% 100.006% ** DB Stats ** Uptime(secs): 12.3 total, 5.3 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 159.97 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.218 H:M:S, 26.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 184.86 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.006 H:M:S, 18.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 8 num-running-flushes: 1 2024/07/16-01:04:49 ... thread 0: (1000000,3000000) ops and (241692.5,183467.2) ops/second in (4.137488,16.351700) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/8 2.57 GB 200.2 1.9 0.0 1.9 4.7 2.8 0.0 1.7 69.2 171.5 28.17 16.82 47 0.599 1979K 10K 0.0 0.0 L1 3/0 183.08 MB 18.3 0.8 0.5 0.3 0.8 0.5 0.0 1.7 95.4 94.7 8.57 4.42 2 4.284 829K 6190 0.0 0.0 L2 1/0 87.36 MB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 979.19 MB 1.0 5.2 3.5 1.6 5.0 3.4 0.3 1.4 110.2 107.3 48.10 28.22 15 3.207 2870K 17K 0.0 0.0 L4 146/0 9.64 GB 1.0 7.0 2.7 4.3 6.1 1.9 0.9 2.2 143.8 126.4 49.81 32.21 22 2.264 3059K 3311 0.0 0.0 L5 157/0 11.81 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 339/8 25.23 GB 0.0 14.9 6.7 8.1 16.7 8.6 4.2 5.9 113.1 127.0 134.65 81.66 86 1.566 8739K 37K 0.0 0.0 Int 0/0 0.00 KB 0.0 14.9 6.7 8.1 16.7 8.6 4.2 5.9 113.1 127.0 134.65 81.66 86 1.566 8739K 37K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.9 6.7 8.1 13.9 5.7 0.0 0.0 127.9 119.3 119.10 73.14 46 2.589 8739K 37K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 186.1 15.54 8.52 40 0.389 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.4 total, 16.4 interval Flush(GB): cumulative 2.825, interval 2.825 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 16.70 GB write, 1040.01 MB/s write, 14.87 GB read, 926.26 MB/s read, 134.6 seconds Interval compaction: 16.70 GB write, 1045.49 MB/s write, 14.87 GB read, 931.14 MB/s read, 134.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 20, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 20, interval: 20 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 729989 Average: 27.7460 StdDev: 302.51 Min: 0 Median: 1.8999 Max: 72380 Percentiles: P50: 1.90 P75: 2.86 P99: 540.67 P99.9: 2741.05 P99.99: 4036.43 ------------------------------------------------------ [ 0, 1 ] 47135 6.457% 6.457% # ( 1, 2 ] 353234 48.389% 54.846% ########## ( 2, 3 ] 170146 23.308% 78.154% ##### ( 3, 4 ] 52634 7.210% 85.364% # ( 4, 6 ] 55581 7.614% 92.978% ## ( 6, 10 ] 14803 2.028% 95.006% ( 10, 15 ] 2985 0.409% 95.415% ( 15, 22 ] 382 0.052% 95.467% ( 22, 34 ] 62 0.008% 95.476% ( 34, 51 ] 38 0.005% 95.481% ( 51, 76 ] 35 0.005% 95.486% ( 76, 110 ] 168 0.023% 95.509% ( 110, 170 ] 4033 0.552% 96.061% ( 170, 250 ] 11133 1.525% 97.586% ( 250, 380 ] 6319 0.866% 98.452% ( 380, 580 ] 4983 0.683% 99.135% ( 580, 870 ] 3070 0.421% 99.555% ( 870, 1300 ] 1399 0.192% 99.747% ( 1300, 1900 ] 539 0.074% 99.821% ( 1900, 2900 ] 692 0.095% 99.915% ( 2900, 4400 ] 722 0.099% 100.014% ( 4400, 6600 ] 160 0.022% 100.036% ( 6600, 9900 ] 39 0.005% 100.042% ( 9900, 14000 ] 16 0.002% 100.044% ( 14000, 22000 ] 30 0.004% 100.048% ( 22000, 33000 ] 8 0.001% 100.049% ( 33000, 50000 ] 4 0.001% 100.049% ( 50000, 75000 ] 5 0.001% 100.050% ** Level 1 read latency histogram (micros): Count: 119668 Average: 24.6644 StdDev: 339.80 Min: 1 Median: 2.3682 Max: 44471 Percentiles: P50: 2.37 P75: 3.49 P99: 511.92 P99.9: 3484.89 P99.99: 14918.40 ------------------------------------------------------ [ 0, 1 ] 1299 1.086% 1.086% ( 1, 2 ] 44699 37.353% 38.438% ####### ( 2, 3 ] 37579 31.403% 69.841% ###### ( 3, 4 ] 12619 10.545% 80.386% ## ( 4, 6 ] 15284 12.772% 93.158% ### ( 6, 10 ] 4067 3.399% 96.556% # ( 10, 15 ] 682 0.570% 97.126% ( 15, 22 ] 77 0.064% 97.191% ( 22, 34 ] 12 0.010% 97.201% ( 34, 51 ] 33 0.028% 97.228% ( 51, 76 ] 21 0.018% 97.246% ( 76, 110 ] 17 0.014% 97.260% ( 110, 170 ] 304 0.254% 97.514% ( 170, 250 ] 809 0.676% 98.190% ( 250, 380 ] 563 0.470% 98.660% ( 380, 580 ] 616 0.515% 99.175% ( 580, 870 ] 409 0.342% 99.517% ( 870, 1300 ] 221 0.185% 99.702% ( 1300, 1900 ] 83 0.069% 99.771% ( 1900, 2900 ] 113 0.094% 99.865% ( 2900, 4400 ] 106 0.089% 99.954% ( 4400, 6600 ] 25 0.021% 99.975% ( 6600, 9900 ] 15 0.013% 99.987% ( 9900, 14000 ] 2 0.002% 99.989% ( 14000, 22000 ] 9 0.008% 99.997% ( 22000, 33000 ] 2 0.002% 99.998% ( 33000, 50000 ] 3 0.003% 100.001% ** Level 2 read latency histogram (micros): Count: 728314 Average: 18.1404 StdDev: 363.76 Min: 1 Median: 2.3848 Max: 69191 Percentiles: P50: 2.38 P75: 3.44 P99: 13.99 P99.9: 1447.37 P99.99: 2605.98 ------------------------------------------------------ [ 0, 1 ] 4784 0.657% 0.657% ( 1, 2 ] 268101 36.811% 37.468% ####### ( 2, 3 ] 237167 32.564% 70.032% ####### ( 3, 4 ] 82407 11.315% 81.347% ## ( 4, 6 ] 100349 13.778% 95.125% ### ( 6, 10 ] 24029 3.299% 98.424% # ( 10, 15 ] 5258 0.722% 99.146% ( 15, 22 ] 680 0.093% 99.239% ( 22, 34 ] 122 0.017% 99.256% ( 34, 51 ] 272 0.037% 99.294% ( 51, 76 ] 211 0.029% 99.323% ( 76, 110 ] 62 0.009% 99.331% ( 110, 170 ] 106 0.015% 99.346% ( 170, 250 ] 308 0.042% 99.388% ( 250, 380 ] 842 0.116% 99.504% ( 380, 580 ] 1295 0.178% 99.681% ( 580, 870 ] 990 0.136% 99.817% ( 870, 1300 ] 515 0.071% 99.888% ( 1300, 1900 ] 357 0.049% 99.937% ( 1900, 2900 ] 547 0.075% 100.012% ( 2900, 4400 ] 578 0.079% 100.091% ( 4400, 6600 ] 213 0.029% 100.121% ( 6600, 9900 ] 88 0.012% 100.133% ( 9900, 14000 ] 34 0.005% 100.137% ( 14000, 22000 ] 74 0.010% 100.148% ( 22000, 33000 ] 22 0.003% 100.151% ( 33000, 50000 ] 12 0.002% 100.152% ( 50000, 75000 ] 3 0.000% 100.153% ** Level 3 read latency histogram (micros): Count: 1069895 Average: 16.1863 StdDev: 233.91 Min: 1 Median: 2.3396 Max: 45515 Percentiles: P50: 2.34 P75: 3.28 P99: 214.94 P99.9: 1167.87 P99.99: 1836.52 ------------------------------------------------------ [ 0, 1 ] 6227 0.582% 0.582% ( 1, 2 ] 407855 38.121% 38.703% ######## ( 2, 3 ] 355937 33.268% 71.971% ####### ( 3, 4 ] 114236 10.677% 82.649% ## ( 4, 6 ] 131474 12.288% 94.937% ## ( 6, 10 ] 33014 3.086% 98.023% # ( 10, 15 ] 6194 0.579% 98.602% ( 15, 22 ] 764 0.071% 98.673% ( 22, 34 ] 134 0.013% 98.686% ( 34, 51 ] 430 0.040% 98.726% ( 51, 76 ] 234 0.022% 98.748% ( 76, 110 ] 120 0.011% 98.759% ( 110, 170 ] 976 0.091% 98.850% ( 170, 250 ] 2850 0.266% 99.117% ( 250, 380 ] 2532 0.237% 99.353% ( 380, 580 ] 2764 0.258% 99.612% ( 580, 870 ] 2266 0.212% 99.824% ( 870, 1300 ] 1181 0.110% 99.934% ( 1300, 1900 ] 671 0.063% 99.997% ( 1900, 2900 ] 725 0.068% 100.064% ( 2900, 4400 ] 744 0.070% 100.134% ( 4400, 6600 ] 200 0.019% 100.153% ( 6600, 9900 ] 57 0.005% 100.158% ( 9900, 14000 ] 13 0.001% 100.159% ( 14000, 22000 ] 41 0.004% 100.163% ( 22000, 33000 ] 14 0.001% 100.164% ( 33000, 50000 ] 5 0.000% 100.165% ** Level 4 read latency histogram (micros): Count: 1145104 Average: 31.8057 StdDev: 380.91 Min: 1 Median: 2.3967 Max: 69143 Percentiles: P50: 2.40 P75: 3.56 P99: 528.98 P99.9: 1930.70 P99.99: 2651.39 ------------------------------------------------------ [ 0, 1 ] 7700 0.672% 0.672% ( 1, 2 ] 422706 36.914% 37.587% ####### ( 2, 3 ] 358352 31.294% 68.881% ###### ( 3, 4 ] 124402 10.864% 79.745% ## ( 4, 6 ] 144670 12.634% 92.379% ### ( 6, 10 ] 35424 3.094% 95.472% # ( 10, 15 ] 7086 0.619% 96.091% ( 15, 22 ] 849 0.074% 96.165% ( 22, 34 ] 169 0.015% 96.180% ( 34, 51 ] 328 0.029% 96.208% ( 51, 76 ] 246 0.021% 96.230% ( 76, 110 ] 267 0.023% 96.253% ( 110, 170 ] 4907 0.429% 96.682% ( 170, 250 ] 12975 1.133% 97.815% ( 250, 380 ] 8103 0.708% 98.522% ( 380, 580 ] 7342 0.641% 99.164% ( 580, 870 ] 4899 0.428% 99.591% ( 870, 1300 ] 2332 0.204% 99.795% ( 1300, 1900 ] 1158 0.101% 99.896% ( 1900, 2900 ] 1430 0.125% 100.021% ( 2900, 4400 ] 1561 0.136% 100.157% ( 4400, 6600 ] 416 0.036% 100.194% ( 6600, 9900 ] 144 0.013% 100.206% ( 9900, 14000 ] 41 0.004% 100.210% ( 14000, 22000 ] 64 0.006% 100.215% ( 22000, 33000 ] 33 0.003% 100.218% ( 33000, 50000 ] 16 0.001% 100.220% ( 50000, 75000 ] 9 0.001% 100.221% ** Level 5 read latency histogram (micros): Count: 32260 Average: 628.6987 StdDev: 1483.20 Min: 91 Median: 309.7375 Max: 72038 Percentiles: P50: 309.74 P75: 566.85 P99: 4943.15 P99.9: 22911.43 P99.99: 49615.80 ------------------------------------------------------ ( 76, 110 ] 139 0.431% 0.431% ( 110, 170 ] 3537 10.964% 11.395% ## ( 170, 250 ] 9741 30.195% 41.590% ###### ( 250, 380 ] 5904 18.301% 59.892% #### ( 380, 580 ] 5217 16.172% 76.063% ### ( 580, 870 ] 3416 10.589% 86.652% ## ( 870, 1300 ] 1568 4.861% 91.513% # ( 1300, 1900 ] 691 2.142% 93.655% ( 1900, 2900 ] 762 2.362% 96.017% ( 2900, 4400 ] 895 2.774% 98.791% # ( 4400, 6600 ] 273 0.846% 99.637% ( 6600, 9900 ] 62 0.192% 99.830% ( 9900, 14000 ] 10 0.031% 99.861% ( 14000, 22000 ] 11 0.034% 99.895% ( 22000, 33000 ] 21 0.065% 99.960% ( 33000, 50000 ] 10 0.031% 99.991% ( 50000, 75000 ] 4 0.012% 100.003% ** DB Stats ** Uptime(secs): 16.4 total, 4.1 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 179.57 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.523 H:M:S, 21.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 237.87 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.305 H:M:S, 7.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 2024/07/16-01:04:52 ... thread 0: (1000000,4000000) ops and (275255.7,200153.3) ops/second in (3.632985,19.984685) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/4 3.48 GB 331.5 3.3 0.0 3.3 7.0 3.7 0.0 1.9 79.7 171.0 41.83 25.08 67 0.624 3385K 18K 0.0 0.0 L1 3/0 183.08 MB 18.3 0.8 0.5 0.3 0.8 0.5 0.0 1.7 95.4 94.7 8.57 4.42 2 4.284 829K 6190 0.0 0.0 L2 1/0 87.36 MB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 979.19 MB 1.0 5.2 3.5 1.6 5.0 3.4 0.3 1.4 110.2 107.3 48.10 28.22 15 3.207 2870K 17K 0.0 0.0 L4 146/0 9.64 GB 1.0 7.0 2.7 4.3 6.1 1.9 0.9 2.2 143.8 126.4 49.81 32.21 22 2.264 3059K 3311 0.0 0.0 L5 157/0 11.81 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 339/4 26.15 GB 0.0 16.2 6.7 9.5 19.0 9.5 4.2 5.1 112.0 130.9 148.31 89.92 106 1.399 10M 45K 0.0 0.0 Int 0/0 0.00 KB 0.0 16.2 6.7 9.5 19.0 9.5 4.2 5.1 112.0 130.9 148.31 89.92 106 1.399 10M 45K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 16.2 6.7 9.5 15.2 5.7 0.0 0.0 127.9 119.9 129.93 78.86 51 2.548 10M 45K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.7 3.7 0.0 0.0 0.0 208.7 18.38 11.06 55 0.334 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.1 total, 20.0 interval Flush(GB): cumulative 3.747, interval 3.747 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 18.96 GB write, 967.35 MB/s write, 16.22 GB read, 827.56 MB/s read, 148.3 seconds Interval compaction: 18.96 GB write, 971.52 MB/s write, 16.22 GB read, 831.13 MB/s read, 148.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 20, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 20, interval: 20 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1009389 Average: 22.5158 StdDev: 258.20 Min: 0 Median: 1.7549 Max: 72380 Percentiles: P50: 1.75 P75: 2.64 P99: 433.57 P99.9: 2166.01 P99.99: 3729.07 ------------------------------------------------------ [ 0, 1 ] 111279 11.024% 11.024% ## ( 1, 2 ] 521115 51.627% 62.651% ########## ( 2, 3 ] 194154 19.235% 81.886% #### ( 3, 4 ] 59377 5.882% 87.768% # ( 4, 6 ] 60754 6.019% 93.787% # ( 6, 10 ] 16119 1.597% 95.384% ( 10, 15 ] 3296 0.327% 95.711% ( 15, 22 ] 442 0.044% 95.755% ( 22, 34 ] 83 0.008% 95.763% ( 34, 51 ] 42 0.004% 95.767% ( 51, 76 ] 37 0.004% 95.771% ( 76, 110 ] 383 0.038% 95.809% ( 110, 170 ] 7481 0.741% 96.550% ( 170, 250 ] 16269 1.612% 98.161% ( 250, 380 ] 7056 0.699% 98.860% ( 380, 580 ] 5257 0.521% 99.381% ( 580, 870 ] 3107 0.308% 99.689% ( 870, 1300 ] 1402 0.139% 99.828% ( 1300, 1900 ] 542 0.054% 99.882% ( 1900, 2900 ] 694 0.069% 99.950% ( 2900, 4400 ] 722 0.072% 100.022% ( 4400, 6600 ] 160 0.016% 100.038% ( 6600, 9900 ] 39 0.004% 100.042% ( 9900, 14000 ] 16 0.002% 100.043% ( 14000, 22000 ] 30 0.003% 100.046% ( 22000, 33000 ] 8 0.001% 100.047% ( 33000, 50000 ] 4 0.000% 100.047% ( 50000, 75000 ] 5 0.000% 100.048% ** Level 1 read latency histogram (micros): Count: 121222 Average: 26.8364 StdDev: 338.24 Min: 1 Median: 2.3889 Max: 44471 Percentiles: P50: 2.39 P75: 3.58 P99: 512.46 P99.9: 3462.90 P99.99: 14780.27 ------------------------------------------------------ [ 0, 1 ] 1299 1.072% 1.072% ( 1, 2 ] 44699 36.874% 37.945% ####### ( 2, 3 ] 37579 31.000% 68.945% ###### ( 3, 4 ] 12619 10.410% 79.355% ## ( 4, 6 ] 15284 12.608% 91.964% ### ( 6, 10 ] 4067 3.355% 95.319% # ( 10, 15 ] 682 0.563% 95.881% ( 15, 22 ] 77 0.064% 95.945% ( 22, 34 ] 12 0.010% 95.955% ( 34, 51 ] 33 0.027% 95.982% ( 51, 76 ] 21 0.017% 95.999% ( 76, 110 ] 49 0.040% 96.039% ( 110, 170 ] 864 0.713% 96.752% ( 170, 250 ] 1610 1.328% 98.080% ( 250, 380 ] 679 0.560% 98.641% ( 380, 580 ] 658 0.543% 99.183% ( 580, 870 ] 412 0.340% 99.523% ( 870, 1300 ] 221 0.182% 99.705% ( 1300, 1900 ] 83 0.068% 99.774% ( 1900, 2900 ] 113 0.093% 99.867% ( 2900, 4400 ] 106 0.087% 99.955% ( 4400, 6600 ] 25 0.021% 99.975% ( 6600, 9900 ] 15 0.012% 99.988% ( 9900, 14000 ] 2 0.002% 99.989% ( 14000, 22000 ] 9 0.007% 99.997% ( 22000, 33000 ] 2 0.002% 99.998% ( 33000, 50000 ] 3 0.002% 100.001% ** Level 2 read latency histogram (micros): Count: 728314 Average: 18.1404 StdDev: 363.76 Min: 1 Median: 2.3848 Max: 69191 Percentiles: P50: 2.38 P75: 3.44 P99: 13.99 P99.9: 1447.37 P99.99: 2605.98 ------------------------------------------------------ [ 0, 1 ] 4784 0.657% 0.657% ( 1, 2 ] 268101 36.811% 37.468% ####### ( 2, 3 ] 237167 32.564% 70.032% ####### ( 3, 4 ] 82407 11.315% 81.347% ## ( 4, 6 ] 100349 13.778% 95.125% ### ( 6, 10 ] 24029 3.299% 98.424% # ( 10, 15 ] 5258 0.722% 99.146% ( 15, 22 ] 680 0.093% 99.239% ( 22, 34 ] 122 0.017% 99.256% ( 34, 51 ] 272 0.037% 99.294% ( 51, 76 ] 211 0.029% 99.323% ( 76, 110 ] 62 0.009% 99.331% ( 110, 170 ] 106 0.015% 99.346% ( 170, 250 ] 308 0.042% 99.388% ( 250, 380 ] 842 0.116% 99.504% ( 380, 580 ] 1295 0.178% 99.681% ( 580, 870 ] 990 0.136% 99.817% ( 870, 1300 ] 515 0.071% 99.888% ( 1300, 1900 ] 357 0.049% 99.937% ( 1900, 2900 ] 547 0.075% 100.012% ( 2900, 4400 ] 578 0.079% 100.091% ( 4400, 6600 ] 213 0.029% 100.121% ( 6600, 9900 ] 88 0.012% 100.133% ( 9900, 14000 ] 34 0.005% 100.137% ( 14000, 22000 ] 74 0.010% 100.148% ( 22000, 33000 ] 22 0.003% 100.151% ( 33000, 50000 ] 12 0.002% 100.152% ( 50000, 75000 ] 3 0.000% 100.153% ** Level 3 read latency histogram (micros): Count: 1074393 Average: 16.9328 StdDev: 233.75 Min: 1 Median: 2.3459 Max: 45515 Percentiles: P50: 2.35 P75: 3.31 P99: 237.31 P99.9: 1166.46 P99.99: 1835.32 ------------------------------------------------------ [ 0, 1 ] 6227 0.580% 0.580% ( 1, 2 ] 407855 37.961% 38.541% ######## ( 2, 3 ] 355937 33.129% 71.670% ####### ( 3, 4 ] 114236 10.633% 82.303% ## ( 4, 6 ] 131474 12.237% 94.540% ## ( 6, 10 ] 33014 3.073% 97.613% # ( 10, 15 ] 6194 0.577% 98.189% ( 15, 22 ] 764 0.071% 98.260% ( 22, 34 ] 134 0.012% 98.273% ( 34, 51 ] 430 0.040% 98.313% ( 51, 76 ] 234 0.022% 98.335% ( 76, 110 ] 227 0.021% 98.356% ( 110, 170 ] 2591 0.241% 98.597% ( 170, 250 ] 5149 0.479% 99.076% ( 250, 380 ] 2880 0.268% 99.344% ( 380, 580 ] 2873 0.267% 99.612% ( 580, 870 ] 2284 0.213% 99.824% ( 870, 1300 ] 1183 0.110% 99.934% ( 1300, 1900 ] 672 0.063% 99.997% ( 1900, 2900 ] 725 0.067% 100.064% ( 2900, 4400 ] 744 0.069% 100.133% ( 4400, 6600 ] 200 0.019% 100.152% ( 6600, 9900 ] 57 0.005% 100.157% ( 9900, 14000 ] 13 0.001% 100.159% ( 14000, 22000 ] 41 0.004% 100.162% ( 22000, 33000 ] 14 0.001% 100.164% ( 33000, 50000 ] 5 0.000% 100.164% ** Level 4 read latency histogram (micros): Count: 1172287 Average: 35.4899 StdDev: 377.39 Min: 1 Median: 2.4344 Max: 69143 Percentiles: P50: 2.43 P75: 3.73 P99: 528.86 P99.9: 1910.98 P99.99: 2648.27 ------------------------------------------------------ [ 0, 1 ] 7700 0.657% 0.657% ( 1, 2 ] 422741 36.061% 36.718% ####### ( 2, 3 ] 358463 30.578% 67.296% ###### ( 3, 4 ] 124505 10.621% 77.917% ## ( 4, 6 ] 144918 12.362% 90.279% ## ( 6, 10 ] 35444 3.023% 93.302% # ( 10, 15 ] 7100 0.606% 93.908% ( 15, 22 ] 851 0.073% 93.981% ( 22, 34 ] 173 0.015% 93.995% ( 34, 51 ] 328 0.028% 94.023% ( 51, 76 ] 246 0.021% 94.044% ( 76, 110 ] 896 0.076% 94.121% ( 110, 170 ] 14429 1.231% 95.352% ( 170, 250 ] 26589 2.268% 97.620% ( 250, 380 ] 10209 0.871% 98.491% ( 380, 580 ] 8024 0.684% 99.175% ( 580, 870 ] 4987 0.425% 99.600% ( 870, 1300 ] 2338 0.199% 99.800% ( 1300, 1900 ] 1158 0.099% 99.899% ( 1900, 2900 ] 1431 0.122% 100.021% ( 2900, 4400 ] 1561 0.133% 100.154% ( 4400, 6600 ] 416 0.035% 100.189% ( 6600, 9900 ] 144 0.012% 100.202% ( 9900, 14000 ] 41 0.003% 100.205% ( 14000, 22000 ] 64 0.005% 100.211% ( 22000, 33000 ] 33 0.003% 100.213% ( 33000, 50000 ] 16 0.001% 100.215% ( 50000, 75000 ] 9 0.001% 100.216% ** Level 5 read latency histogram (micros): Count: 51234 Average: 468.0324 StdDev: 1196.29 Min: 89 Median: 230.5701 Max: 72038 Percentiles: P50: 230.57 P75: 404.09 P99: 4194.96 P99.9: 11444.06 P99.99: 46390.22 ------------------------------------------------------ ( 76, 110 ] 549 1.072% 1.072% ( 110, 170 ] 10379 20.258% 21.330% #### ( 170, 250 ] 19401 37.867% 59.197% ######## ( 250, 380 ] 7411 14.465% 73.662% ### ( 380, 580 ] 5692 11.110% 84.772% ## ( 580, 870 ] 3488 6.808% 91.580% # ( 870, 1300 ] 1569 3.062% 94.642% # ( 1300, 1900 ] 695 1.357% 95.999% ( 1900, 2900 ] 765 1.493% 97.492% ( 2900, 4400 ] 895 1.747% 99.239% ( 4400, 6600 ] 273 0.533% 99.772% ( 6600, 9900 ] 62 0.121% 99.893% ( 9900, 14000 ] 10 0.020% 99.912% ( 14000, 22000 ] 11 0.021% 99.934% ( 22000, 33000 ] 21 0.041% 99.975% ( 33000, 50000 ] 10 0.020% 99.994% ( 50000, 75000 ] 4 0.008% 100.002% ** DB Stats ** Uptime(secs): 20.1 total, 3.6 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 196.10 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.523 H:M:S, 17.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 270.92 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 517.059 micros/op 7719 ops/sec 20.208 seconds 155996 operations; 7.5 MB/s (38999 of 38999 found) Microseconds per read: Count: 155996 Average: 517.0738 StdDev: 1525.11 Min: 2 Median: 260.8416 Max: 96205 Percentiles: P50: 260.84 P75: 459.89 P99: 4328.96 P99.9: 22363.12 P99.99: 58889.44 ------------------------------------------------------ ( 1, 2 ] 13 0.008% 0.008% ( 2, 3 ] 50 0.032% 0.040% ( 3, 4 ] 53 0.034% 0.074% ( 4, 6 ] 163 0.104% 0.179% ( 6, 10 ] 281 0.180% 0.359% ( 10, 15 ] 658 0.422% 0.781% ( 15, 22 ] 2014 1.291% 2.072% ( 22, 34 ] 3071 1.969% 4.040% ( 34, 51 ] 1713 1.098% 5.139% ( 51, 76 ] 314 0.201% 5.340% ( 76, 110 ] 112 0.072% 5.412% ( 110, 170 ] 9943 6.374% 11.786% # ( 170, 250 ] 56879 36.462% 48.247% ####### ( 250, 380 ] 32783 21.015% 69.263% #### ( 380, 580 ] 22406 14.363% 83.626% ### ( 580, 870 ] 12086 7.748% 91.373% ## ( 870, 1300 ] 5486 3.517% 94.890% # ( 1300, 1900 ] 2196 1.408% 96.298% ( 1900, 2900 ] 1963 1.258% 97.556% ( 2900, 4400 ] 2364 1.515% 99.072% ( 4400, 6600 ] 711 0.456% 99.528% ( 6600, 9900 ] 290 0.186% 99.713% ( 9900, 14000 ] 115 0.074% 99.787% ( 14000, 22000 ] 173 0.111% 99.898% ( 22000, 33000 ] 91 0.058% 99.956% ( 33000, 50000 ] 46 0.029% 99.986% ( 50000, 75000 ] 18 0.012% 99.997% ( 75000, 110000 ] 4 0.003% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 2186187 rocksdb.bloom.filter.full.positive COUNT : 176475 rocksdb.bloom.filter.full.true.positive COUNT : 155143 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 853 rocksdb.memtable.miss COUNT : 155143 rocksdb.l0.hit COUNT : 16020 rocksdb.l1.hit COUNT : 1346 rocksdb.l2andup.hit COUNT : 137777 rocksdb.compaction.key.drop.new COUNT : 1046082 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 4060915 rocksdb.number.keys.read COUNT : 155996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 4190864280 rocksdb.bytes.read COUNT : 155996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 560 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 3522646 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 17518542794 rocksdb.compact.write.bytes COUNT : 16437381529 rocksdb.flush.write.bytes COUNT : 4088794626 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 33 rocksdb.number.superversion_acquires COUNT : 772 rocksdb.number.superversion_releases COUNT : 750 rocksdb.number.superversion_cleanups COUNT : 191 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 216 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 4062540800 rocksdb.memtable.garbage.bytes.at.flush COUNT : 7745536 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 17447664333 rocksdb.non.last.level.read.count COUNT : 4188048 rocksdb.block.checksum.compute.count COUNT : 4422337 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 258.932033 P95 : 1344.924761 P99 : 4328.264183 P100 : 96203.000000 COUNT : 155996 SUM : 80526768 rocksdb.db.write.micros P50 : 1.612685 P95 : 5.041241 P99 : 6.717102 P100 : 783830.000000 COUNT : 4060915 SUM : 14889052 rocksdb.compaction.times.micros P50 : 2295833.333333 P95 : 6995000.000000 P99 : 8540704.000000 P100 : 8540704.000000 COUNT : 51 SUM : 129928076 rocksdb.compaction.times.cpu_micros P50 : 1428846.153846 P95 : 2808333.333333 P99 : 5109151.000000 P100 : 5109151.000000 COUNT : 51 SUM : 78859980 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 1042.000000 P95 : 5720.000000 P99 : 8976.000000 P100 : 9140.000000 COUNT : 56 SUM : 95599 rocksdb.compaction.outfile.sync.micros P50 : 2419.230769 P95 : 8589.705882 P99 : 12052.500000 P100 : 21032.000000 COUNT : 195 SUM : 595529 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 226.666667 P95 : 1820.000000 P99 : 4295.000000 P100 : 6128.000000 COUNT : 128 SUM : 59917 rocksdb.table.open.io.micros P50 : 6126.582278 P95 : 20344.827586 P99 : 32400.000000 P100 : 130488.000000 COUNT : 560 SUM : 4221839 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.572025 P95 : 9.613612 P99 : 43.017706 P100 : 72388.000000 COUNT : 4243215 SUM : 66754492 rocksdb.read.block.get.micros P50 : 220.816967 P95 : 1189.569671 P99 : 3990.748632 P100 : 72045.000000 COUNT : 178791 SUM : 73215746 rocksdb.write.raw.block.micros P50 : 0.530940 P95 : 1.219713 P99 : 3.677651 P100 : 95005.000000 COUNT : 4982237 SUM : 19681862 rocksdb.numfiles.in.singlecompaction P50 : 3.370968 P95 : 7.080000 P99 : 9.000000 P100 : 9.000000 COUNT : 73 SUM : 285 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 110000.000000 P95 : 783637.000000 P99 : 783637.000000 P100 : 783637.000000 COUNT : 12 SUM : 3522646 rocksdb.sst.read.micros P50 : 2.239883 P95 : 14.153331 P99 : 539.905704 P100 : 72380.000000 COUNT : 4188445 SUM : 124062718 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 155996 SUM : 155996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 4060915 SUM : 4190864280 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 242000.000000 P95 : 882666.666667 P99 : 1059133.000000 P100 : 1059133.000000 COUNT : 56 SUM : 18558118 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 220 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0