[2024-07-11 20:25:10.640687] Starting SPDK v24.09-pre git sha1 719d03c6a / DPDK 22.11.4 initialization... [2024-07-11 20:25:10.640788] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid543264 ] Set seed to 1720722310638322 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-11 20:25:10.741665] app.c: 908:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-11 20:25:10.789606] app.c: 603:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-11 20:25:10.789664] app.c: 604:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 543264' to capture a snapshot of events at runtime. [2024-07-11 20:25:10.789680] app.c: 609:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-11 20:25:10.789700] app.c: 610:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-11 20:25:10.789712] app.c: 611:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid543264 for offline analysis/debug. [2024-07-11 20:25:10.789824] reactor.c: 941:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Thu Jul 11 20:25:15 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/07/11-20:25:28 ... thread 0: (1000000,1000000) ops and (75788.9,75788.9) ops/second in (13.194548,13.194548) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/12 2.13 GB 69.2 0.7 0.0 0.7 1.6 0.9 0.0 1.7 56.4 133.1 12.29 6.82 14 0.878 703K 876 0.0 0.0 L1 13/13 1.15 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 13/0 994.82 MB 1.0 3.6 2.6 1.0 3.6 2.6 0.0 1.4 91.4 90.7 40.40 23.98 11 3.673 333K 0 0.0 0.0 L4 162/0 9.68 GB 1.0 4.8 3.5 1.3 4.7 3.4 0.1 1.4 90.3 89.1 53.91 32.71 39 1.382 3245K 546 0.0 0.0 L5 1072/0 91.54 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1274/25 105.46 GB 0.0 9.0 6.1 3.0 9.9 6.9 3.4 10.7 86.8 94.8 106.61 63.51 64 1.666 4282K 1422 0.0 0.0 Int 0/0 0.00 KB 0.0 9.0 6.1 3.0 9.9 6.9 3.4 10.7 86.8 94.8 106.61 63.51 64 1.666 4282K 1422 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.0 6.1 3.0 8.9 6.0 0.0 0.0 94.0 93.0 98.47 60.31 52 1.894 4282K 1422 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 116.1 8.14 3.20 12 0.678 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 13.4 total, 13.2 interval Flush(GB): cumulative 0.923, interval 0.923 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 9.87 GB write, 756.99 MB/s write, 9.04 GB read, 693.21 MB/s read, 106.6 seconds Interval compaction: 9.87 GB write, 765.75 MB/s write, 9.04 GB read, 701.24 MB/s read, 106.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 10, pending-compaction-bytes-stops: 0, total-delays: 10, total-stops: 26, interval: 35 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 443549 Average: 23.1958 StdDev: 332.65 Min: 1 Median: 2.5127 Max: 75505 Percentiles: P50: 2.51 P75: 3.77 P99: 326.89 P99.9: 3320.95 P99.99: 6048.92 ------------------------------------------------------ [ 0, 1 ] 6710 1.513% 1.513% ( 1, 2 ] 131528 29.654% 31.166% ###### ( 2, 3 ] 162923 36.732% 67.898% ####### ( 3, 4 ] 41138 9.275% 77.173% ## ( 4, 6 ] 55479 12.508% 89.681% ### ( 6, 10 ] 29074 6.555% 96.236% # ( 10, 15 ] 3545 0.799% 97.035% ( 15, 22 ] 425 0.096% 97.131% ( 22, 34 ] 51 0.011% 97.142% ( 34, 51 ] 25 0.006% 97.148% ( 51, 76 ] 46 0.010% 97.158% ( 76, 110 ] 192 0.043% 97.201% ( 110, 170 ] 3294 0.743% 97.944% ( 170, 250 ] 3833 0.864% 98.808% ( 250, 380 ] 1443 0.325% 99.134% ( 380, 580 ] 926 0.209% 99.342% ( 580, 870 ] 684 0.154% 99.497% ( 870, 1300 ] 695 0.157% 99.653% ( 1300, 1900 ] 580 0.131% 99.784% ( 1900, 2900 ] 424 0.096% 99.880% ( 2900, 4400 ] 333 0.075% 99.955% ( 4400, 6600 ] 213 0.048% 100.003% ( 6600, 9900 ] 38 0.009% 100.011% ( 9900, 14000 ] 11 0.002% 100.014% ( 14000, 22000 ] 9 0.002% 100.016% ( 22000, 33000 ] 5 0.001% 100.017% ( 33000, 50000 ] 10 0.002% 100.019% ( 50000, 75000 ] 1 0.000% 100.019% ( 75000, 110000 ] 1 0.000% 100.020% ** Level 1 read latency histogram (micros): Count: 577 Average: 801.7695 StdDev: 1737.04 Min: 96 Median: 246.5625 Max: 32693 Percentiles: P50: 246.56 P75: 846.44 P99: 5962.00 P99.9: 26653.00 P99.99: 32365.30 ------------------------------------------------------ ( 76, 110 ] 11 1.906% 1.906% ( 110, 170 ] 155 26.863% 28.769% ##### ( 170, 250 ] 128 22.184% 50.953% #### ( 250, 380 ] 57 9.879% 60.832% ## ( 380, 580 ] 45 7.799% 68.631% ## ( 580, 870 ] 40 6.932% 75.563% # ( 870, 1300 ] 52 9.012% 84.575% ## ( 1300, 1900 ] 31 5.373% 89.948% # ( 1900, 2900 ] 19 3.293% 93.241% # ( 2900, 4400 ] 24 4.159% 97.400% # ( 4400, 6600 ] 13 2.253% 99.653% ( 6600, 9900 ] 1 0.173% 99.827% ( 22000, 33000 ] 1 0.173% 100.000% ** Level 2 read latency histogram (micros): Count: 413019 Average: 20.5889 StdDev: 311.76 Min: 1 Median: 2.9111 Max: 38690 Percentiles: P50: 2.91 P75: 4.90 P99: 14.65 P99.9: 2535.19 P99.99: 3541.99 ------------------------------------------------------ [ 0, 1 ] 749 0.181% 0.181% ( 1, 2 ] 55800 13.510% 13.692% ### ( 2, 3 ] 164597 39.852% 53.544% ######## ( 3, 4 ] 51836 12.551% 66.094% ### ( 4, 6 ] 81327 19.691% 85.785% #### ( 6, 10 ] 47788 11.570% 97.356% ## ( 10, 15 ] 7304 1.768% 99.124% ( 15, 22 ] 1262 0.306% 99.430% ( 22, 34 ] 148 0.036% 99.465% ( 34, 51 ] 61 0.015% 99.480% ( 51, 76 ] 60 0.015% 99.495% ( 76, 110 ] 83 0.020% 99.515% ( 110, 170 ] 36 0.009% 99.524% ( 170, 250 ] 35 0.008% 99.532% ( 250, 380 ] 66 0.016% 99.548% ( 380, 580 ] 111 0.027% 99.575% ( 580, 870 ] 252 0.061% 99.636% ( 870, 1300 ] 404 0.098% 99.734% ( 1300, 1900 ] 431 0.104% 99.838% ( 1900, 2900 ] 403 0.098% 99.936% ( 2900, 4400 ] 525 0.127% 100.063% ( 4400, 6600 ] 245 0.059% 100.122% ( 6600, 9900 ] 58 0.014% 100.136% ( 9900, 14000 ] 14 0.003% 100.139% ( 14000, 22000 ] 15 0.004% 100.143% ( 22000, 33000 ] 9 0.002% 100.145% ( 33000, 50000 ] 5 0.001% 100.146% ** Level 3 read latency histogram (micros): Count: 1029811 Average: 13.2466 StdDev: 257.67 Min: 1 Median: 2.8514 Max: 59190 Percentiles: P50: 2.85 P75: 4.72 P99: 12.47 P99.9: 20.83 P99.99: 90.02 ------------------------------------------------------ [ 0, 1 ] 3867 0.376% 0.376% ( 1, 2 ] 170973 16.602% 16.978% ### ( 2, 3 ] 399403 38.784% 55.762% ######## ( 3, 4 ] 123502 11.993% 67.755% ## ( 4, 6 ] 207118 20.112% 87.867% #### ( 6, 10 ] 107665 10.455% 98.322% ## ( 10, 15 ] 14128 1.372% 99.694% ( 15, 22 ] 2551 0.248% 99.941% ( 22, 34 ] 187 0.018% 99.960% ( 34, 51 ] 81 0.008% 99.967% ( 51, 76 ] 179 0.017% 99.985% ( 76, 110 ] 131 0.013% 99.997% ( 110, 170 ] 239 0.023% 100.021% ( 170, 250 ] 218 0.021% 100.042% ( 250, 380 ] 167 0.016% 100.058% ( 380, 580 ] 238 0.023% 100.081% ( 580, 870 ] 365 0.035% 100.117% ( 870, 1300 ] 568 0.055% 100.172% ( 1300, 1900 ] 508 0.049% 100.221% ( 1900, 2900 ] 433 0.042% 100.263% ( 2900, 4400 ] 655 0.064% 100.327% ( 4400, 6600 ] 330 0.032% 100.359% ( 6600, 9900 ] 81 0.008% 100.367% ( 9900, 14000 ] 35 0.003% 100.370% ( 14000, 22000 ] 37 0.004% 100.374% ( 22000, 33000 ] 21 0.002% 100.376% ( 33000, 50000 ] 5 0.000% 100.376% ( 50000, 75000 ] 1 0.000% 100.376% ** Level 4 read latency histogram (micros): Count: 339212 Average: 41.8287 StdDev: 455.65 Min: 1 Median: 2.8739 Max: 54733 Percentiles: P50: 2.87 P75: 4.88 P99: 1020.21 P99.9: 3899.50 P99.99: 4932.91 ------------------------------------------------------ [ 0, 1 ] 1788 0.527% 0.527% ( 1, 2 ] 59504 17.542% 18.069% #### ( 2, 3 ] 123946 36.539% 54.608% ####### ( 3, 4 ] 40423 11.917% 66.525% ## ( 4, 6 ] 65305 19.252% 85.777% #### ( 6, 10 ] 33068 9.748% 95.526% ## ( 10, 15 ] 5162 1.522% 97.047% ( 15, 22 ] 928 0.274% 97.321% ( 22, 34 ] 93 0.027% 97.348% ( 34, 51 ] 24 0.007% 97.355% ( 51, 76 ] 61 0.018% 97.373% ( 76, 110 ] 103 0.030% 97.404% ( 110, 170 ] 1451 0.428% 97.831% ( 170, 250 ] 1463 0.431% 98.263% ( 250, 380 ] 635 0.187% 98.450% ( 380, 580 ] 616 0.182% 98.632% ( 580, 870 ] 845 0.249% 98.881% ( 870, 1300 ] 1159 0.342% 99.222% ( 1300, 1900 ] 1096 0.323% 99.545% ( 1900, 2900 ] 775 0.228% 99.774% ( 2900, 4400 ] 642 0.189% 99.963% ( 4400, 6600 ] 376 0.111% 100.074% ( 6600, 9900 ] 102 0.030% 100.104% ( 9900, 14000 ] 23 0.007% 100.111% ( 14000, 22000 ] 21 0.006% 100.117% ( 22000, 33000 ] 10 0.003% 100.120% ( 33000, 50000 ] 10 0.003% 100.123% ( 50000, 75000 ] 2 0.001% 100.124% ** Level 5 read latency histogram (micros): Count: 40623 Average: 920.5877 StdDev: 2125.08 Min: 83 Median: 242.1001 Max: 105188 Percentiles: P50: 242.10 P75: 890.46 P99: 6995.30 P99.9: 29354.22 P99.99: 71206.07 ------------------------------------------------------ ( 76, 110 ] 535 1.317% 1.317% ( 110, 170 ] 10344 25.463% 26.780% ##### ( 170, 250 ] 10466 25.764% 52.544% ##### ( 250, 380 ] 3902 9.605% 62.150% ## ( 380, 580 ] 2767 6.811% 68.961% # ( 580, 870 ] 2329 5.733% 74.694% # ( 870, 1300 ] 2611 6.427% 81.122% # ( 1300, 1900 ] 2289 5.635% 86.756% # ( 1900, 2900 ] 1798 4.426% 91.182% # ( 2900, 4400 ] 1837 4.522% 95.704% # ( 4400, 6600 ] 1299 3.198% 98.902% # ( 6600, 9900 ] 332 0.817% 99.719% ( 9900, 14000 ] 41 0.101% 99.820% ( 14000, 22000 ] 17 0.042% 99.862% ( 22000, 33000 ] 23 0.057% 99.919% ( 33000, 50000 ] 23 0.057% 99.975% ( 50000, 75000 ] 7 0.017% 99.993% ( 75000, 110000 ] 3 0.007% 100.000% ** DB Stats ** Uptime(secs): 13.4 total, 13.2 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 73.72 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.637 H:M:S, 49.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 74.57 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:6.637 H:M:S, 50.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 2024/07/11-20:25:37 ... thread 0: (1000000,2000000) ops and (106839.8,88674.7) ops/second in (9.359805,22.554353) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/8 3.05 GB 188.8 1.7 0.0 1.7 3.5 1.8 0.0 1.9 82.5 174.0 20.62 14.44 33 0.625 1726K 1828 0.0 0.0 L1 13/13 1.15 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 13/0 994.82 MB 1.0 3.6 2.6 1.0 3.6 2.6 0.0 1.4 91.4 90.7 40.40 23.98 11 3.673 333K 0 0.0 0.0 L4 162/0 9.68 GB 1.0 4.8 3.5 1.3 4.7 3.4 0.1 1.4 90.3 89.1 53.91 32.71 39 1.382 3245K 546 0.0 0.0 L5 1072/0 91.54 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1277/21 106.39 GB 0.0 10.0 6.1 4.0 11.8 7.8 3.4 6.4 89.3 104.9 114.93 71.13 83 1.385 5305K 2374 0.0 0.0 Int 0/0 0.00 KB 0.0 10.0 6.1 4.0 11.8 7.8 3.4 6.4 89.3 104.9 114.93 71.13 83 1.385 5305K 2374 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.0 6.1 4.0 9.9 6.0 0.0 0.0 99.4 98.5 103.21 64.92 56 1.843 5305K 2374 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 161.2 11.72 6.21 27 0.434 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 22.7 total, 22.6 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.78 GB write, 530.95 MB/s write, 10.02 GB read, 451.89 MB/s read, 114.9 seconds Interval compaction: 11.78 GB write, 534.55 MB/s write, 10.02 GB read, 454.96 MB/s read, 114.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 10, pending-compaction-bytes-stops: 0, total-delays: 10, total-stops: 26, interval: 35 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 746482 Average: 26.7771 StdDev: 262.18 Min: 1 Median: 2.2643 Max: 75505 Percentiles: P50: 2.26 P75: 3.36 P99: 354.05 P99.9: 2180.72 P99.99: 4506.92 ------------------------------------------------------ [ 0, 1 ] 33598 4.501% 4.501% # ( 1, 2 ] 278981 37.373% 41.874% ####### ( 2, 3 ] 229506 30.745% 72.619% ###### ( 3, 4 ] 49545 6.637% 79.256% # ( 4, 6 ] 63229 8.470% 87.726% ## ( 6, 10 ] 30389 4.071% 91.797% # ( 10, 15 ] 3719 0.498% 92.295% ( 15, 22 ] 447 0.060% 92.355% ( 22, 34 ] 55 0.007% 92.362% ( 34, 51 ] 27 0.004% 92.366% ( 51, 76 ] 48 0.006% 92.372% ( 76, 110 ] 1243 0.167% 92.539% ( 110, 170 ] 22074 2.957% 95.496% # ( 170, 250 ] 21706 2.908% 98.404% # ( 250, 380 ] 5560 0.745% 99.149% ( 380, 580 ] 2844 0.381% 99.530% ( 580, 870 ] 1284 0.172% 99.702% ( 870, 1300 ] 764 0.102% 99.804% ( 1300, 1900 ] 593 0.079% 99.883% ( 1900, 2900 ] 440 0.059% 99.942% ( 2900, 4400 ] 345 0.046% 99.989% ( 4400, 6600 ] 213 0.029% 100.017% ( 6600, 9900 ] 38 0.005% 100.022% ( 9900, 14000 ] 11 0.001% 100.024% ( 14000, 22000 ] 9 0.001% 100.025% ( 22000, 33000 ] 5 0.001% 100.026% ( 33000, 50000 ] 10 0.001% 100.027% ( 50000, 75000 ] 1 0.000% 100.027% ( 75000, 110000 ] 1 0.000% 100.027% ** Level 1 read latency histogram (micros): Count: 123720 Average: 10.1684 StdDev: 133.73 Min: 1 Median: 2.2557 Max: 32693 Percentiles: P50: 2.26 P75: 3.31 P99: 190.49 P99.9: 1041.08 P99.99: 4675.51 ------------------------------------------------------ [ 0, 1 ] 6922 5.595% 5.595% # ( 1, 2 ] 45622 36.875% 42.470% ####### ( 2, 3 ] 36438 29.452% 71.922% ###### ( 3, 4 ] 12428 10.045% 81.967% ## ( 4, 6 ] 15385 12.435% 94.403% ## ( 6, 10 ] 3377 2.730% 97.132% # ( 10, 15 ] 587 0.474% 97.607% ( 15, 22 ] 257 0.208% 97.814% ( 22, 34 ] 17 0.014% 97.828% ( 76, 110 ] 104 0.084% 97.912% ( 110, 170 ] 1117 0.903% 98.815% ( 170, 250 ] 897 0.725% 99.540% ( 250, 380 ] 222 0.179% 99.720% ( 380, 580 ] 138 0.112% 99.831% ( 580, 870 ] 64 0.052% 99.883% ( 870, 1300 ] 56 0.045% 99.928% ( 1300, 1900 ] 32 0.026% 99.954% ( 1900, 2900 ] 20 0.016% 99.970% ( 2900, 4400 ] 24 0.019% 99.989% ( 4400, 6600 ] 13 0.011% 100.000% ( 6600, 9900 ] 1 0.001% 100.001% ( 22000, 33000 ] 1 0.001% 100.002% ** Level 2 read latency histogram (micros): Count: 413019 Average: 20.5889 StdDev: 311.76 Min: 1 Median: 2.9111 Max: 38690 Percentiles: P50: 2.91 P75: 4.90 P99: 14.65 P99.9: 2535.19 P99.99: 3541.99 ------------------------------------------------------ [ 0, 1 ] 749 0.181% 0.181% ( 1, 2 ] 55800 13.510% 13.692% ### ( 2, 3 ] 164597 39.852% 53.544% ######## ( 3, 4 ] 51836 12.551% 66.094% ### ( 4, 6 ] 81327 19.691% 85.785% #### ( 6, 10 ] 47788 11.570% 97.356% ## ( 10, 15 ] 7304 1.768% 99.124% ( 15, 22 ] 1262 0.306% 99.430% ( 22, 34 ] 148 0.036% 99.465% ( 34, 51 ] 61 0.015% 99.480% ( 51, 76 ] 60 0.015% 99.495% ( 76, 110 ] 83 0.020% 99.515% ( 110, 170 ] 36 0.009% 99.524% ( 170, 250 ] 35 0.008% 99.532% ( 250, 380 ] 66 0.016% 99.548% ( 380, 580 ] 111 0.027% 99.575% ( 580, 870 ] 252 0.061% 99.636% ( 870, 1300 ] 404 0.098% 99.734% ( 1300, 1900 ] 431 0.104% 99.838% ( 1900, 2900 ] 403 0.098% 99.936% ( 2900, 4400 ] 525 0.127% 100.063% ( 4400, 6600 ] 245 0.059% 100.122% ( 6600, 9900 ] 58 0.014% 100.136% ( 9900, 14000 ] 14 0.003% 100.139% ( 14000, 22000 ] 15 0.004% 100.143% ( 22000, 33000 ] 9 0.002% 100.145% ( 33000, 50000 ] 5 0.001% 100.146% ** Level 3 read latency histogram (micros): Count: 1031630 Average: 13.5770 StdDev: 257.61 Min: 1 Median: 2.8537 Max: 59190 Percentiles: P50: 2.85 P75: 4.73 P99: 13.11 P99.9: 153.19 P99.99: 223.61 ------------------------------------------------------ [ 0, 1 ] 3867 0.375% 0.375% ( 1, 2 ] 170973 16.573% 16.948% ### ( 2, 3 ] 399403 38.716% 55.664% ######## ( 3, 4 ] 123502 11.972% 67.635% ## ( 4, 6 ] 207118 20.077% 87.712% #### ( 6, 10 ] 107665 10.436% 98.148% ## ( 10, 15 ] 14128 1.369% 99.518% ( 15, 22 ] 2551 0.247% 99.765% ( 22, 34 ] 187 0.018% 99.783% ( 34, 51 ] 81 0.008% 99.791% ( 51, 76 ] 179 0.017% 99.808% ( 76, 110 ] 162 0.016% 99.824% ( 110, 170 ] 1087 0.105% 99.930% ( 170, 250 ] 931 0.090% 100.020% ( 250, 380 ] 293 0.028% 100.048% ( 380, 580 ] 314 0.030% 100.079% ( 580, 870 ] 385 0.037% 100.116% ( 870, 1300 ] 571 0.055% 100.171% ( 1300, 1900 ] 509 0.049% 100.221% ( 1900, 2900 ] 434 0.042% 100.263% ( 2900, 4400 ] 655 0.063% 100.326% ( 4400, 6600 ] 330 0.032% 100.358% ( 6600, 9900 ] 81 0.008% 100.366% ( 9900, 14000 ] 35 0.003% 100.369% ( 14000, 22000 ] 37 0.004% 100.373% ( 22000, 33000 ] 21 0.002% 100.375% ( 33000, 50000 ] 5 0.000% 100.376% ( 50000, 75000 ] 1 0.000% 100.376% ** Level 4 read latency histogram (micros): Count: 358903 Average: 49.0874 StdDev: 444.90 Min: 1 Median: 2.9456 Max: 54733 Percentiles: P50: 2.95 P75: 5.27 P99: 960.46 P99.9: 3848.82 P99.99: 4909.68 ------------------------------------------------------ [ 0, 1 ] 1789 0.498% 0.498% ( 1, 2 ] 59736 16.644% 17.143% ### ( 2, 3 ] 124716 34.749% 51.892% ####### ( 3, 4 ] 41051 11.438% 63.330% ## ( 4, 6 ] 66012 18.393% 81.722% #### ( 6, 10 ] 33155 9.238% 90.960% ## ( 10, 15 ] 5188 1.446% 92.406% ( 15, 22 ] 929 0.259% 92.665% ( 22, 34 ] 94 0.026% 92.691% ( 34, 51 ] 24 0.007% 92.697% ( 51, 76 ] 61 0.017% 92.714% ( 76, 110 ] 534 0.149% 92.863% ( 110, 170 ] 9212 2.567% 95.430% # ( 170, 250 ] 8260 2.301% 97.731% ( 250, 380 ] 2010 0.560% 98.291% ( 380, 580 ] 1238 0.345% 98.636% ( 580, 870 ] 1055 0.294% 98.930% ( 870, 1300 ] 1193 0.332% 99.263% ( 1300, 1900 ] 1100 0.306% 99.569% ( 1900, 2900 ] 782 0.218% 99.787% ( 2900, 4400 ] 642 0.179% 99.966% ( 4400, 6600 ] 376 0.105% 100.071% ( 6600, 9900 ] 102 0.028% 100.099% ( 9900, 14000 ] 23 0.006% 100.106% ( 14000, 22000 ] 21 0.006% 100.111% ( 22000, 33000 ] 10 0.003% 100.114% ( 33000, 50000 ] 10 0.003% 100.117% ( 50000, 75000 ] 2 0.001% 100.118% ** Level 5 read latency histogram (micros): Count: 163878 Average: 376.6185 StdDev: 1106.68 Min: 81 Median: 186.7218 Max: 105188 Percentiles: P50: 186.72 P75: 242.46 P99: 4579.90 P99.9: 9404.22 P99.99: 45278.58 ------------------------------------------------------ ( 76, 110 ] 3545 2.163% 2.163% ( 110, 170 ] 66102 40.336% 42.499% ######## ( 170, 250 ] 58807 35.885% 78.384% ####### ( 250, 380 ] 13811 8.428% 86.812% ## ( 380, 580 ] 7374 4.500% 91.311% # ( 580, 870 ] 3751 2.289% 93.600% ( 870, 1300 ] 2773 1.692% 95.292% ( 1300, 1900 ] 2306 1.407% 96.699% ( 1900, 2900 ] 1826 1.114% 97.814% ( 2900, 4400 ] 1838 1.122% 98.935% ( 4400, 6600 ] 1299 0.793% 99.728% ( 6600, 9900 ] 332 0.203% 99.930% ( 9900, 14000 ] 41 0.025% 99.955% ( 14000, 22000 ] 17 0.010% 99.966% ( 22000, 33000 ] 23 0.014% 99.980% ( 33000, 50000 ] 23 0.014% 99.994% ( 50000, 75000 ] 7 0.004% 99.998% ( 75000, 110000 ] 3 0.002% 100.000% ** DB Stats ** Uptime(secs): 22.7 total, 9.4 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 86.67 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.637 H:M:S, 29.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 105.15 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/07/11-20:25:49 ... thread 0: (1000000,3000000) ops and (86079.5,87792.4) ops/second in (11.617169,34.171522) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/12 3.96 GB 256.9 3.9 0.0 3.9 6.7 2.8 0.0 2.4 114.4 194.4 35.24 28.14 52 0.678 4092K 13K 0.0 0.0 L1 13/13 1.15 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 13/0 994.82 MB 1.0 3.6 2.6 1.0 3.6 2.6 0.0 1.4 91.4 90.7 40.40 23.98 11 3.673 333K 0 0.0 0.0 L4 162/0 9.68 GB 1.0 4.8 3.5 1.3 4.7 3.4 0.1 1.4 90.3 89.1 53.91 32.71 39 1.382 3245K 546 0.0 0.0 L5 1072/0 91.54 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1277/25 107.30 GB 0.0 12.3 6.1 6.2 15.0 8.7 3.4 5.4 97.2 118.3 129.55 84.83 102 1.270 7671K 14K 0.0 0.0 Int 0/0 0.00 KB 0.0 12.3 6.1 6.2 15.0 8.7 3.4 5.4 97.2 118.3 129.55 84.83 102 1.270 7671K 14K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.3 6.1 6.2 12.2 6.0 0.0 0.0 110.3 109.3 114.21 75.59 60 1.903 7671K 14K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 184.8 15.34 9.24 42 0.365 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 34.3 total, 34.2 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.96 GB write, 446.32 MB/s write, 12.30 GB read, 366.83 MB/s read, 129.6 seconds Interval compaction: 14.96 GB write, 448.32 MB/s write, 12.30 GB read, 368.47 MB/s read, 129.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 7, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 7, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 10, pending-compaction-bytes-stops: 0, total-delays: 17, total-stops: 26, interval: 42 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1442161 Average: 23.2170 StdDev: 194.61 Min: 0 Median: 1.9865 Max: 75505 Percentiles: P50: 1.99 P75: 2.87 P99: 333.34 P99.9: 1142.62 P99.99: 2661.54 ------------------------------------------------------ [ 0, 1 ] 103542 7.180% 7.180% # ( 1, 2 ] 625991 43.406% 50.586% ######### ( 2, 3 ] 406917 28.216% 78.802% ###### ( 3, 4 ] 71744 4.975% 83.777% # ( 4, 6 ] 82338 5.709% 89.486% # ( 6, 10 ] 34894 2.420% 91.906% ( 10, 15 ] 4099 0.284% 92.190% ( 15, 22 ] 482 0.033% 92.223% ( 22, 34 ] 59 0.004% 92.227% ( 34, 51 ] 33 0.002% 92.230% ( 51, 76 ] 48 0.003% 92.233% ( 76, 110 ] 2281 0.158% 92.391% ( 110, 170 ] 43100 2.989% 95.380% # ( 170, 250 ] 44599 3.093% 98.472% # ( 250, 380 ] 11876 0.823% 99.296% ( 380, 580 ] 5868 0.407% 99.703% ( 580, 870 ] 2270 0.157% 99.860% ( 870, 1300 ] 913 0.063% 99.923% ( 1300, 1900 ] 615 0.043% 99.966% ( 1900, 2900 ] 458 0.032% 99.998% ( 2900, 4400 ] 350 0.024% 100.022% ( 4400, 6600 ] 216 0.015% 100.037% ( 6600, 9900 ] 38 0.003% 100.040% ( 9900, 14000 ] 11 0.001% 100.040% ( 14000, 22000 ] 9 0.001% 100.041% ( 22000, 33000 ] 5 0.000% 100.041% ( 33000, 50000 ] 10 0.001% 100.042% ( 50000, 75000 ] 1 0.000% 100.042% ( 75000, 110000 ] 1 0.000% 100.042% ** Level 1 read latency histogram (micros): Count: 256892 Average: 9.0446 StdDev: 95.57 Min: 1 Median: 2.8138 Max: 32693 Percentiles: P50: 2.81 P75: 4.55 P99: 180.65 P99.9: 578.52 P99.99: 3544.42 ------------------------------------------------------ [ 0, 1 ] 10700 4.165% 4.165% # ( 1, 2 ] 71996 28.026% 32.191% ###### ( 2, 3 ] 56221 21.885% 54.076% #### ( 3, 4 ] 36627 14.258% 68.334% ### ( 4, 6 ] 62290 24.248% 92.581% ##### ( 6, 10 ] 10667 4.152% 96.734% # ( 10, 15 ] 2012 0.783% 97.517% ( 15, 22 ] 1110 0.432% 97.949% ( 22, 34 ] 58 0.023% 97.972% ( 34, 51 ] 3 0.001% 97.973% ( 76, 110 ] 176 0.069% 98.041% ( 110, 170 ] 2222 0.865% 98.906% ( 170, 250 ] 1811 0.705% 99.611% ( 250, 380 ] 488 0.190% 99.801% ( 380, 580 ] 256 0.100% 99.901% ( 580, 870 ] 102 0.040% 99.940% ( 870, 1300 ] 65 0.025% 99.966% ( 1300, 1900 ] 32 0.012% 99.978% ( 1900, 2900 ] 20 0.008% 99.986% ( 2900, 4400 ] 24 0.009% 99.995% ( 4400, 6600 ] 13 0.005% 100.000% ( 6600, 9900 ] 1 0.000% 100.001% ( 22000, 33000 ] 1 0.000% 100.001% ** Level 2 read latency histogram (micros): Count: 413019 Average: 20.5889 StdDev: 311.76 Min: 1 Median: 2.9111 Max: 38690 Percentiles: P50: 2.91 P75: 4.90 P99: 14.65 P99.9: 2535.19 P99.99: 3541.99 ------------------------------------------------------ [ 0, 1 ] 749 0.181% 0.181% ( 1, 2 ] 55800 13.510% 13.692% ### ( 2, 3 ] 164597 39.852% 53.544% ######## ( 3, 4 ] 51836 12.551% 66.094% ### ( 4, 6 ] 81327 19.691% 85.785% #### ( 6, 10 ] 47788 11.570% 97.356% ## ( 10, 15 ] 7304 1.768% 99.124% ( 15, 22 ] 1262 0.306% 99.430% ( 22, 34 ] 148 0.036% 99.465% ( 34, 51 ] 61 0.015% 99.480% ( 51, 76 ] 60 0.015% 99.495% ( 76, 110 ] 83 0.020% 99.515% ( 110, 170 ] 36 0.009% 99.524% ( 170, 250 ] 35 0.008% 99.532% ( 250, 380 ] 66 0.016% 99.548% ( 380, 580 ] 111 0.027% 99.575% ( 580, 870 ] 252 0.061% 99.636% ( 870, 1300 ] 404 0.098% 99.734% ( 1300, 1900 ] 431 0.104% 99.838% ( 1900, 2900 ] 403 0.098% 99.936% ( 2900, 4400 ] 525 0.127% 100.063% ( 4400, 6600 ] 245 0.059% 100.122% ( 6600, 9900 ] 58 0.014% 100.136% ( 9900, 14000 ] 14 0.003% 100.139% ( 14000, 22000 ] 15 0.004% 100.143% ( 22000, 33000 ] 9 0.002% 100.145% ( 33000, 50000 ] 5 0.001% 100.146% ** Level 3 read latency histogram (micros): Count: 1033753 Average: 13.9727 StdDev: 257.56 Min: 1 Median: 2.8564 Max: 59190 Percentiles: P50: 2.86 P75: 4.75 P99: 13.85 P99.9: 209.16 P99.99: 259.98 ------------------------------------------------------ [ 0, 1 ] 3867 0.374% 0.374% ( 1, 2 ] 170973 16.539% 16.913% ### ( 2, 3 ] 399403 38.636% 55.549% ######## ( 3, 4 ] 123502 11.947% 67.496% ## ( 4, 6 ] 207118 20.036% 87.532% #### ( 6, 10 ] 107665 10.415% 97.947% ## ( 10, 15 ] 14128 1.367% 99.313% ( 15, 22 ] 2551 0.247% 99.560% ( 22, 34 ] 187 0.018% 99.578% ( 34, 51 ] 81 0.008% 99.586% ( 51, 76 ] 179 0.017% 99.603% ( 76, 110 ] 211 0.020% 99.624% ( 110, 170 ] 1999 0.193% 99.817% ( 170, 250 ] 1747 0.169% 99.986% ( 250, 380 ] 503 0.049% 100.035% ( 380, 580 ] 415 0.040% 100.075% ( 580, 870 ] 413 0.040% 100.115% ( 870, 1300 ] 576 0.056% 100.171% ( 1300, 1900 ] 509 0.049% 100.220% ( 1900, 2900 ] 436 0.042% 100.262% ( 2900, 4400 ] 655 0.063% 100.326% ( 4400, 6600 ] 330 0.032% 100.357% ( 6600, 9900 ] 81 0.008% 100.365% ( 9900, 14000 ] 35 0.003% 100.369% ( 14000, 22000 ] 37 0.004% 100.372% ( 22000, 33000 ] 21 0.002% 100.374% ( 33000, 50000 ] 5 0.000% 100.375% ( 50000, 75000 ] 1 0.000% 100.375% ** Level 4 read latency histogram (micros): Count: 381210 Average: 58.0967 StdDev: 434.08 Min: 1 Median: 3.1025 Max: 54733 Percentiles: P50: 3.10 P75: 5.77 P99: 898.45 P99.9: 3796.71 P99.99: 4896.63 ------------------------------------------------------ [ 0, 1 ] 1790 0.470% 0.470% ( 1, 2 ] 59777 15.681% 16.150% ### ( 2, 3 ] 124822 32.744% 48.894% ####### ( 3, 4 ] 41137 10.791% 59.685% ## ( 4, 6 ] 66114 17.343% 77.028% ### ( 6, 10 ] 33166 8.700% 85.729% ## ( 10, 15 ] 5191 1.362% 87.090% ( 15, 22 ] 929 0.244% 87.334% ( 22, 34 ] 94 0.025% 87.359% ( 34, 51 ] 24 0.006% 87.365% ( 51, 76 ] 61 0.016% 87.381% ( 76, 110 ] 1032 0.271% 87.652% ( 110, 170 ] 18597 4.878% 92.530% # ( 170, 250 ] 16739 4.391% 96.921% # ( 250, 380 ] 4175 1.095% 98.016% ( 380, 580 ] 2253 0.591% 98.607% ( 580, 870 ] 1415 0.371% 98.979% ( 870, 1300 ] 1238 0.325% 99.303% ( 1300, 1900 ] 1105 0.290% 99.593% ( 1900, 2900 ] 786 0.206% 99.799% ( 2900, 4400 ] 642 0.168% 99.968% ( 4400, 6600 ] 376 0.099% 100.066% ( 6600, 9900 ] 102 0.027% 100.093% ( 9900, 14000 ] 23 0.006% 100.099% ( 14000, 22000 ] 21 0.006% 100.105% ( 22000, 33000 ] 10 0.003% 100.107% ( 33000, 50000 ] 10 0.003% 100.110% ( 50000, 75000 ] 2 0.001% 100.110% ** Level 5 read latency histogram (micros): Count: 304711 Average: 298.1135 StdDev: 819.78 Min: 81 Median: 183.7379 Max: 105188 Percentiles: P50: 183.74 P75: 237.74 P99: 3339.55 P99.9: 8005.85 P99.99: 34130.06 ------------------------------------------------------ ( 76, 110 ] 6876 2.257% 2.257% ( 110, 170 ] 126101 41.384% 43.640% ######## ( 170, 250 ] 112847 37.034% 80.674% ####### ( 250, 380 ] 27862 9.144% 89.818% ## ( 380, 580 ] 14369 4.716% 94.534% # ( 580, 870 ] 5863 1.924% 96.458% ( 870, 1300 ] 3020 0.991% 97.449% ( 1300, 1900 ] 2337 0.767% 98.216% ( 1900, 2900 ] 1850 0.607% 98.823% ( 2900, 4400 ] 1839 0.604% 99.427% ( 4400, 6600 ] 1300 0.427% 99.853% ( 6600, 9900 ] 334 0.110% 99.963% ( 9900, 14000 ] 41 0.013% 99.976% ( 14000, 22000 ] 17 0.006% 99.982% ( 22000, 33000 ] 23 0.008% 99.989% ( 33000, 50000 ] 23 0.008% 99.997% ( 50000, 75000 ] 7 0.002% 99.999% ( 75000, 110000 ] 3 0.001% 100.000% ** DB Stats ** Uptime(secs): 34.3 total, 11.6 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 86.01 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:12.730 H:M:S, 37.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 84.73 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:6.093 H:M:S, 52.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 283.637 micros/op 14094 ops/sec 60.165 seconds 847996 operations; 13.7 MB/s (213999 of 213999 found) Microseconds per read: Count: 847996 Average: 283.6412 StdDev: 1052.93 Min: 1 Median: 210.8845 Max: 254918 Percentiles: P50: 210.88 P75: 247.64 P99: 1129.17 P99.9: 17158.73 P99.99: 37586.13 ------------------------------------------------------ [ 0, 1 ] 2 0.000% 0.000% ( 1, 2 ] 8 0.001% 0.001% ( 2, 3 ] 34 0.004% 0.005% ( 3, 4 ] 48 0.006% 0.011% ( 4, 6 ] 87 0.010% 0.021% ( 6, 10 ] 329 0.039% 0.060% ( 10, 15 ] 2880 0.340% 0.400% ( 15, 22 ] 3568 0.421% 0.820% ( 22, 34 ] 8148 0.961% 1.781% ( 34, 51 ] 2340 0.276% 2.057% ( 51, 76 ] 240 0.028% 2.085% ( 76, 110 ] 851 0.100% 2.186% ( 110, 170 ] 169619 20.002% 22.188% #### ( 170, 250 ] 461484 54.421% 76.609% ########### ( 250, 380 ] 114293 13.478% 90.087% ### ( 380, 580 ] 55298 6.521% 96.608% # ( 580, 870 ] 17230 2.032% 98.639% ( 870, 1300 ] 5072 0.598% 99.238% ( 1300, 1900 ] 2034 0.240% 99.477% ( 1900, 2900 ] 1315 0.155% 99.633% ( 2900, 4400 ] 1051 0.124% 99.756% ( 4400, 6600 ] 583 0.069% 99.825% ( 6600, 9900 ] 227 0.027% 99.852% ( 9900, 14000 ] 193 0.023% 99.875% ( 14000, 22000 ] 542 0.064% 99.939% ( 22000, 33000 ] 412 0.049% 99.987% ( 33000, 50000 ] 86 0.010% 99.997% ( 50000, 75000 ] 11 0.001% 99.999% ( 75000, 110000 ] 5 0.001% 99.999% ( 110000, 170000 ] 3 0.000% 100.000% ( 170000, 250000 ] 2 0.000% 100.000% ( 250000, 380000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 12447957 rocksdb.bloom.filter.full.positive COUNT : 969308 rocksdb.bloom.filter.full.true.positive COUNT : 847620 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 376 rocksdb.memtable.miss COUNT : 847620 rocksdb.l0.hit COUNT : 27941 rocksdb.l1.hit COUNT : 15385 rocksdb.l2andup.hit COUNT : 804294 rocksdb.compaction.key.drop.new COUNT : 144373 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3612743 rocksdb.number.keys.read COUNT : 847996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3728350776 rocksdb.bytes.read COUNT : 847996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1499 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 35191487 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 17379026038 rocksdb.compact.write.bytes COUNT : 17222361337 rocksdb.flush.write.bytes COUNT : 3699303374 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 46 rocksdb.number.superversion_acquires COUNT : 757 rocksdb.number.superversion_releases COUNT : 718 rocksdb.number.superversion_cleanups COUNT : 183 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 215 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3669346304 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1314816 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 20316442472 rocksdb.non.last.level.read.count COUNT : 4764073 rocksdb.block.checksum.compute.count COUNT : 5186044 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 210.170479 P95 : 529.698119 P99 : 1127.867063 P100 : 254916.000000 COUNT : 847996 SUM : 239716203 rocksdb.db.write.micros P50 : 3.287780 P95 : 9.414267 P99 : 17.402040 P100 : 1047168.000000 COUNT : 3612743 SUM : 51752232 rocksdb.compaction.times.micros P50 : 1562962.962963 P95 : 7380000.000000 P99 : 35315602.000000 P100 : 35315602.000000 COUNT : 64 SUM : 153400936 rocksdb.compaction.times.cpu_micros P50 : 987500.000000 P95 : 4020000.000000 P99 : 16838825.000000 P100 : 16838825.000000 COUNT : 64 SUM : 96204439 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 1108.035714 P95 : 9363.750000 P99 : 12385.000000 P100 : 12385.000000 COUNT : 53 SUM : 108817 rocksdb.compaction.outfile.sync.micros P50 : 8146.875000 P95 : 47450.000000 P99 : 100900.000000 P100 : 118919.000000 COUNT : 178 SUM : 2259182 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 249.111111 P95 : 5930.000000 P99 : 8503.000000 P100 : 9633.000000 COUNT : 127 SUM : 175874 rocksdb.table.open.io.micros P50 : 12255.429293 P95 : 24392.168675 P99 : 32338.674699 P100 : 240894.000000 COUNT : 1499 SUM : 19135143 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 5.324621 P95 : 17.732968 P99 : 237.604844 P100 : 59195.000000 COUNT : 4207023 SUM : 90610239 rocksdb.read.block.get.micros P50 : 170.977461 P95 : 375.473437 P99 : 1109.026914 P100 : 105362.000000 COUNT : 977483 SUM : 206884028 rocksdb.write.raw.block.micros P50 : 0.580511 P95 : 1.781838 P99 : 3.939860 P100 : 155969.000000 COUNT : 5067359 SUM : 19019407 rocksdb.numfiles.in.singlecompaction P50 : 2.681818 P95 : 5.200000 P99 : 35.000000 P100 : 35.000000 COUNT : 76 SUM : 288 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.572955 P95 : 1219.138894 P99 : 2610.828526 P100 : 1047005.000000 COUNT : 187135 SUM : 35210942 rocksdb.sst.read.micros P50 : 2.909334 P95 : 228.961566 P99 : 502.290301 P100 : 105188.000000 COUNT : 4764087 SUM : 279785745 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 847996 SUM : 847996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3612743 SUM : 3728350776 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 227297.297297 P95 : 919500.000000 P99 : 1411398.000000 P100 : 1411398.000000 COUNT : 53 SUM : 17969995 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 218 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0