00:00:00.000  Started by user sys_sgci
00:00:00.007  Checking out git https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool into /var/jenkins_home/workspace/autotest-per-patch_script/33b20b30f0a51e6b52980845e0f6aa336787973ad45e341fbbf98d1b65b265d4 to read jbp/jenkins/jjb-config/jobs/autotest-upstream/autotest.groovy
00:00:00.008  The recommended git tool is: git
00:00:00.008  using credential 00000000-0000-0000-0000-000000000002
00:00:00.011   > git rev-parse --resolve-git-dir /var/jenkins_home/workspace/autotest-per-patch_script/33b20b30f0a51e6b52980845e0f6aa336787973ad45e341fbbf98d1b65b265d4/jbp/.git # timeout=10
00:00:00.023  Fetching changes from the remote Git repository
00:00:00.025   > git config remote.origin.url https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool # timeout=10
00:00:00.037  Using shallow fetch with depth 1
00:00:00.037  Fetching upstream changes from https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool
00:00:00.037   > git --version # timeout=10
00:00:00.048   > git --version # 'git version 2.39.2'
00:00:00.048  using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials
00:00:00.048  Setting http proxy: proxy-dmz.intel.com:911
00:00:00.048   > git fetch --tags --force --progress --depth=1 -- https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool refs/heads/master # timeout=5
00:00:02.113   > git rev-parse origin/FETCH_HEAD^{commit} # timeout=10
00:00:02.124   > git rev-parse FETCH_HEAD^{commit} # timeout=10
00:00:02.136  Checking out Revision a704ed4d86859cb8cbec080c78b138476da6ee34 (FETCH_HEAD)
00:00:02.136   > git config core.sparsecheckout # timeout=10
00:00:02.146   > git read-tree -mu HEAD # timeout=10
00:00:02.161   > git checkout -f a704ed4d86859cb8cbec080c78b138476da6ee34 # timeout=5
00:00:02.187  Commit message: "packer: Insert post-processors only if at least one is defined"
00:00:02.187   > git rev-list --no-walk a704ed4d86859cb8cbec080c78b138476da6ee34 # timeout=10
00:00:02.834  [Pipeline] Start of Pipeline
00:00:02.849  [Pipeline] library
00:00:02.851  Loading library shm_lib@master
00:00:02.851  Library shm_lib@master is cached. Copying from home.
00:00:02.883  [Pipeline] node
00:00:02.897  Running on ME2 in /var/jenkins/workspace/autotest-per-patch_2
00:00:02.899  [Pipeline] {
00:00:02.917  [Pipeline] cleanWs
00:00:02.927  [WS-CLEANUP] Deleting project workspace...
00:00:02.927  [WS-CLEANUP] Deferred wipeout is used...
00:00:02.935  [WS-CLEANUP] done
00:00:02.940  [Pipeline] stage
00:00:02.944  [Pipeline] { (Prologue)
00:00:03.032  [Pipeline] withCredentials
00:00:03.043   > git --version # timeout=10
00:00:03.058   > git --version # 'git version 2.39.2'
00:00:03.079  Masking supported pattern matches of $GIT_USERNAME or $GIT_PASSWORD or $GIT_ASKPASS
00:00:03.081  [Pipeline] {
00:00:03.091  [Pipeline] retry
00:00:03.093  [Pipeline] {
00:00:03.274  [Pipeline] sh
00:00:03.555  + git ls-remote https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool refs/heads/master
00:00:05.477  [Pipeline] }
00:00:05.501  [Pipeline] // retry
00:00:05.508  [Pipeline] }
00:00:05.535  [Pipeline] // withCredentials
00:00:05.543  [Pipeline] nodesByLabel
00:00:05.545  Found a total of 1 nodes with the 'sorcerer' label
00:00:05.557  [Pipeline] httpRequest
00:00:05.561  HttpMethod: GET
00:00:05.562  URL: http://10.211.164.101/packages/jbp_a704ed4d86859cb8cbec080c78b138476da6ee34.tar.gz
00:00:05.563  Sending request to url: http://10.211.164.101/packages/jbp_a704ed4d86859cb8cbec080c78b138476da6ee34.tar.gz
00:00:05.564  Response Code: HTTP/1.1 200 OK
00:00:05.564  Success: Status code 200 is in the accepted range: 200,404
00:00:05.565  Saving response body to /var/jenkins/workspace/autotest-per-patch_2/jbp_a704ed4d86859cb8cbec080c78b138476da6ee34.tar.gz
00:00:05.701  [Pipeline] sh
00:00:05.983  + tar --no-same-owner -xf jbp_a704ed4d86859cb8cbec080c78b138476da6ee34.tar.gz
00:00:06.004  [Pipeline] httpRequest
00:00:06.008  HttpMethod: GET
00:00:06.009  URL: http://10.211.164.101/packages/spdk_77a84e60e073c769797deff624cc274e83a9e621.tar.gz
00:00:06.009  Sending request to url: http://10.211.164.101/packages/spdk_77a84e60e073c769797deff624cc274e83a9e621.tar.gz
00:00:06.010  Response Code: HTTP/1.1 404 Not Found
00:00:06.010  Success: Status code 404 is in the accepted range: 200,404
00:00:06.011  Saving response body to /var/jenkins/workspace/autotest-per-patch_2/spdk_77a84e60e073c769797deff624cc274e83a9e621.tar.gz
00:00:06.018  [Pipeline] sh
00:00:06.299  + rm -f spdk_77a84e60e073c769797deff624cc274e83a9e621.tar.gz
00:00:06.316  [Pipeline] node
00:00:06.329  Running on GP10 in /var/jenkins/workspace/autotest-per-patch
00:00:06.332  [Pipeline] {
00:00:06.353  [Pipeline] cleanWs
00:00:06.363  [WS-CLEANUP] Deleting project workspace...
00:00:06.363  [WS-CLEANUP] Deferred wipeout is used...
00:00:06.370  [WS-CLEANUP] done
00:00:06.375  [Pipeline] sh
00:00:06.668  + find /storage/packages/ -type f -mtime +34 -print -delete
00:00:06.684  [Pipeline] lock
00:00:06.685  Trying to acquire lock on [sorcerer_spdk_77a84e60e073c769797deff624cc274e83a9e621]
00:00:06.688  Resource [sorcerer_spdk_77a84e60e073c769797deff624cc274e83a9e621] did not exist. Created.
00:00:06.690  Lock acquired on [sorcerer_spdk_77a84e60e073c769797deff624cc274e83a9e621]
00:00:06.693  [Pipeline] {
00:00:06.712  [Pipeline] fileExists
00:00:06.732  [Pipeline] retry
00:00:06.734  [Pipeline] {
00:00:06.759  [Pipeline] checkout
00:00:06.765  The recommended git tool is: NONE
00:00:06.778  using credential 00000000-0000-0000-0000-000000000002
00:00:06.784  Cloning the remote Git repository
00:00:06.788  Honoring refspec on initial clone
00:00:06.776  Cloning repository https://review.spdk.io/gerrit/a/spdk/spdk
00:00:06.776   > git init /var/jenkins/workspace/autotest-per-patch/spdk # timeout=10
00:00:06.783  Using reference repository: /var/ci_repos/spdk_multi
00:00:06.783  Fetching upstream changes from https://review.spdk.io/gerrit/a/spdk/spdk
00:00:06.783   > git --version # timeout=10
00:00:06.786   > git --version # 'git version 2.41.0'
00:00:06.786  using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials
00:00:06.787  Setting http proxy: proxy-dmz.intel.com:911
00:00:06.787   > git fetch --tags --force --progress -- https://review.spdk.io/gerrit/a/spdk/spdk refs/changes/49/22649/3 +refs/heads/master:refs/remotes/origin/master # timeout=10
00:00:22.058  Avoid second fetch
00:00:22.073  Checking out Revision 77a84e60e073c769797deff624cc274e83a9e621 (FETCH_HEAD)
00:00:22.332  Commit message: "nvmf/tcp: add nvmf_qpair_set_ctrlr helper function"
00:00:22.339  First time build. Skipping changelog.
00:00:22.023   > git config remote.origin.url https://review.spdk.io/gerrit/a/spdk/spdk # timeout=10
00:00:22.027   > git config --add remote.origin.fetch refs/changes/49/22649/3 # timeout=10
00:00:22.031   > git config --add remote.origin.fetch +refs/heads/master:refs/remotes/origin/master # timeout=10
00:00:22.044   > git rev-parse origin/FETCH_HEAD^{commit} # timeout=10
00:00:22.051   > git rev-parse FETCH_HEAD^{commit} # timeout=10
00:00:22.059   > git config core.sparsecheckout # timeout=10
00:00:22.063   > git checkout -f 77a84e60e073c769797deff624cc274e83a9e621 # timeout=10
00:00:22.317   > git rev-list --no-walk 21e05a094e97587f0a37f5056c38480834b2b487 # timeout=10
00:00:22.328   > git remote # timeout=10
00:00:22.332   > git submodule init # timeout=10
00:00:22.377   > git submodule sync # timeout=10
00:00:22.422   > git config --get remote.origin.url # timeout=10
00:00:22.429   > git submodule init # timeout=10
00:00:22.470   > git config -f .gitmodules --get-regexp ^submodule\.(.+)\.url # timeout=10
00:00:22.473   > git config --get submodule.dpdk.url # timeout=10
00:00:22.476   > git remote # timeout=10
00:00:22.480   > git config --get remote.origin.url # timeout=10
00:00:22.483   > git config -f .gitmodules --get submodule.dpdk.path # timeout=10
00:00:22.486   > git config --get submodule.intel-ipsec-mb.url # timeout=10
00:00:22.489   > git remote # timeout=10
00:00:22.493   > git config --get remote.origin.url # timeout=10
00:00:22.496   > git config -f .gitmodules --get submodule.intel-ipsec-mb.path # timeout=10
00:00:22.501   > git config --get submodule.isa-l.url # timeout=10
00:00:22.505   > git remote # timeout=10
00:00:22.508   > git config --get remote.origin.url # timeout=10
00:00:22.511   > git config -f .gitmodules --get submodule.isa-l.path # timeout=10
00:00:22.514   > git config --get submodule.ocf.url # timeout=10
00:00:22.517   > git remote # timeout=10
00:00:22.520   > git config --get remote.origin.url # timeout=10
00:00:22.523   > git config -f .gitmodules --get submodule.ocf.path # timeout=10
00:00:22.527   > git config --get submodule.libvfio-user.url # timeout=10
00:00:22.530   > git remote # timeout=10
00:00:22.533   > git config --get remote.origin.url # timeout=10
00:00:22.536   > git config -f .gitmodules --get submodule.libvfio-user.path # timeout=10
00:00:22.539   > git config --get submodule.xnvme.url # timeout=10
00:00:22.542   > git remote # timeout=10
00:00:22.546   > git config --get remote.origin.url # timeout=10
00:00:22.549   > git config -f .gitmodules --get submodule.xnvme.path # timeout=10
00:00:22.552   > git config --get submodule.isa-l-crypto.url # timeout=10
00:00:22.555   > git remote # timeout=10
00:00:22.559   > git config --get remote.origin.url # timeout=10
00:00:22.562   > git config -f .gitmodules --get submodule.isa-l-crypto.path # timeout=10
00:00:22.565  using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials
00:00:22.565  using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials
00:00:22.565  using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials
00:00:22.566  using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials
00:00:22.566  using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials
00:00:22.566  using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials
00:00:22.566  Setting http proxy: proxy-dmz.intel.com:911
00:00:22.566   > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi dpdk # timeout=10
00:00:22.566  using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials
00:00:22.566  Setting http proxy: proxy-dmz.intel.com:911
00:00:22.566   > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi intel-ipsec-mb # timeout=10
00:00:22.566  Setting http proxy: proxy-dmz.intel.com:911
00:00:22.566   > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi isa-l # timeout=10
00:00:22.566  Setting http proxy: proxy-dmz.intel.com:911
00:00:22.566  Setting http proxy: proxy-dmz.intel.com:911
00:00:22.566   > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi libvfio-user # timeout=10
00:00:22.566   > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi xnvme # timeout=10
00:00:22.566  Setting http proxy: proxy-dmz.intel.com:911
00:00:22.567   > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi ocf # timeout=10
00:00:22.567  Setting http proxy: proxy-dmz.intel.com:911
00:00:22.567   > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi isa-l-crypto # timeout=10
00:00:32.278  [Pipeline] dir
00:00:32.278  Running in /var/jenkins/workspace/autotest-per-patch/spdk
00:00:32.280  [Pipeline] {
00:00:32.302  [Pipeline] sh
00:00:32.591  ++ nproc
00:00:32.591  + threads=48
00:00:32.591  + git repack -a -d --threads=48
00:00:40.724  + git submodule foreach git repack -a -d --threads=48
00:00:40.724  Entering 'dpdk'
00:00:46.010  Entering 'intel-ipsec-mb'
00:00:46.010  Entering 'isa-l'
00:00:46.010  Entering 'isa-l-crypto'
00:00:46.270  Entering 'libvfio-user'
00:00:46.270  Entering 'ocf'
00:00:46.531  Entering 'xnvme'
00:00:47.100  + find .git -type f -name alternates -print -delete
00:00:47.100  .git/modules/libvfio-user/objects/info/alternates
00:00:47.100  .git/modules/dpdk/objects/info/alternates
00:00:47.100  .git/modules/ocf/objects/info/alternates
00:00:47.100  .git/modules/intel-ipsec-mb/objects/info/alternates
00:00:47.100  .git/modules/isa-l/objects/info/alternates
00:00:47.100  .git/modules/xnvme/objects/info/alternates
00:00:47.100  .git/modules/isa-l-crypto/objects/info/alternates
00:00:47.100  .git/objects/info/alternates
00:00:47.113  [Pipeline] }
00:00:47.138  [Pipeline] // dir
00:00:47.143  [Pipeline] }
00:00:47.167  [Pipeline] // retry
00:00:47.175  [Pipeline] sh
00:00:47.460  + tar -cf spdk_77a84e60e073c769797deff624cc274e83a9e621.tar.gz -I pigz spdk
00:00:48.043  [Pipeline] sh
00:00:48.329  + mv spdk_77a84e60e073c769797deff624cc274e83a9e621.tar.gz /storage/tmp/
00:00:48.602  [Pipeline] sh
00:00:48.887  + mv /storage/tmp/spdk_77a84e60e073c769797deff624cc274e83a9e621.tar.gz /storage/packages/
00:00:48.899  [Pipeline] }
00:00:48.903  Lock released on resource [sorcerer_spdk_77a84e60e073c769797deff624cc274e83a9e621]
00:00:48.927  [Pipeline] // lock
00:00:48.937  [Pipeline] cleanWs
00:00:48.948  [WS-CLEANUP] Deleting project workspace...
00:00:48.948  [WS-CLEANUP] Deferred wipeout is used...
00:00:48.955  [WS-CLEANUP] done
00:00:48.959  [Pipeline] }
00:00:48.991  [Pipeline] // node
00:00:49.002  [Pipeline] retry
00:00:49.004  [Pipeline] {
00:00:49.028  [Pipeline] httpRequest
00:00:49.033  HttpMethod: GET
00:00:49.033  URL: http://10.211.164.101/packages/spdk_77a84e60e073c769797deff624cc274e83a9e621.tar.gz
00:00:49.034  Sending request to url: http://10.211.164.101/packages/spdk_77a84e60e073c769797deff624cc274e83a9e621.tar.gz
00:00:49.036  Response Code: HTTP/1.1 200 OK
00:00:49.036  Success: Status code 200 is in the accepted range: 200,404
00:00:49.037  Saving response body to /var/jenkins/workspace/autotest-per-patch_2/spdk_77a84e60e073c769797deff624cc274e83a9e621.tar.gz
00:00:51.196  [Pipeline] }
00:00:51.219  [Pipeline] // retry
00:00:51.228  [Pipeline] sh
00:00:51.512  + tar --no-same-owner -xf spdk_77a84e60e073c769797deff624cc274e83a9e621.tar.gz
00:00:54.059  [Pipeline] sh
00:00:54.345  + git -C spdk log --oneline -n5
00:00:54.345  77a84e60e nvmf/tcp: add nvmf_qpair_set_ctrlr helper function
00:00:54.345  2731ac8c5 app/trace: emit owner descriptions
00:00:54.345  c064dc584 trace: rename trace_event's poller_id to owner_id
00:00:54.345  23f700383 trace: add concept of "owner" to trace files
00:00:54.345  67f328f92 trace: rename "per_lcore_history" to just "data"
00:00:54.365  [Pipeline] setCustomBuildProperty
00:00:54.377  [Pipeline] setCustomBuildProperty
00:00:54.388  [Pipeline] catchError
00:00:54.390  [Pipeline] {
00:00:54.413  [Pipeline] sh
00:00:54.698  + git -C spdk describe --tags --abbrev=0 origin/master
00:00:54.713  [Pipeline] sh
00:00:54.995  + git -C spdk describe --tags --abbrev=0 --exclude=LTS HEAD
00:00:55.022  [Pipeline] echo
00:00:55.025  Branch: master
00:00:55.029  [Pipeline] fileExists
00:00:55.042  [Pipeline] readJSON
00:00:55.054  [Pipeline] }
00:00:55.070  [Pipeline] // catchError
00:00:55.073  [Pipeline] }
00:00:55.089  [Pipeline] // stage
00:00:55.101  [Pipeline] catchError
00:00:55.102  [Pipeline] {
00:00:55.118  [Pipeline] stage
00:00:55.120  [Pipeline] { (Pre tests)
00:00:55.162  [Pipeline] parallel
00:00:55.174  [Pipeline] { (Branch: check-format-docker-autotest)
00:00:55.176  [Pipeline] { (Branch: check-so-deps-docker-autotest)
00:00:55.177  [Pipeline] { (Branch: doc-docker-autotest)
00:00:55.178  [Pipeline] { (Branch: build-files-docker-autotest)
00:00:55.194  [Pipeline] retry
00:00:55.196  [Pipeline] {
00:00:55.200  [Pipeline] retry
00:00:55.202  [Pipeline] {
00:00:55.205  [Pipeline] retry
00:00:55.206  [Pipeline] {
00:00:55.209  [Pipeline] retry
00:00:55.210  [Pipeline] {
00:00:55.231  [Pipeline] build
00:00:55.235  Scheduling project: check-format-docker-autotest
00:00:55.242  [Pipeline] build
00:00:55.245  Scheduling project: check-so-deps-docker-autotest
00:00:55.252  [Pipeline] build
00:00:55.256  Scheduling project: doc-docker-autotest
00:00:55.264  [Pipeline] build
00:00:55.267  Scheduling project: build-files-docker-autotest
00:01:00.851  Starting building: check-format-docker-autotest #19502
00:01:00.854  Starting building: check-so-deps-docker-autotest #19522
00:01:00.857  Starting building: build-files-docker-autotest #19494
00:01:00.862  Starting building: doc-docker-autotest #19665
00:01:34.579  Build doc-docker-autotest #19665 completed: SUCCESS
00:01:34.582  [Pipeline] }
00:01:34.616  [Pipeline] // retry
00:01:34.621  [Pipeline] }
00:01:46.102  Build check-format-docker-autotest #19502 completed: SUCCESS
00:01:46.105  [Pipeline] }
00:01:46.140  [Pipeline] // retry
00:01:46.146  [Pipeline] }
00:03:07.338  Build build-files-docker-autotest #19494 completed: SUCCESS
00:03:07.342  [Pipeline] }
00:03:07.382  [Pipeline] // retry
00:03:07.388  [Pipeline] }
00:03:45.519  Build check-so-deps-docker-autotest #19522 completed: SUCCESS
00:03:45.522  [Pipeline] }
00:03:45.555  [Pipeline] // retry
00:03:45.560  [Pipeline] }
00:03:45.611  [Pipeline] // parallel
00:03:45.616  [Pipeline] }
00:03:45.644  [Pipeline] // stage
00:03:45.650  [Pipeline] }
00:03:45.677  [Pipeline] // catchError
00:03:45.687  [Pipeline] catchError
00:03:45.688  [Pipeline] {
00:03:45.710  [Pipeline] stage
00:03:45.712  [Pipeline] { (Tests)
00:03:45.875  [Pipeline] parallel
00:03:45.965  [Pipeline] { (Branch: unittest-clang-docker-autotest)
00:03:45.966  [Pipeline] { (Branch: unittest-main-docker-autotest)
00:03:45.966  [Pipeline] { (Branch: autobuild-external-vg-autotest)
00:03:45.967  [Pipeline] { (Branch: scan-build-docker-autotest)
00:03:45.968  [Pipeline] { (Branch: unittest-valgrind-docker-autotest)
00:03:45.969  [Pipeline] { (Branch: lvol-vg-autotest)
00:03:45.970  [Pipeline] { (Branch: vhost-initiator-vg-autotest)
00:03:45.971  [Pipeline] { (Branch: nvmf-tcp-vg-autotest)
00:03:45.972  [Pipeline] { (Branch: nvmf-tcp-uring-vg-autotest)
00:03:45.973  [Pipeline] { (Branch: nvme-vg-autotest)
00:03:45.973  [Pipeline] { (Branch: centos7-vg-autotest)
00:03:45.974  [Pipeline] { (Branch: rocky9-vg-autotest)
00:03:45.975  [Pipeline] { (Branch: ubuntu20-vg-autotest)
00:03:45.976  [Pipeline] { (Branch: ubuntu22-vg-autotest)
00:03:45.977  [Pipeline] { (Branch: freebsd-vg-autotest)
00:03:45.978  [Pipeline] { (Branch: iscsi-vg-autotest)
00:03:45.979  [Pipeline] { (Branch: iscsi-uring-vg-autotest)
00:03:45.979  [Pipeline] { (Branch: zns-vg-autotest)
00:03:45.980  [Pipeline] { (Branch: nvme-cmb-pmr-vg-autotest)
00:03:45.981  [Pipeline] { (Branch: BlobFS-phy-autotest)
00:03:45.982  [Pipeline] { (Branch: crypto-phy-autotest)
00:03:45.983  [Pipeline] { (Branch: vhost-phy-autotest)
00:03:45.984  [Pipeline] { (Branch: nvmf-phy-autotest)
00:03:45.985  [Pipeline] { (Branch: nvmf-tcp-phy-autotest)
00:03:45.985  [Pipeline] { (Branch: nvme-phy-autotest)
00:03:45.986  [Pipeline] { (Branch: vfio-user-phy-autotest)
00:03:45.987  [Pipeline] { (Branch: short-fuzz-phy-autotest)
00:03:45.988  [Pipeline] { (Branch: packaging-docker-autotest)
00:03:46.017  [Pipeline] retry
00:03:46.019  [Pipeline] {
00:03:46.024  [Pipeline] retry
00:03:46.025  [Pipeline] {
00:03:46.030  [Pipeline] retry
00:03:46.031  [Pipeline] {
00:03:46.036  [Pipeline] retry
00:03:46.038  [Pipeline] {
00:03:46.043  [Pipeline] retry
00:03:46.044  [Pipeline] {
00:03:46.049  [Pipeline] retry
00:03:46.051  [Pipeline] {
00:03:46.056  [Pipeline] retry
00:03:46.057  [Pipeline] {
00:03:46.062  [Pipeline] retry
00:03:46.064  [Pipeline] {
00:03:46.068  [Pipeline] retry
00:03:46.070  [Pipeline] {
00:03:46.074  [Pipeline] retry
00:03:46.076  [Pipeline] {
00:03:46.080  [Pipeline] retry
00:03:46.082  [Pipeline] {
00:03:46.087  [Pipeline] retry
00:03:46.089  [Pipeline] {
00:03:46.093  [Pipeline] retry
00:03:46.095  [Pipeline] {
00:03:46.100  [Pipeline] retry
00:03:46.101  [Pipeline] {
00:03:46.106  [Pipeline] retry
00:03:46.107  [Pipeline] {
00:03:46.112  [Pipeline] retry
00:03:46.114  [Pipeline] {
00:03:46.118  [Pipeline] retry
00:03:46.120  [Pipeline] {
00:03:46.125  [Pipeline] retry
00:03:46.127  [Pipeline] {
00:03:46.131  [Pipeline] retry
00:03:46.133  [Pipeline] {
00:03:46.138  [Pipeline] retry
00:03:46.139  [Pipeline] {
00:03:46.144  [Pipeline] retry
00:03:46.145  [Pipeline] {
00:03:46.150  [Pipeline] retry
00:03:46.151  [Pipeline] {
00:03:46.156  [Pipeline] retry
00:03:46.157  [Pipeline] {
00:03:46.162  [Pipeline] retry
00:03:46.164  [Pipeline] {
00:03:46.168  [Pipeline] retry
00:03:46.170  [Pipeline] {
00:03:46.174  [Pipeline] retry
00:03:46.176  [Pipeline] {
00:03:46.180  [Pipeline] retry
00:03:46.182  [Pipeline] {
00:03:46.186  [Pipeline] retry
00:03:46.188  [Pipeline] {
00:03:46.215  [Pipeline] build
00:03:46.218  Scheduling project: unittest-clang-docker-autotest
00:03:46.225  [Pipeline] build
00:03:46.227  Scheduling project: unittest-main-docker-autotest
00:03:46.234  [Pipeline] build
00:03:46.238  Scheduling project: autobuild-external-vg-autotest
00:03:46.244  [Pipeline] build
00:03:46.247  Scheduling project: scan-build-docker-autotest
00:03:46.254  [Pipeline] build
00:03:46.257  Scheduling project: unittest-valgrind-docker-autotest
00:03:46.264  [Pipeline] build
00:03:46.266  Scheduling project: lvol-vg-autotest
00:03:46.273  [Pipeline] build
00:03:46.276  Scheduling project: vhost-initiator-vg-autotest
00:03:46.283  [Pipeline] build
00:03:46.285  Scheduling project: nvmf-tcp-vg-autotest
00:03:46.292  [Pipeline] build
00:03:46.294  Scheduling project: nvmf-tcp-uring-vg-autotest
00:03:46.301  [Pipeline] build
00:03:46.303  Scheduling project: nvme-vg-autotest
00:03:46.310  [Pipeline] build
00:03:46.312  Scheduling project: centos7-vg-autotest
00:03:46.340  [Pipeline] build
00:03:46.348  Scheduling project: rocky9-vg-autotest
00:03:46.355  [Pipeline] build
00:03:46.357  Scheduling project: ubuntu20-vg-autotest
00:03:46.363  [Pipeline] build
00:03:46.365  Scheduling project: ubuntu22-vg-autotest
00:03:46.371  [Pipeline] build
00:03:46.374  Scheduling project: freebsd-vg-autotest
00:03:46.380  [Pipeline] build
00:03:46.382  Scheduling project: iscsi-vg-autotest
00:03:46.388  [Pipeline] build
00:03:46.390  Scheduling project: iscsi-uring-vg-autotest
00:03:46.397  [Pipeline] build
00:03:46.400  Scheduling project: zns-vg-autotest
00:03:46.406  [Pipeline] build
00:03:46.409  Scheduling project: nvme-cmb-pmr-vg-autotest
00:03:46.415  [Pipeline] build
00:03:46.417  Scheduling project: BlobFS-phy-autotest
00:03:46.423  [Pipeline] build
00:03:46.426  Scheduling project: crypto-phy-autotest
00:03:46.431  [Pipeline] build
00:03:46.434  Scheduling project: vhost-phy-autotest
00:03:46.439  [Pipeline] build
00:03:46.441  Scheduling project: nvmf-phy-autotest
00:03:46.447  [Pipeline] build
00:03:46.449  Scheduling project: nvmf-tcp-phy-autotest
00:03:46.455  [Pipeline] build
00:03:46.457  Scheduling project: nvme-phy-autotest
00:03:46.463  [Pipeline] build
00:03:46.465  Scheduling project: vfio-user-phy-autotest
00:03:46.470  [Pipeline] build
00:03:46.472  Scheduling project: short-fuzz-phy-autotest
00:03:46.478  [Pipeline] build
00:03:46.480  Scheduling project: packaging-docker-autotest
00:03:56.197  Starting building: short-fuzz-phy-autotest #23405
00:03:56.202  Starting building: unittest-main-docker-autotest #17621
00:03:56.205  Starting building: crypto-phy-autotest #23409
00:03:56.207  Starting building: vfio-user-phy-autotest #23417
00:03:56.210  Starting building: rocky9-vg-autotest #13462
00:03:56.212  Starting building: scan-build-docker-autotest #17621
00:03:56.215  Starting building: packaging-docker-autotest #17622
00:03:56.217  Starting building: BlobFS-phy-autotest #23411
00:03:56.238  Starting building: nvme-phy-autotest #69577
00:03:56.240  Starting building: vhost-initiator-vg-autotest #74615
00:03:56.243  Starting building: nvmf-tcp-phy-autotest #69540
00:03:56.246  Starting building: nvmf-tcp-uring-vg-autotest #74515
00:03:56.249  Starting building: nvme-vg-autotest #74594
00:03:56.252  Starting building: nvmf-phy-autotest #69651
00:03:56.256  Starting building: vhost-phy-autotest #23432
00:03:56.261  Starting building: unittest-clang-docker-autotest #17619
00:03:56.263  Starting building: autobuild-external-vg-autotest #17454
00:03:56.267  Starting building: iscsi-vg-autotest #74631
00:03:56.269  Starting building: lvol-vg-autotest #74542
00:03:56.272  Starting building: nvme-cmb-pmr-vg-autotest #75716
00:03:56.275  Starting building: nvmf-tcp-vg-autotest #74582
00:03:56.279  Starting building: ubuntu20-vg-autotest #71172
00:03:56.283  Starting building: ubuntu22-vg-autotest #35184
00:03:56.288  Starting building: zns-vg-autotest #73696
00:03:56.291  Starting building: centos7-vg-autotest #71090
00:03:56.294  Starting building: iscsi-uring-vg-autotest #74496
00:03:56.297  Starting building: freebsd-vg-autotest #56903
00:03:56.300  Starting building: unittest-valgrind-docker-autotest #17624
00:06:07.884  Build short-fuzz-phy-autotest #23405 completed: FAILURE
00:06:07.933  [Pipeline] echo
00:06:07.935  No retry patterns found.
00:06:07.936  [Pipeline] }
00:06:07.979  [Pipeline] // retry
00:06:07.988  [Pipeline] error
00:06:07.994  [Pipeline] }
00:06:07.999  Failed in branch short-fuzz-phy-autotest
00:06:17.495  Build unittest-clang-docker-autotest #17619 completed: FAILURE
00:06:17.555  [Pipeline] echo
00:06:17.557  No retry patterns found.
00:06:17.558  [Pipeline] }
00:06:17.598  [Pipeline] // retry
00:06:17.605  [Pipeline] error
00:06:17.609  [Pipeline] }
00:06:17.612  Failed in branch unittest-clang-docker-autotest
00:07:54.662  Build packaging-docker-autotest #17622 completed: SUCCESS
00:07:54.665  [Pipeline] }
00:07:54.713  [Pipeline] // retry
00:07:54.720  [Pipeline] }
00:08:03.753  Build scan-build-docker-autotest #17621 completed: FAILURE
00:08:03.779  [Pipeline] echo
00:08:03.781  No retry patterns found.
00:08:03.783  [Pipeline] }
00:08:03.833  [Pipeline] // retry
00:08:03.843  [Pipeline] error
00:08:03.852  [Pipeline] }
00:08:03.859  Failed in branch scan-build-docker-autotest
00:09:05.700  Build unittest-main-docker-autotest #17621 completed: SUCCESS
00:09:05.702  [Pipeline] }
00:09:05.743  [Pipeline] // retry
00:09:05.750  [Pipeline] }
00:10:28.797  Build vfio-user-phy-autotest #23417 completed: FAILURE
00:10:29.141  [Pipeline] echo
00:10:29.142  No retry patterns found.
00:10:29.143  [Pipeline] }
00:10:29.183  [Pipeline] // retry
00:10:29.190  [Pipeline] error
00:10:29.195  [Pipeline] }
00:10:29.200  Failed in branch vfio-user-phy-autotest
00:12:33.303  Build unittest-valgrind-docker-autotest #17624 completed: SUCCESS
00:12:33.305  [Pipeline] }
00:12:33.346  [Pipeline] // retry
00:12:33.353  [Pipeline] }
00:12:38.353  Build BlobFS-phy-autotest #23411 completed: FAILURE
00:12:38.661  [Pipeline] echo
00:12:38.663  No retry patterns found.
00:12:38.664  [Pipeline] }
00:12:38.703  [Pipeline] // retry
00:12:38.710  [Pipeline] error
00:12:38.716  [Pipeline] }
00:12:38.721  Failed in branch BlobFS-phy-autotest
00:14:20.982  Build vhost-phy-autotest #23432 completed: FAILURE
00:14:21.403  [Pipeline] echo
00:14:21.404  No retry patterns found.
00:14:21.405  [Pipeline] }
00:14:21.449  [Pipeline] // retry
00:14:21.458  [Pipeline] error
00:14:21.466  [Pipeline] }
00:14:21.472  Failed in branch vhost-phy-autotest
00:18:19.491  Build vhost-initiator-vg-autotest #74615 completed: SUCCESS
00:18:19.494  [Pipeline] }
00:18:19.541  [Pipeline] // retry
00:18:19.547  [Pipeline] }
00:18:29.636  Build nvme-vg-autotest #74594 completed: FAILURE
00:18:29.940  [Pipeline] echo
00:18:29.941  No retry patterns found.
00:18:29.942  [Pipeline] }
00:18:29.978  [Pipeline] // retry
00:18:29.986  [Pipeline] error
00:18:29.990  [Pipeline] }
00:18:29.993  Failed in branch nvme-vg-autotest
00:18:51.381  Build rocky9-vg-autotest #13462 completed: FAILURE
00:18:51.807  [Pipeline] echo
00:18:51.809  No retry patterns found.
00:18:51.810  [Pipeline] }
00:18:51.840  [Pipeline] // retry
00:18:51.847  [Pipeline] error
00:18:51.853  [Pipeline] }
00:18:51.856  Failed in branch rocky9-vg-autotest
00:19:21.618  Build freebsd-vg-autotest #56903 completed: FAILURE
00:19:21.657  [Pipeline] echo
00:19:21.659  No retry patterns found.
00:19:21.660  [Pipeline] }
00:19:21.694  [Pipeline] // retry
00:19:21.701  [Pipeline] error
00:19:21.707  [Pipeline] }
00:19:21.712  Failed in branch freebsd-vg-autotest
00:19:32.937  Build lvol-vg-autotest #74542 completed: FAILURE
00:19:33.267  [Pipeline] echo
00:19:33.269  No retry patterns found.
00:19:33.269  [Pipeline] }
00:19:33.298  [Pipeline] // retry
00:19:33.305  [Pipeline] error
00:19:33.310  [Pipeline] }
00:19:33.316  Failed in branch lvol-vg-autotest
00:20:47.261  Build iscsi-vg-autotest #74631 completed: FAILURE
00:20:50.070  [Pipeline] echo
00:20:50.071  No retry patterns found.
00:20:50.072  [Pipeline] }
00:20:50.111  [Pipeline] // retry
00:20:50.119  [Pipeline] error
00:20:50.124  [Pipeline] }
00:20:50.128  Failed in branch iscsi-vg-autotest
00:22:47.486  Build ubuntu20-vg-autotest #71172 completed: FAILURE
00:22:47.875  [Pipeline] echo
00:22:47.876  No retry patterns found.
00:22:47.877  [Pipeline] }
00:22:47.918  [Pipeline] // retry
00:22:47.929  [Pipeline] error
00:22:47.935  [Pipeline] }
00:22:47.939  Failed in branch ubuntu20-vg-autotest
00:23:14.881  Build autobuild-external-vg-autotest #17454 completed: SUCCESS
00:23:14.884  [Pipeline] }
00:23:14.940  [Pipeline] // retry
00:23:14.945  [Pipeline] }
00:23:59.931  Build nvme-cmb-pmr-vg-autotest #75716 completed: SUCCESS
00:23:59.934  [Pipeline] }
00:23:59.984  [Pipeline] // retry
00:23:59.990  [Pipeline] }
00:24:25.007  Build iscsi-uring-vg-autotest #74496 completed: FAILURE
00:24:27.081  [Pipeline] echo
00:24:27.082  No retry patterns found.
00:24:27.083  [Pipeline] }
00:24:27.121  [Pipeline] // retry
00:24:27.129  [Pipeline] error
00:24:27.132  [Pipeline] }
00:24:27.135  Failed in branch iscsi-uring-vg-autotest
00:25:15.146  Build ubuntu22-vg-autotest #35184 completed: FAILURE
00:25:15.558  [Pipeline] echo
00:25:15.560  No retry patterns found.
00:25:15.561  [Pipeline] }
00:25:15.602  [Pipeline] // retry
00:25:15.611  [Pipeline] error
00:25:15.616  [Pipeline] }
00:25:15.621  Failed in branch ubuntu22-vg-autotest
00:25:42.729  Build zns-vg-autotest #73696 completed: SUCCESS
00:25:42.731  [Pipeline] }
00:25:42.784  [Pipeline] // retry
00:25:42.789  [Pipeline] }
00:26:56.441  Build crypto-phy-autotest #23409 completed: SUCCESS
00:26:56.445  [Pipeline] }
00:26:56.495  [Pipeline] // retry
00:26:56.502  [Pipeline] }
00:27:47.768  Build nvme-phy-autotest #69577 completed: SUCCESS
00:27:47.770  [Pipeline] }
00:27:47.820  [Pipeline] // retry
00:27:47.825  [Pipeline] }
00:28:05.960  Build nvmf-phy-autotest #69651 completed: SUCCESS
00:28:05.962  [Pipeline] }
00:28:06.009  [Pipeline] // retry
00:28:06.014  [Pipeline] }
00:28:14.433  Build nvmf-tcp-uring-vg-autotest #74515 completed: SUCCESS
00:28:14.435  [Pipeline] }
00:28:14.501  [Pipeline] // retry
00:28:14.507  [Pipeline] }
00:33:13.502  Build centos7-vg-autotest #71090 completed: SUCCESS
00:33:13.504  [Pipeline] }
00:33:13.552  [Pipeline] // retry
00:33:13.559  [Pipeline] }
00:33:24.013  Build nvmf-tcp-phy-autotest #69540 completed: SUCCESS
00:33:24.015  [Pipeline] }
00:33:24.067  [Pipeline] // retry
00:33:24.074  [Pipeline] }
00:35:04.634  Build nvmf-tcp-vg-autotest #74582 completed: SUCCESS
00:35:04.637  [Pipeline] }
00:35:04.674  [Pipeline] // retry
00:35:04.681  [Pipeline] }
00:35:04.859  [Pipeline] // parallel
00:35:04.876  [Pipeline] }
00:35:04.944  [Pipeline] // stage
00:35:04.966  [Pipeline] }
00:35:04.984  ERROR: Build short-fuzz-phy-autotest #23405 failed
00:35:05.006  [Pipeline] // catchError
00:35:05.013  [Pipeline] stage
00:35:05.014  [Pipeline] { (Autorun Post and Coverage)
00:35:05.034  [Pipeline] setCustomBuildProperty
00:35:05.051  [Pipeline] dir
00:35:05.052  Running in /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_19665
00:35:05.053  [Pipeline] {
00:35:05.073  [Pipeline] copyArtifacts
00:35:05.333  Copied 6 artifacts from "doc-docker-autotest" build number 19665
00:35:05.337  [Pipeline] writeFile
00:35:05.360  [Pipeline] }
00:35:05.392  [Pipeline] // dir
00:35:05.402  [Pipeline] dir
00:35:05.403  Running in /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_19502
00:35:05.404  [Pipeline] {
00:35:05.426  [Pipeline] copyArtifacts
00:35:05.496  Copied 5 artifacts from "check-format-docker-autotest" build number 19502
00:35:05.501  [Pipeline] writeFile
00:35:05.531  [Pipeline] }
00:35:05.566  [Pipeline] // dir
00:35:05.658  [Pipeline] dir
00:35:05.658  Running in /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_19494
00:35:05.659  [Pipeline] {
00:35:05.682  [Pipeline] copyArtifacts
00:35:05.753  Copied 5 artifacts from "build-files-docker-autotest" build number 19494
00:35:05.758  [Pipeline] writeFile
00:35:05.809  [Pipeline] }
00:35:05.845  [Pipeline] // dir
00:35:05.886  [Pipeline] dir
00:35:05.887  Running in /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_19522
00:35:05.888  [Pipeline] {
00:35:05.908  [Pipeline] copyArtifacts
00:35:05.986  Copied 5 artifacts from "check-so-deps-docker-autotest" build number 19522
00:35:05.990  [Pipeline] writeFile
00:35:06.020  [Pipeline] }
00:35:06.042  [Pipeline] // dir
00:35:06.091  [Pipeline] dir
00:35:06.091  Running in /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_23405
00:35:06.092  [Pipeline] {
00:35:06.114  [Pipeline] copyArtifacts
00:35:06.205  Copied 7 artifacts from "short-fuzz-phy-autotest" build number 23405
00:35:06.210  [Pipeline] writeFile
00:35:06.237  [Pipeline] }
00:35:06.265  [Pipeline] // dir
00:35:06.301  [Pipeline] dir
00:35:06.301  Running in /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_17619
00:35:06.302  [Pipeline] {
00:35:06.321  [Pipeline] copyArtifacts
00:35:06.394  Copied 5 artifacts from "unittest-clang-docker-autotest" build number 17619
00:35:06.398  [Pipeline] writeFile
00:35:06.431  [Pipeline] }
00:35:06.478  [Pipeline] // dir
00:35:06.776  [Pipeline] dir
00:35:06.776  Running in /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_17622
00:35:06.777  [Pipeline] {
00:35:06.818  [Pipeline] copyArtifacts
00:35:06.898  Copied 5 artifacts from "packaging-docker-autotest" build number 17622
00:35:06.904  [Pipeline] writeFile
00:35:07.094  [Pipeline] }
00:35:07.207  [Pipeline] // dir
00:35:07.239  [Pipeline] dir
00:35:07.239  Running in /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_17621
00:35:07.241  [Pipeline] {
00:35:07.268  [Pipeline] copyArtifacts
00:35:07.462  Copied 10 artifacts from "scan-build-docker-autotest" build number 17621
00:35:07.466  [Pipeline] writeFile
00:35:07.495  [Pipeline] }
00:35:07.532  [Pipeline] // dir
00:35:07.693  [Pipeline] dir
00:35:07.693  Running in /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_17621
00:35:07.694  [Pipeline] {
00:35:07.724  [Pipeline] copyArtifacts
00:35:08.030  Copied 6 artifacts from "unittest-main-docker-autotest" build number 17621
00:35:08.035  [Pipeline] writeFile
00:35:08.093  [Pipeline] }
00:35:08.139  [Pipeline] // dir
00:35:08.357  [Pipeline] dir
00:35:08.357  Running in /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_23417
00:35:08.358  [Pipeline] {
00:35:08.390  [Pipeline] copyArtifacts
00:35:10.046  Copied 16 artifacts from "vfio-user-phy-autotest" build number 23417
00:35:10.051  [Pipeline] writeFile
00:35:10.123  [Pipeline] }
00:35:10.188  [Pipeline] // dir
00:35:10.509  [Pipeline] dir
00:35:10.509  Running in /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_17624
00:35:10.510  [Pipeline] {
00:35:10.548  [Pipeline] copyArtifacts
00:35:10.796  Copied 6 artifacts from "unittest-valgrind-docker-autotest" build number 17624
00:35:10.800  [Pipeline] writeFile
00:35:10.973  [Pipeline] }
00:35:11.061  [Pipeline] // dir
00:35:11.273  [Pipeline] dir
00:35:11.273  Running in /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_23411
00:35:11.274  [Pipeline] {
00:35:11.303  [Pipeline] copyArtifacts
00:35:12.644  Copied 16 artifacts from "BlobFS-phy-autotest" build number 23411
00:35:12.648  [Pipeline] writeFile
00:35:12.706  [Pipeline] }
00:35:12.750  [Pipeline] // dir
00:35:12.952  [Pipeline] dir
00:35:12.952  Running in /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_23432
00:35:12.954  [Pipeline] {
00:35:12.982  [Pipeline] copyArtifacts
00:35:14.276  Copied 16 artifacts from "vhost-phy-autotest" build number 23432
00:35:14.280  [Pipeline] writeFile
00:35:14.343  [Pipeline] }
00:35:14.422  [Pipeline] // dir
00:35:14.718  [Pipeline] dir
00:35:14.719  Running in /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_74615
00:35:14.720  [Pipeline] {
00:35:14.752  [Pipeline] copyArtifacts
00:35:15.013  Copied 15 artifacts from "vhost-initiator-vg-autotest" build number 74615
00:35:15.017  [Pipeline] writeFile
00:35:15.166  [Pipeline] }
00:35:15.255  [Pipeline] // dir
00:35:15.460  [Pipeline] dir
00:35:15.460  Running in /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_74594
00:35:15.461  [Pipeline] {
00:35:15.494  [Pipeline] copyArtifacts
00:35:16.049  Copied 16 artifacts from "nvme-vg-autotest" build number 74594
00:35:16.053  [Pipeline] writeFile
00:35:16.164  [Pipeline] }
00:35:16.222  [Pipeline] // dir
00:35:16.525  [Pipeline] dir
00:35:16.525  Running in /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_13462
00:35:16.526  [Pipeline] {
00:35:16.560  [Pipeline] copyArtifacts
00:35:17.951  Copied 17 artifacts from "rocky9-vg-autotest" build number 13462
00:35:17.955  [Pipeline] writeFile
00:35:18.060  [Pipeline] }
00:35:18.137  [Pipeline] // dir
00:35:18.160  [Pipeline] dir
00:35:18.161  Running in /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_56903
00:35:18.162  [Pipeline] {
00:35:18.181  [Pipeline] copyArtifacts
00:35:18.221  Copied 4 artifacts from "freebsd-vg-autotest" build number 56903
00:35:18.224  [Pipeline] writeFile
00:35:18.243  [Pipeline] }
00:35:18.270  [Pipeline] // dir
00:35:18.474  [Pipeline] dir
00:35:18.475  Running in /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_74542
00:35:18.476  [Pipeline] {
00:35:18.506  [Pipeline] copyArtifacts
00:35:19.081  Copied 16 artifacts from "lvol-vg-autotest" build number 74542
00:35:19.086  [Pipeline] writeFile
00:35:19.161  [Pipeline] }
00:35:19.222  [Pipeline] // dir
00:35:21.398  [Pipeline] dir
00:35:21.398  Running in /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_74631
00:35:21.399  [Pipeline] {
00:35:21.581  [Pipeline] copyArtifacts
00:35:22.394  Copied 18 artifacts from "iscsi-vg-autotest" build number 74631
00:35:22.398  [Pipeline] writeFile
00:35:23.802  [Pipeline] }
00:35:24.370  [Pipeline] // dir
00:35:24.636  [Pipeline] dir
00:35:24.637  Running in /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_71172
00:35:24.638  [Pipeline] {
00:35:24.684  [Pipeline] copyArtifacts
00:35:26.865  Copied 17 artifacts from "ubuntu20-vg-autotest" build number 71172
00:35:26.869  [Pipeline] writeFile
00:35:26.980  [Pipeline] }
00:35:27.050  [Pipeline] // dir
00:35:27.189  [Pipeline] dir
00:35:27.189  Running in /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_17454
00:35:27.190  [Pipeline] {
00:35:27.218  [Pipeline] copyArtifacts
00:35:27.309  Copied 8 artifacts from "autobuild-external-vg-autotest" build number 17454
00:35:27.313  [Pipeline] writeFile
00:35:27.397  [Pipeline] }
00:35:27.455  [Pipeline] // dir
00:35:27.796  [Pipeline] dir
00:35:27.796  Running in /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_75716
00:35:27.797  [Pipeline] {
00:35:27.834  [Pipeline] copyArtifacts
00:35:28.061  Copied 12 artifacts from "nvme-cmb-pmr-vg-autotest" build number 75716
00:35:28.065  [Pipeline] writeFile
00:35:28.231  [Pipeline] }
00:35:28.305  [Pipeline] // dir
00:35:30.444  [Pipeline] dir
00:35:30.444  Running in /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_74496
00:35:30.445  [Pipeline] {
00:35:30.612  [Pipeline] copyArtifacts
00:35:31.554  Copied 19 artifacts from "iscsi-uring-vg-autotest" build number 74496
00:35:31.559  [Pipeline] writeFile
00:35:32.741  [Pipeline] }
00:35:33.191  [Pipeline] // dir
00:35:33.427  [Pipeline] dir
00:35:33.427  Running in /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_35184
00:35:33.428  [Pipeline] {
00:35:33.468  [Pipeline] copyArtifacts
00:35:35.667  Copied 17 artifacts from "ubuntu22-vg-autotest" build number 35184
00:35:35.671  [Pipeline] writeFile
00:35:35.768  [Pipeline] }
00:35:35.833  [Pipeline] // dir
00:35:36.127  [Pipeline] dir
00:35:36.128  Running in /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_73696
00:35:36.129  [Pipeline] {
00:35:36.164  [Pipeline] copyArtifacts
00:35:36.379  Copied 12 artifacts from "zns-vg-autotest" build number 73696
00:35:36.384  [Pipeline] writeFile
00:35:36.540  [Pipeline] }
00:35:36.625  [Pipeline] // dir
00:35:37.568  [Pipeline] dir
00:35:37.568  Running in /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_23409
00:35:37.569  [Pipeline] {
00:35:37.639  [Pipeline] copyArtifacts
00:35:37.848  Copied 12 artifacts from "crypto-phy-autotest" build number 23409
00:35:37.853  [Pipeline] writeFile
00:35:38.306  [Pipeline] }
00:35:38.548  [Pipeline] // dir
00:35:39.138  [Pipeline] dir
00:35:39.138  Running in /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_69577
00:35:39.139  [Pipeline] {
00:35:39.185  [Pipeline] copyArtifacts
00:35:39.426  Copied 12 artifacts from "nvme-phy-autotest" build number 69577
00:35:39.430  [Pipeline] writeFile
00:35:39.697  [Pipeline] }
00:35:39.834  [Pipeline] // dir
00:35:40.881  [Pipeline] dir
00:35:40.881  Running in /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_69651
00:35:40.882  [Pipeline] {
00:35:40.952  [Pipeline] copyArtifacts
00:35:41.225  Copied 13 artifacts from "nvmf-phy-autotest" build number 69651
00:35:41.229  [Pipeline] writeFile
00:35:41.690  [Pipeline] }
00:35:41.864  [Pipeline] // dir
00:35:42.711  [Pipeline] dir
00:35:42.712  Running in /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_74515
00:35:42.713  [Pipeline] {
00:35:42.768  [Pipeline] copyArtifacts
00:35:43.007  Copied 13 artifacts from "nvmf-tcp-uring-vg-autotest" build number 74515
00:35:43.012  [Pipeline] writeFile
00:35:43.355  [Pipeline] }
00:35:43.530  [Pipeline] // dir
00:35:44.151  [Pipeline] dir
00:35:44.151  Running in /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_71090
00:35:44.152  [Pipeline] {
00:35:44.203  [Pipeline] copyArtifacts
00:35:44.583  Copied 13 artifacts from "centos7-vg-autotest" build number 71090
00:35:44.589  [Pipeline] writeFile
00:35:44.814  [Pipeline] }
00:35:44.953  [Pipeline] // dir
00:35:46.216  [Pipeline] dir
00:35:46.216  Running in /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_69540
00:35:46.217  [Pipeline] {
00:35:46.298  [Pipeline] copyArtifacts
00:35:46.513  Copied 13 artifacts from "nvmf-tcp-phy-autotest" build number 69540
00:35:46.517  [Pipeline] writeFile
00:35:47.076  [Pipeline] }
00:35:47.304  [Pipeline] // dir
00:35:48.158  [Pipeline] dir
00:35:48.158  Running in /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_74582
00:35:48.159  [Pipeline] {
00:35:48.222  [Pipeline] copyArtifacts
00:35:48.448  Copied 13 artifacts from "nvmf-tcp-vg-autotest" build number 74582
00:35:48.453  [Pipeline] writeFile
00:35:48.933  [Pipeline] }
00:35:49.088  [Pipeline] // dir
00:35:49.098  [Pipeline] catchError
00:35:49.099  [Pipeline] {
00:35:49.117  [Pipeline] sh
00:35:49.394  + jbp/jenkins/jjb-config/jobs/scripts/post_gen_coverage.sh
00:35:49.394  + shopt -s globstar nullglob
00:35:49.394  + echo 'Start stage post_gen_coverage.sh'
00:35:49.394  Start stage post_gen_coverage.sh
00:35:49.394  + cd /var/jenkins/workspace/autotest-per-patch_2
00:35:49.394  + rm -rf /var/jenkins/workspace/autotest-per-patch_2/spdk/doc
00:35:49.394  + trap 'compress_coverage_and_docs; remove_partial_coverage_files && echo '\''End stage post_gen_coverage.sh'\''' EXIT
00:35:49.394  + move_artifacts
00:35:49.394  + local out_dirs
00:35:49.394  + out_dirs=(./**/output/)
00:35:49.394  + for dir in "${out_dirs[@]}"
00:35:49.394  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.394  + [[ -f ./autobuild-external-vg-autotest_17454/output//doc.tar.xz ]]
00:35:49.394  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.394  + [[ -f ./autobuild-external-vg-autotest_17454/output//ut_coverage.tar.xz ]]
00:35:49.394  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.394  + [[ -f ./autobuild-external-vg-autotest_17454/output//llvm.tar.xz ]]
00:35:49.394  + mv ./autobuild-external-vg-autotest_17454/output//build-pkg-manifest.txt ./autobuild-external-vg-autotest_17454/output//build-repo-manifest.txt ./autobuild-external-vg-autotest_17454/output//dmesg.init.log ./autobuild-external-vg-autotest_17454/output//dmesg.log ./autobuild-external-vg-autotest_17454/output//power.tar.xz ./autobuild-external-vg-autotest_17454/output//test_completions.txt ./autobuild-external-vg-autotest_17454/output//timing.svg ./autobuild-external-vg-autotest_17454/output//timing.txt ./autobuild-external-vg-autotest_17454/output//..
00:35:49.394  + rmdir ./autobuild-external-vg-autotest_17454/output/
00:35:49.394  + for dir in "${out_dirs[@]}"
00:35:49.394  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.394  + [[ -f ./BlobFS-phy-autotest_23411/output//doc.tar.xz ]]
00:35:49.394  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.394  + [[ -f ./BlobFS-phy-autotest_23411/output//ut_coverage.tar.xz ]]
00:35:49.394  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.395  + [[ -f ./BlobFS-phy-autotest_23411/output//llvm.tar.xz ]]
00:35:49.395  + mv ./BlobFS-phy-autotest_23411/output//all_tests.txt ./BlobFS-phy-autotest_23411/output//build-pkg-manifest.txt ./BlobFS-phy-autotest_23411/output//build-repo-manifest.txt ./BlobFS-phy-autotest_23411/output//coredumps ./BlobFS-phy-autotest_23411/output//cov_base.info.xz ./BlobFS-phy-autotest_23411/output//dmesg.init.log ./BlobFS-phy-autotest_23411/output//dmesg.log ./BlobFS-phy-autotest_23411/output//power.tar.xz ./BlobFS-phy-autotest_23411/output//proc_list.txt ./BlobFS-phy-autotest_23411/output//test_completions.txt ./BlobFS-phy-autotest_23411/output//timing.txt ./BlobFS-phy-autotest_23411/output//udev.log ./BlobFS-phy-autotest_23411/output//..
00:35:49.395  + rmdir ./BlobFS-phy-autotest_23411/output/
00:35:49.395  + for dir in "${out_dirs[@]}"
00:35:49.395  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.395  + [[ -f ./build-files-docker-autotest_19494/output//doc.tar.xz ]]
00:35:49.395  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.395  + [[ -f ./build-files-docker-autotest_19494/output//ut_coverage.tar.xz ]]
00:35:49.395  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.395  + [[ -f ./build-files-docker-autotest_19494/output//llvm.tar.xz ]]
00:35:49.395  + mv ./build-files-docker-autotest_19494/output//build-pkg-manifest.txt ./build-files-docker-autotest_19494/output//build-repo-manifest.txt ./build-files-docker-autotest_19494/output//power.tar.xz ./build-files-docker-autotest_19494/output//test_completions.txt ./build-files-docker-autotest_19494/output//timing.txt ./build-files-docker-autotest_19494/output//..
00:35:49.395  + rmdir ./build-files-docker-autotest_19494/output/
00:35:49.395  + for dir in "${out_dirs[@]}"
00:35:49.395  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.395  + [[ -f ./centos7-vg-autotest_71090/output//doc.tar.xz ]]
00:35:49.395  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.395  + [[ -f ./centos7-vg-autotest_71090/output//ut_coverage.tar.xz ]]
00:35:49.395  + tar -C ./centos7-vg-autotest_71090/output/ -xf ./centos7-vg-autotest_71090/output//ut_coverage.tar.xz
00:35:49.653  + rm ./centos7-vg-autotest_71090/output//ut_coverage.tar.xz
00:35:49.653  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.653  + [[ -f ./centos7-vg-autotest_71090/output//llvm.tar.xz ]]
00:35:49.653  + mv ./centos7-vg-autotest_71090/output//all_tests.txt ./centos7-vg-autotest_71090/output//build-pkg-manifest.txt ./centos7-vg-autotest_71090/output//build-repo-manifest.txt ./centos7-vg-autotest_71090/output//cov_base.info.xz ./centos7-vg-autotest_71090/output//cov_test.info.xz ./centos7-vg-autotest_71090/output//cov_total.info.xz ./centos7-vg-autotest_71090/output//dmesg.init.log ./centos7-vg-autotest_71090/output//dmesg.log ./centos7-vg-autotest_71090/output//power.tar.xz ./centos7-vg-autotest_71090/output//test_completions.txt ./centos7-vg-autotest_71090/output//timing.svg ./centos7-vg-autotest_71090/output//timing.txt ./centos7-vg-autotest_71090/output//ut_coverage ./centos7-vg-autotest_71090/output//..
00:35:49.653  + rmdir ./centos7-vg-autotest_71090/output/
00:35:49.653  + for dir in "${out_dirs[@]}"
00:35:49.653  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.653  + [[ -f ./check-format-docker-autotest_19502/output//doc.tar.xz ]]
00:35:49.653  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.653  + [[ -f ./check-format-docker-autotest_19502/output//ut_coverage.tar.xz ]]
00:35:49.653  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.653  + [[ -f ./check-format-docker-autotest_19502/output//llvm.tar.xz ]]
00:35:49.653  + mv ./check-format-docker-autotest_19502/output//build-pkg-manifest.txt ./check-format-docker-autotest_19502/output//build-repo-manifest.txt ./check-format-docker-autotest_19502/output//power.tar.xz ./check-format-docker-autotest_19502/output//test_completions.txt ./check-format-docker-autotest_19502/output//timing.txt ./check-format-docker-autotest_19502/output//..
00:35:49.653  + rmdir ./check-format-docker-autotest_19502/output/
00:35:49.653  + for dir in "${out_dirs[@]}"
00:35:49.653  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.653  + [[ -f ./check-so-deps-docker-autotest_19522/output//doc.tar.xz ]]
00:35:49.653  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.653  + [[ -f ./check-so-deps-docker-autotest_19522/output//ut_coverage.tar.xz ]]
00:35:49.653  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.653  + [[ -f ./check-so-deps-docker-autotest_19522/output//llvm.tar.xz ]]
00:35:49.653  + mv ./check-so-deps-docker-autotest_19522/output//build-pkg-manifest.txt ./check-so-deps-docker-autotest_19522/output//build-repo-manifest.txt ./check-so-deps-docker-autotest_19522/output//power.tar.xz ./check-so-deps-docker-autotest_19522/output//test_completions.txt ./check-so-deps-docker-autotest_19522/output//timing.txt ./check-so-deps-docker-autotest_19522/output//..
00:35:49.653  + rmdir ./check-so-deps-docker-autotest_19522/output/
00:35:49.653  + for dir in "${out_dirs[@]}"
00:35:49.653  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.653  + [[ -f ./crypto-phy-autotest_23409/output//doc.tar.xz ]]
00:35:49.653  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.653  + [[ -f ./crypto-phy-autotest_23409/output//ut_coverage.tar.xz ]]
00:35:49.653  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.653  + [[ -f ./crypto-phy-autotest_23409/output//llvm.tar.xz ]]
00:35:49.653  + mv ./crypto-phy-autotest_23409/output//all_tests.txt ./crypto-phy-autotest_23409/output//build-pkg-manifest.txt ./crypto-phy-autotest_23409/output//build-repo-manifest.txt ./crypto-phy-autotest_23409/output//cov_base.info.xz ./crypto-phy-autotest_23409/output//cov_test.info.xz ./crypto-phy-autotest_23409/output//cov_total.info.xz ./crypto-phy-autotest_23409/output//dmesg.init.log ./crypto-phy-autotest_23409/output//dmesg.log ./crypto-phy-autotest_23409/output//power.tar.xz ./crypto-phy-autotest_23409/output//test_completions.txt ./crypto-phy-autotest_23409/output//timing.svg ./crypto-phy-autotest_23409/output//timing.txt ./crypto-phy-autotest_23409/output//..
00:35:49.653  + rmdir ./crypto-phy-autotest_23409/output/
00:35:49.653  + for dir in "${out_dirs[@]}"
00:35:49.653  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.653  + [[ -f ./doc-docker-autotest_19665/output//doc.tar.xz ]]
00:35:49.654  + tar -C ./doc-docker-autotest_19665/output/ -xf ./doc-docker-autotest_19665/output//doc.tar.xz
00:35:49.654  + rm ./doc-docker-autotest_19665/output//doc.tar.xz
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./doc-docker-autotest_19665/output//ut_coverage.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./doc-docker-autotest_19665/output//llvm.tar.xz ]]
00:35:49.654  + mv ./doc-docker-autotest_19665/output//build-pkg-manifest.txt ./doc-docker-autotest_19665/output//build-repo-manifest.txt ./doc-docker-autotest_19665/output//doc ./doc-docker-autotest_19665/output//power.tar.xz ./doc-docker-autotest_19665/output//test_completions.txt ./doc-docker-autotest_19665/output//timing.txt ./doc-docker-autotest_19665/output//..
00:35:49.654  + rmdir ./doc-docker-autotest_19665/output/
00:35:49.654  + for dir in "${out_dirs[@]}"
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./freebsd-vg-autotest_56903/output//doc.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./freebsd-vg-autotest_56903/output//ut_coverage.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./freebsd-vg-autotest_56903/output//llvm.tar.xz ]]
00:35:49.654  + mv ./freebsd-vg-autotest_56903/output//build-pkg-manifest.txt ./freebsd-vg-autotest_56903/output//build-repo-manifest.txt ./freebsd-vg-autotest_56903/output//messages ./freebsd-vg-autotest_56903/output//power.tar.xz ./freebsd-vg-autotest_56903/output//..
00:35:49.654  + rmdir ./freebsd-vg-autotest_56903/output/
00:35:49.654  + for dir in "${out_dirs[@]}"
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./iscsi-uring-vg-autotest_74496/output//doc.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./iscsi-uring-vg-autotest_74496/output//ut_coverage.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./iscsi-uring-vg-autotest_74496/output//llvm.tar.xz ]]
00:35:49.654  + mv ./iscsi-uring-vg-autotest_74496/output//all_tests.txt ./iscsi-uring-vg-autotest_74496/output//build-kernel-manifest.txt ./iscsi-uring-vg-autotest_74496/output//build-pkg-manifest.txt ./iscsi-uring-vg-autotest_74496/output//build-repo-manifest.txt ./iscsi-uring-vg-autotest_74496/output//calsoft.json ./iscsi-uring-vg-autotest_74496/output//calsoft.tar.xz ./iscsi-uring-vg-autotest_74496/output//coredumps ./iscsi-uring-vg-autotest_74496/output//cov_base.info.xz ./iscsi-uring-vg-autotest_74496/output//dmesg.init.log ./iscsi-uring-vg-autotest_74496/output//dmesg.log ./iscsi-uring-vg-autotest_74496/output//power.tar.xz ./iscsi-uring-vg-autotest_74496/output//proc_list.txt ./iscsi-uring-vg-autotest_74496/output//test_completions.txt ./iscsi-uring-vg-autotest_74496/output//timing.txt ./iscsi-uring-vg-autotest_74496/output//udev.log ./iscsi-uring-vg-autotest_74496/output//..
00:35:49.654  + rmdir ./iscsi-uring-vg-autotest_74496/output/
00:35:49.654  + for dir in "${out_dirs[@]}"
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./iscsi-vg-autotest_74631/output//doc.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./iscsi-vg-autotest_74631/output//ut_coverage.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./iscsi-vg-autotest_74631/output//llvm.tar.xz ]]
00:35:49.654  + mv ./iscsi-vg-autotest_74631/output//all_tests.txt ./iscsi-vg-autotest_74631/output//build-pkg-manifest.txt ./iscsi-vg-autotest_74631/output//build-repo-manifest.txt ./iscsi-vg-autotest_74631/output//calsoft.json ./iscsi-vg-autotest_74631/output//calsoft.tar.xz ./iscsi-vg-autotest_74631/output//coredumps ./iscsi-vg-autotest_74631/output//cov_base.info.xz ./iscsi-vg-autotest_74631/output//dmesg.init.log ./iscsi-vg-autotest_74631/output//dmesg.log ./iscsi-vg-autotest_74631/output//power.tar.xz ./iscsi-vg-autotest_74631/output//proc_list.txt ./iscsi-vg-autotest_74631/output//test_completions.txt ./iscsi-vg-autotest_74631/output//timing.txt ./iscsi-vg-autotest_74631/output//udev.log ./iscsi-vg-autotest_74631/output//..
00:35:49.654  + rmdir ./iscsi-vg-autotest_74631/output/
00:35:49.654  + for dir in "${out_dirs[@]}"
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./lvol-vg-autotest_74542/output//doc.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./lvol-vg-autotest_74542/output//ut_coverage.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./lvol-vg-autotest_74542/output//llvm.tar.xz ]]
00:35:49.654  + mv ./lvol-vg-autotest_74542/output//all_tests.txt ./lvol-vg-autotest_74542/output//build-pkg-manifest.txt ./lvol-vg-autotest_74542/output//build-repo-manifest.txt ./lvol-vg-autotest_74542/output//coredumps ./lvol-vg-autotest_74542/output//cov_base.info.xz ./lvol-vg-autotest_74542/output//dmesg.init.log ./lvol-vg-autotest_74542/output//dmesg.log ./lvol-vg-autotest_74542/output//power.tar.xz ./lvol-vg-autotest_74542/output//proc_list.txt ./lvol-vg-autotest_74542/output//test_completions.txt ./lvol-vg-autotest_74542/output//timing.txt ./lvol-vg-autotest_74542/output//udev.log ./lvol-vg-autotest_74542/output//..
00:35:49.654  + rmdir ./lvol-vg-autotest_74542/output/
00:35:49.654  + for dir in "${out_dirs[@]}"
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvme-cmb-pmr-vg-autotest_75716/output//doc.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvme-cmb-pmr-vg-autotest_75716/output//ut_coverage.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvme-cmb-pmr-vg-autotest_75716/output//llvm.tar.xz ]]
00:35:49.654  + mv ./nvme-cmb-pmr-vg-autotest_75716/output//all_tests.txt ./nvme-cmb-pmr-vg-autotest_75716/output//build-pkg-manifest.txt ./nvme-cmb-pmr-vg-autotest_75716/output//build-repo-manifest.txt ./nvme-cmb-pmr-vg-autotest_75716/output//cov_base.info.xz ./nvme-cmb-pmr-vg-autotest_75716/output//cov_test.info.xz ./nvme-cmb-pmr-vg-autotest_75716/output//cov_total.info.xz ./nvme-cmb-pmr-vg-autotest_75716/output//dmesg.init.log ./nvme-cmb-pmr-vg-autotest_75716/output//dmesg.log ./nvme-cmb-pmr-vg-autotest_75716/output//power.tar.xz ./nvme-cmb-pmr-vg-autotest_75716/output//test_completions.txt ./nvme-cmb-pmr-vg-autotest_75716/output//timing.svg ./nvme-cmb-pmr-vg-autotest_75716/output//timing.txt ./nvme-cmb-pmr-vg-autotest_75716/output//..
00:35:49.654  + rmdir ./nvme-cmb-pmr-vg-autotest_75716/output/
00:35:49.654  + for dir in "${out_dirs[@]}"
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvme-phy-autotest_69577/output//doc.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvme-phy-autotest_69577/output//ut_coverage.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvme-phy-autotest_69577/output//llvm.tar.xz ]]
00:35:49.654  + mv ./nvme-phy-autotest_69577/output//all_tests.txt ./nvme-phy-autotest_69577/output//build-pkg-manifest.txt ./nvme-phy-autotest_69577/output//build-repo-manifest.txt ./nvme-phy-autotest_69577/output//cov_base.info.xz ./nvme-phy-autotest_69577/output//cov_test.info.xz ./nvme-phy-autotest_69577/output//cov_total.info.xz ./nvme-phy-autotest_69577/output//dmesg.init.log ./nvme-phy-autotest_69577/output//dmesg.log ./nvme-phy-autotest_69577/output//power.tar.xz ./nvme-phy-autotest_69577/output//test_completions.txt ./nvme-phy-autotest_69577/output//timing.svg ./nvme-phy-autotest_69577/output//timing.txt ./nvme-phy-autotest_69577/output//..
00:35:49.654  + rmdir ./nvme-phy-autotest_69577/output/
00:35:49.654  + for dir in "${out_dirs[@]}"
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvme-vg-autotest_74594/output//doc.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvme-vg-autotest_74594/output//ut_coverage.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvme-vg-autotest_74594/output//llvm.tar.xz ]]
00:35:49.654  + mv ./nvme-vg-autotest_74594/output//all_tests.txt ./nvme-vg-autotest_74594/output//build-pkg-manifest.txt ./nvme-vg-autotest_74594/output//build-repo-manifest.txt ./nvme-vg-autotest_74594/output//coredumps ./nvme-vg-autotest_74594/output//cov_base.info.xz ./nvme-vg-autotest_74594/output//dmesg.init.log ./nvme-vg-autotest_74594/output//dmesg.log ./nvme-vg-autotest_74594/output//power.tar.xz ./nvme-vg-autotest_74594/output//proc_list.txt ./nvme-vg-autotest_74594/output//test_completions.txt ./nvme-vg-autotest_74594/output//timing.txt ./nvme-vg-autotest_74594/output//udev.log ./nvme-vg-autotest_74594/output//..
00:35:49.654  + rmdir ./nvme-vg-autotest_74594/output/
00:35:49.654  + for dir in "${out_dirs[@]}"
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvmf-phy-autotest_69651/output//doc.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvmf-phy-autotest_69651/output//ut_coverage.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvmf-phy-autotest_69651/output//llvm.tar.xz ]]
00:35:49.654  + mv ./nvmf-phy-autotest_69651/output//all_tests.txt ./nvmf-phy-autotest_69651/output//build-pkg-manifest.txt ./nvmf-phy-autotest_69651/output//build-repo-manifest.txt ./nvmf-phy-autotest_69651/output//cov_base.info.xz ./nvmf-phy-autotest_69651/output//cov_test.info.xz ./nvmf-phy-autotest_69651/output//cov_total.info.xz ./nvmf-phy-autotest_69651/output//dmesg.init.log ./nvmf-phy-autotest_69651/output//dmesg.log ./nvmf-phy-autotest_69651/output//nvmf_trace.0_shm.tar.gz ./nvmf-phy-autotest_69651/output//power.tar.xz ./nvmf-phy-autotest_69651/output//test_completions.txt ./nvmf-phy-autotest_69651/output//timing.svg ./nvmf-phy-autotest_69651/output//timing.txt ./nvmf-phy-autotest_69651/output//..
00:35:49.654  + rmdir ./nvmf-phy-autotest_69651/output/
00:35:49.654  + for dir in "${out_dirs[@]}"
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvmf-tcp-phy-autotest_69540/output//doc.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvmf-tcp-phy-autotest_69540/output//ut_coverage.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvmf-tcp-phy-autotest_69540/output//llvm.tar.xz ]]
00:35:49.654  + mv ./nvmf-tcp-phy-autotest_69540/output//all_tests.txt ./nvmf-tcp-phy-autotest_69540/output//build-pkg-manifest.txt ./nvmf-tcp-phy-autotest_69540/output//build-repo-manifest.txt ./nvmf-tcp-phy-autotest_69540/output//cov_base.info.xz ./nvmf-tcp-phy-autotest_69540/output//cov_test.info.xz ./nvmf-tcp-phy-autotest_69540/output//cov_total.info.xz ./nvmf-tcp-phy-autotest_69540/output//dmesg.init.log ./nvmf-tcp-phy-autotest_69540/output//dmesg.log ./nvmf-tcp-phy-autotest_69540/output//nvmf_trace.0_shm.tar.gz ./nvmf-tcp-phy-autotest_69540/output//power.tar.xz ./nvmf-tcp-phy-autotest_69540/output//test_completions.txt ./nvmf-tcp-phy-autotest_69540/output//timing.svg ./nvmf-tcp-phy-autotest_69540/output//timing.txt ./nvmf-tcp-phy-autotest_69540/output//..
00:35:49.654  + rmdir ./nvmf-tcp-phy-autotest_69540/output/
00:35:49.654  + for dir in "${out_dirs[@]}"
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvmf-tcp-uring-vg-autotest_74515/output//doc.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvmf-tcp-uring-vg-autotest_74515/output//ut_coverage.tar.xz ]]
00:35:49.654  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.654  + [[ -f ./nvmf-tcp-uring-vg-autotest_74515/output//llvm.tar.xz ]]
00:35:49.655  + mv ./nvmf-tcp-uring-vg-autotest_74515/output//all_tests.txt ./nvmf-tcp-uring-vg-autotest_74515/output//build-pkg-manifest.txt ./nvmf-tcp-uring-vg-autotest_74515/output//build-repo-manifest.txt ./nvmf-tcp-uring-vg-autotest_74515/output//cov_base.info.xz ./nvmf-tcp-uring-vg-autotest_74515/output//cov_test.info.xz ./nvmf-tcp-uring-vg-autotest_74515/output//cov_total.info.xz ./nvmf-tcp-uring-vg-autotest_74515/output//dmesg.init.log ./nvmf-tcp-uring-vg-autotest_74515/output//dmesg.log ./nvmf-tcp-uring-vg-autotest_74515/output//nvmf_trace.0_shm.tar.gz ./nvmf-tcp-uring-vg-autotest_74515/output//power.tar.xz ./nvmf-tcp-uring-vg-autotest_74515/output//test_completions.txt ./nvmf-tcp-uring-vg-autotest_74515/output//timing.svg ./nvmf-tcp-uring-vg-autotest_74515/output//timing.txt ./nvmf-tcp-uring-vg-autotest_74515/output//..
00:35:49.655  + rmdir ./nvmf-tcp-uring-vg-autotest_74515/output/
00:35:49.655  + for dir in "${out_dirs[@]}"
00:35:49.655  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.655  + [[ -f ./nvmf-tcp-vg-autotest_74582/output//doc.tar.xz ]]
00:35:49.655  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.655  + [[ -f ./nvmf-tcp-vg-autotest_74582/output//ut_coverage.tar.xz ]]
00:35:49.655  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.655  + [[ -f ./nvmf-tcp-vg-autotest_74582/output//llvm.tar.xz ]]
00:35:49.655  + mv ./nvmf-tcp-vg-autotest_74582/output//all_tests.txt ./nvmf-tcp-vg-autotest_74582/output//build-pkg-manifest.txt ./nvmf-tcp-vg-autotest_74582/output//build-repo-manifest.txt ./nvmf-tcp-vg-autotest_74582/output//cov_base.info.xz ./nvmf-tcp-vg-autotest_74582/output//cov_test.info.xz ./nvmf-tcp-vg-autotest_74582/output//cov_total.info.xz ./nvmf-tcp-vg-autotest_74582/output//dmesg.init.log ./nvmf-tcp-vg-autotest_74582/output//dmesg.log ./nvmf-tcp-vg-autotest_74582/output//nvmf_trace.0_shm.tar.gz ./nvmf-tcp-vg-autotest_74582/output//power.tar.xz ./nvmf-tcp-vg-autotest_74582/output//test_completions.txt ./nvmf-tcp-vg-autotest_74582/output//timing.svg ./nvmf-tcp-vg-autotest_74582/output//timing.txt ./nvmf-tcp-vg-autotest_74582/output//..
00:35:49.655  + rmdir ./nvmf-tcp-vg-autotest_74582/output/
00:35:49.655  + for dir in "${out_dirs[@]}"
00:35:49.655  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.655  + [[ -f ./packaging-docker-autotest_17622/output//doc.tar.xz ]]
00:35:49.655  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.655  + [[ -f ./packaging-docker-autotest_17622/output//ut_coverage.tar.xz ]]
00:35:49.655  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.655  + [[ -f ./packaging-docker-autotest_17622/output//llvm.tar.xz ]]
00:35:49.655  + mv ./packaging-docker-autotest_17622/output//build-pkg-manifest.txt ./packaging-docker-autotest_17622/output//build-repo-manifest.txt ./packaging-docker-autotest_17622/output//power.tar.xz ./packaging-docker-autotest_17622/output//test_completions.txt ./packaging-docker-autotest_17622/output//timing.txt ./packaging-docker-autotest_17622/output//..
00:35:49.655  + rmdir ./packaging-docker-autotest_17622/output/
00:35:49.655  + for dir in "${out_dirs[@]}"
00:35:49.655  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.655  + [[ -f ./rocky9-vg-autotest_13462/output//doc.tar.xz ]]
00:35:49.655  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.655  + [[ -f ./rocky9-vg-autotest_13462/output//ut_coverage.tar.xz ]]
00:35:49.655  + tar -C ./rocky9-vg-autotest_13462/output/ -xf ./rocky9-vg-autotest_13462/output//ut_coverage.tar.xz
00:35:49.913  + rm ./rocky9-vg-autotest_13462/output//ut_coverage.tar.xz
00:35:49.913  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.913  + [[ -f ./rocky9-vg-autotest_13462/output//llvm.tar.xz ]]
00:35:49.913  + mv ./rocky9-vg-autotest_13462/output//all_tests.txt ./rocky9-vg-autotest_13462/output//build-pkg-manifest.txt ./rocky9-vg-autotest_13462/output//build-repo-manifest.txt ./rocky9-vg-autotest_13462/output//coredumps ./rocky9-vg-autotest_13462/output//cov_base.info.xz ./rocky9-vg-autotest_13462/output//dmesg.init.log ./rocky9-vg-autotest_13462/output//dmesg.log ./rocky9-vg-autotest_13462/output//power.tar.xz ./rocky9-vg-autotest_13462/output//proc_list.txt ./rocky9-vg-autotest_13462/output//test_completions.txt ./rocky9-vg-autotest_13462/output//timing.txt ./rocky9-vg-autotest_13462/output//udev.log ./rocky9-vg-autotest_13462/output//ut_coverage ./rocky9-vg-autotest_13462/output//..
00:35:49.913  + rmdir ./rocky9-vg-autotest_13462/output/
00:35:49.913  + for dir in "${out_dirs[@]}"
00:35:49.913  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.913  + [[ -f ./scan-build-docker-autotest_17621/output//doc.tar.xz ]]
00:35:49.913  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.913  + [[ -f ./scan-build-docker-autotest_17621/output//ut_coverage.tar.xz ]]
00:35:49.913  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.913  + [[ -f ./scan-build-docker-autotest_17621/output//llvm.tar.xz ]]
00:35:49.913  + mv ./scan-build-docker-autotest_17621/output//build_output.txt ./scan-build-docker-autotest_17621/output//build-pkg-manifest.txt ./scan-build-docker-autotest_17621/output//build-repo-manifest.txt ./scan-build-docker-autotest_17621/output//power.tar.xz ./scan-build-docker-autotest_17621/output//scan-build ./scan-build-docker-autotest_17621/output//test_completions.txt ./scan-build-docker-autotest_17621/output//timing.txt ./scan-build-docker-autotest_17621/output//..
00:35:49.913  + rmdir ./scan-build-docker-autotest_17621/output/
00:35:49.913  + for dir in "${out_dirs[@]}"
00:35:49.913  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.913  + [[ -f ./short-fuzz-phy-autotest_23405/output//doc.tar.xz ]]
00:35:49.913  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.913  + [[ -f ./short-fuzz-phy-autotest_23405/output//ut_coverage.tar.xz ]]
00:35:49.913  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.913  + [[ -f ./short-fuzz-phy-autotest_23405/output//llvm.tar.xz ]]
00:35:49.913  + mv ./short-fuzz-phy-autotest_23405/output//build-pkg-manifest.txt ./short-fuzz-phy-autotest_23405/output//build-repo-manifest.txt ./short-fuzz-phy-autotest_23405/output//dmesg.init.log ./short-fuzz-phy-autotest_23405/output//dmesg.log ./short-fuzz-phy-autotest_23405/output//power.tar.xz ./short-fuzz-phy-autotest_23405/output//test_completions.txt ./short-fuzz-phy-autotest_23405/output//timing.txt ./short-fuzz-phy-autotest_23405/output//..
00:35:49.913  + rmdir ./short-fuzz-phy-autotest_23405/output/
00:35:49.913  + for dir in "${out_dirs[@]}"
00:35:49.913  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.913  + [[ -f ./ubuntu20-vg-autotest_71172/output//doc.tar.xz ]]
00:35:49.913  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:49.913  + [[ -f ./ubuntu20-vg-autotest_71172/output//ut_coverage.tar.xz ]]
00:35:49.913  + tar -C ./ubuntu20-vg-autotest_71172/output/ -xf ./ubuntu20-vg-autotest_71172/output//ut_coverage.tar.xz
00:35:50.170  + rm ./ubuntu20-vg-autotest_71172/output//ut_coverage.tar.xz
00:35:50.170  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.170  + [[ -f ./ubuntu20-vg-autotest_71172/output//llvm.tar.xz ]]
00:35:50.170  + mv ./ubuntu20-vg-autotest_71172/output//all_tests.txt ./ubuntu20-vg-autotest_71172/output//build-pkg-manifest.txt ./ubuntu20-vg-autotest_71172/output//build-repo-manifest.txt ./ubuntu20-vg-autotest_71172/output//coredumps ./ubuntu20-vg-autotest_71172/output//cov_base.info.xz ./ubuntu20-vg-autotest_71172/output//dmesg.init.log ./ubuntu20-vg-autotest_71172/output//dmesg.log ./ubuntu20-vg-autotest_71172/output//power.tar.xz ./ubuntu20-vg-autotest_71172/output//proc_list.txt ./ubuntu20-vg-autotest_71172/output//test_completions.txt ./ubuntu20-vg-autotest_71172/output//timing.txt ./ubuntu20-vg-autotest_71172/output//udev.log ./ubuntu20-vg-autotest_71172/output//ut_coverage ./ubuntu20-vg-autotest_71172/output//..
00:35:50.170  + rmdir ./ubuntu20-vg-autotest_71172/output/
00:35:50.170  + for dir in "${out_dirs[@]}"
00:35:50.171  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.171  + [[ -f ./ubuntu22-vg-autotest_35184/output//doc.tar.xz ]]
00:35:50.171  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.171  + [[ -f ./ubuntu22-vg-autotest_35184/output//ut_coverage.tar.xz ]]
00:35:50.171  + tar -C ./ubuntu22-vg-autotest_35184/output/ -xf ./ubuntu22-vg-autotest_35184/output//ut_coverage.tar.xz
00:35:50.428  + rm ./ubuntu22-vg-autotest_35184/output//ut_coverage.tar.xz
00:35:50.428  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.428  + [[ -f ./ubuntu22-vg-autotest_35184/output//llvm.tar.xz ]]
00:35:50.428  + mv ./ubuntu22-vg-autotest_35184/output//all_tests.txt ./ubuntu22-vg-autotest_35184/output//build-pkg-manifest.txt ./ubuntu22-vg-autotest_35184/output//build-repo-manifest.txt ./ubuntu22-vg-autotest_35184/output//coredumps ./ubuntu22-vg-autotest_35184/output//cov_base.info.xz ./ubuntu22-vg-autotest_35184/output//dmesg.init.log ./ubuntu22-vg-autotest_35184/output//dmesg.log ./ubuntu22-vg-autotest_35184/output//power.tar.xz ./ubuntu22-vg-autotest_35184/output//proc_list.txt ./ubuntu22-vg-autotest_35184/output//test_completions.txt ./ubuntu22-vg-autotest_35184/output//timing.txt ./ubuntu22-vg-autotest_35184/output//udev.log ./ubuntu22-vg-autotest_35184/output//ut_coverage ./ubuntu22-vg-autotest_35184/output//..
00:35:50.428  + rmdir ./ubuntu22-vg-autotest_35184/output/
00:35:50.428  + for dir in "${out_dirs[@]}"
00:35:50.428  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.428  + [[ -f ./unittest-clang-docker-autotest_17619/output//doc.tar.xz ]]
00:35:50.428  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.428  + [[ -f ./unittest-clang-docker-autotest_17619/output//ut_coverage.tar.xz ]]
00:35:50.428  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.428  + [[ -f ./unittest-clang-docker-autotest_17619/output//llvm.tar.xz ]]
00:35:50.428  + mv ./unittest-clang-docker-autotest_17619/output//build-pkg-manifest.txt ./unittest-clang-docker-autotest_17619/output//build-repo-manifest.txt ./unittest-clang-docker-autotest_17619/output//power.tar.xz ./unittest-clang-docker-autotest_17619/output//test_completions.txt ./unittest-clang-docker-autotest_17619/output//timing.txt ./unittest-clang-docker-autotest_17619/output//..
00:35:50.428  + rmdir ./unittest-clang-docker-autotest_17619/output/
00:35:50.428  + for dir in "${out_dirs[@]}"
00:35:50.428  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.428  + [[ -f ./unittest-main-docker-autotest_17621/output//doc.tar.xz ]]
00:35:50.428  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.428  + [[ -f ./unittest-main-docker-autotest_17621/output//ut_coverage.tar.xz ]]
00:35:50.428  + tar -C ./unittest-main-docker-autotest_17621/output/ -xf ./unittest-main-docker-autotest_17621/output//ut_coverage.tar.xz
00:35:50.686  + rm ./unittest-main-docker-autotest_17621/output//ut_coverage.tar.xz
00:35:50.686  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.686  + [[ -f ./unittest-main-docker-autotest_17621/output//llvm.tar.xz ]]
00:35:50.686  + mv ./unittest-main-docker-autotest_17621/output//build-pkg-manifest.txt ./unittest-main-docker-autotest_17621/output//build-repo-manifest.txt ./unittest-main-docker-autotest_17621/output//power.tar.xz ./unittest-main-docker-autotest_17621/output//test_completions.txt ./unittest-main-docker-autotest_17621/output//timing.txt ./unittest-main-docker-autotest_17621/output//ut_coverage ./unittest-main-docker-autotest_17621/output//..
00:35:50.686  + rmdir ./unittest-main-docker-autotest_17621/output/
00:35:50.686  + for dir in "${out_dirs[@]}"
00:35:50.686  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.686  + [[ -f ./unittest-valgrind-docker-autotest_17624/output//doc.tar.xz ]]
00:35:50.686  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.686  + [[ -f ./unittest-valgrind-docker-autotest_17624/output//ut_coverage.tar.xz ]]
00:35:50.686  + tar -C ./unittest-valgrind-docker-autotest_17624/output/ -xf ./unittest-valgrind-docker-autotest_17624/output//ut_coverage.tar.xz
00:35:50.944  + rm ./unittest-valgrind-docker-autotest_17624/output//ut_coverage.tar.xz
00:35:50.944  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.944  + [[ -f ./unittest-valgrind-docker-autotest_17624/output//llvm.tar.xz ]]
00:35:50.944  + mv ./unittest-valgrind-docker-autotest_17624/output//build-pkg-manifest.txt ./unittest-valgrind-docker-autotest_17624/output//build-repo-manifest.txt ./unittest-valgrind-docker-autotest_17624/output//power.tar.xz ./unittest-valgrind-docker-autotest_17624/output//test_completions.txt ./unittest-valgrind-docker-autotest_17624/output//timing.txt ./unittest-valgrind-docker-autotest_17624/output//ut_coverage ./unittest-valgrind-docker-autotest_17624/output//..
00:35:50.944  + rmdir ./unittest-valgrind-docker-autotest_17624/output/
00:35:50.944  + for dir in "${out_dirs[@]}"
00:35:50.944  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.944  + [[ -f ./vfio-user-phy-autotest_23417/output//doc.tar.xz ]]
00:35:50.944  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.944  + [[ -f ./vfio-user-phy-autotest_23417/output//ut_coverage.tar.xz ]]
00:35:50.944  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.944  + [[ -f ./vfio-user-phy-autotest_23417/output//llvm.tar.xz ]]
00:35:50.944  + mv ./vfio-user-phy-autotest_23417/output//all_tests.txt ./vfio-user-phy-autotest_23417/output//build-pkg-manifest.txt ./vfio-user-phy-autotest_23417/output//build-repo-manifest.txt ./vfio-user-phy-autotest_23417/output//coredumps ./vfio-user-phy-autotest_23417/output//cov_base.info.xz ./vfio-user-phy-autotest_23417/output//dmesg.init.log ./vfio-user-phy-autotest_23417/output//dmesg.log ./vfio-user-phy-autotest_23417/output//power.tar.xz ./vfio-user-phy-autotest_23417/output//proc_list.txt ./vfio-user-phy-autotest_23417/output//test_completions.txt ./vfio-user-phy-autotest_23417/output//timing.txt ./vfio-user-phy-autotest_23417/output//udev.log ./vfio-user-phy-autotest_23417/output//..
00:35:50.944  + rmdir ./vfio-user-phy-autotest_23417/output/
00:35:50.944  + for dir in "${out_dirs[@]}"
00:35:50.944  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.944  + [[ -f ./vhost-initiator-vg-autotest_74615/output//doc.tar.xz ]]
00:35:50.944  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.944  + [[ -f ./vhost-initiator-vg-autotest_74615/output//ut_coverage.tar.xz ]]
00:35:50.944  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.944  + [[ -f ./vhost-initiator-vg-autotest_74615/output//llvm.tar.xz ]]
00:35:50.944  + mv ./vhost-initiator-vg-autotest_74615/output//all_tests.txt ./vhost-initiator-vg-autotest_74615/output//build-pkg-manifest.txt ./vhost-initiator-vg-autotest_74615/output//build-repo-manifest.txt ./vhost-initiator-vg-autotest_74615/output//cov_base.info.xz ./vhost-initiator-vg-autotest_74615/output//cov_test.info.xz ./vhost-initiator-vg-autotest_74615/output//cov_total.info.xz ./vhost-initiator-vg-autotest_74615/output//dmesg.init.log ./vhost-initiator-vg-autotest_74615/output//dmesg.log ./vhost-initiator-vg-autotest_74615/output//power.tar.xz ./vhost-initiator-vg-autotest_74615/output//test_completions.txt ./vhost-initiator-vg-autotest_74615/output//timing.svg ./vhost-initiator-vg-autotest_74615/output//timing.txt ./vhost-initiator-vg-autotest_74615/output//vhost_fuzz_output1.txt.xz ./vhost-initiator-vg-autotest_74615/output//vhost_fuzz_output2.txt.xz ./vhost-initiator-vg-autotest_74615/output//vhost_fuzz_tgt_output.txt.xz ./vhost-initiator-vg-autotest_74615/output//..
00:35:50.944  + rmdir ./vhost-initiator-vg-autotest_74615/output/
00:35:50.944  + for dir in "${out_dirs[@]}"
00:35:50.944  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.944  + [[ -f ./vhost-phy-autotest_23432/output//doc.tar.xz ]]
00:35:50.944  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.944  + [[ -f ./vhost-phy-autotest_23432/output//ut_coverage.tar.xz ]]
00:35:50.944  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.944  + [[ -f ./vhost-phy-autotest_23432/output//llvm.tar.xz ]]
00:35:50.944  + mv ./vhost-phy-autotest_23432/output//all_tests.txt ./vhost-phy-autotest_23432/output//build-pkg-manifest.txt ./vhost-phy-autotest_23432/output//build-repo-manifest.txt ./vhost-phy-autotest_23432/output//coredumps ./vhost-phy-autotest_23432/output//cov_base.info.xz ./vhost-phy-autotest_23432/output//dmesg.init.log ./vhost-phy-autotest_23432/output//dmesg.log ./vhost-phy-autotest_23432/output//power.tar.xz ./vhost-phy-autotest_23432/output//proc_list.txt ./vhost-phy-autotest_23432/output//test_completions.txt ./vhost-phy-autotest_23432/output//timing.txt ./vhost-phy-autotest_23432/output//udev.log ./vhost-phy-autotest_23432/output//..
00:35:50.944  + rmdir ./vhost-phy-autotest_23432/output/
00:35:50.944  + for dir in "${out_dirs[@]}"
00:35:50.944  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.944  + [[ -f ./zns-vg-autotest_73696/output//doc.tar.xz ]]
00:35:50.944  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.944  + [[ -f ./zns-vg-autotest_73696/output//ut_coverage.tar.xz ]]
00:35:50.944  + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz
00:35:50.944  + [[ -f ./zns-vg-autotest_73696/output//llvm.tar.xz ]]
00:35:50.944  + mv ./zns-vg-autotest_73696/output//all_tests.txt ./zns-vg-autotest_73696/output//build-pkg-manifest.txt ./zns-vg-autotest_73696/output//build-repo-manifest.txt ./zns-vg-autotest_73696/output//cov_base.info.xz ./zns-vg-autotest_73696/output//cov_test.info.xz ./zns-vg-autotest_73696/output//cov_total.info.xz ./zns-vg-autotest_73696/output//dmesg.init.log ./zns-vg-autotest_73696/output//dmesg.log ./zns-vg-autotest_73696/output//power.tar.xz ./zns-vg-autotest_73696/output//test_completions.txt ./zns-vg-autotest_73696/output//timing.svg ./zns-vg-autotest_73696/output//timing.txt ./zns-vg-autotest_73696/output//..
00:35:50.944  + rmdir ./zns-vg-autotest_73696/output/
00:35:50.944  + unpack_cov_files
00:35:50.944  + local info_files
00:35:50.944  + info_files=(*/cov_*.info.xz)
00:35:50.944  + xargs -P0 -r -n1 xz -d
00:35:50.944  + printf '%s\n' BlobFS-phy-autotest_23411/cov_base.info.xz centos7-vg-autotest_71090/cov_base.info.xz centos7-vg-autotest_71090/cov_test.info.xz centos7-vg-autotest_71090/cov_total.info.xz crypto-phy-autotest_23409/cov_base.info.xz crypto-phy-autotest_23409/cov_test.info.xz crypto-phy-autotest_23409/cov_total.info.xz iscsi-uring-vg-autotest_74496/cov_base.info.xz iscsi-vg-autotest_74631/cov_base.info.xz lvol-vg-autotest_74542/cov_base.info.xz nvme-cmb-pmr-vg-autotest_75716/cov_base.info.xz nvme-cmb-pmr-vg-autotest_75716/cov_test.info.xz nvme-cmb-pmr-vg-autotest_75716/cov_total.info.xz nvme-phy-autotest_69577/cov_base.info.xz nvme-phy-autotest_69577/cov_test.info.xz nvme-phy-autotest_69577/cov_total.info.xz nvme-vg-autotest_74594/cov_base.info.xz nvmf-phy-autotest_69651/cov_base.info.xz nvmf-phy-autotest_69651/cov_test.info.xz nvmf-phy-autotest_69651/cov_total.info.xz nvmf-tcp-phy-autotest_69540/cov_base.info.xz nvmf-tcp-phy-autotest_69540/cov_test.info.xz nvmf-tcp-phy-autotest_69540/cov_total.info.xz nvmf-tcp-uring-vg-autotest_74515/cov_base.info.xz nvmf-tcp-uring-vg-autotest_74515/cov_test.info.xz nvmf-tcp-uring-vg-autotest_74515/cov_total.info.xz nvmf-tcp-vg-autotest_74582/cov_base.info.xz nvmf-tcp-vg-autotest_74582/cov_test.info.xz nvmf-tcp-vg-autotest_74582/cov_total.info.xz rocky9-vg-autotest_13462/cov_base.info.xz ubuntu20-vg-autotest_71172/cov_base.info.xz ubuntu22-vg-autotest_35184/cov_base.info.xz vfio-user-phy-autotest_23417/cov_base.info.xz vhost-initiator-vg-autotest_74615/cov_base.info.xz vhost-initiator-vg-autotest_74615/cov_test.info.xz vhost-initiator-vg-autotest_74615/cov_total.info.xz vhost-phy-autotest_23432/cov_base.info.xz zns-vg-autotest_73696/cov_base.info.xz zns-vg-autotest_73696/cov_test.info.xz zns-vg-autotest_73696/cov_total.info.xz
00:35:51.202  + fix_downstream_job_paths
00:35:51.202  + sed -i -e 's#^SF:/.\+/spdk/#SF:/var/jenkins/workspace/autotest-per-patch_2/spdk/#g' centos7-vg-autotest_71090/cov_total.info crypto-phy-autotest_23409/cov_total.info nvme-cmb-pmr-vg-autotest_75716/cov_total.info nvme-phy-autotest_69577/cov_total.info nvmf-phy-autotest_69651/cov_total.info nvmf-tcp-phy-autotest_69540/cov_total.info nvmf-tcp-uring-vg-autotest_74515/cov_total.info nvmf-tcp-vg-autotest_74582/cov_total.info vhost-initiator-vg-autotest_74615/cov_total.info zns-vg-autotest_73696/cov_total.info
00:35:51.459  + ignore_ocf_headers
00:35:51.459  + sed -i 's#--show-details#--show-details --ignore-errors source#g' spdk/autorun_post.py
00:35:51.459  + create_errtrace_files_for_all_builds
00:35:51.459  + local build_log_list
00:35:51.459  + build_log_list=($(find "$WORKSPACE" -type f -name 'build.log'))
00:35:51.459  ++ find /var/jenkins/workspace/autotest-per-patch_2 -type f -name build.log
00:35:51.724  + for build in "${build_log_list[@]}"
00:35:51.724  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_74496/build.log
00:35:51.724  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.724  + build_log=/var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_74496/build.log
00:35:51.724  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_74496/build.log
00:35:51.724  ++ basename /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_74496
00:35:51.724  + job_name=iscsi-uring-vg-autotest_74496
00:35:51.724  + start_pattern='Backtrace start'
00:35:51.724  + end_pattern='Backtrace end'
00:35:51.724  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_74496/build.log ]]
00:35:51.724  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_74496/build.log
00:35:51.724  + backtraces='00:19:48.472  ========== Backtrace start: ==========
00:35:51.724  00:19:48.472  
00:35:51.724  00:19:48.472  in /home/vagrant/spdk_repo/spdk/test/iscsi_tgt/trace_record/trace_record.sh:96 -> main([])
00:35:51.724  00:19:48.472       ...
00:35:51.724  00:19:48.472     91  	
00:35:51.724  00:19:48.472     92  	trap '\''delete_tmp_files; iscsitestfini; exit 1'\'' SIGINT SIGTERM EXIT
00:35:51.724  00:19:48.472     93  	
00:35:51.724  00:19:48.472     94  	killprocess $iscsi_pid
00:35:51.724  00:19:48.472     95  	killprocess $record_pid
00:35:51.724  00:19:48.472  => 96  	$rootdir/build/bin/spdk_trace -f ${TRACE_RECORD_OUTPUT} > ${TRACE_TOOL_LOG}
00:35:51.724  00:19:48.472     97  	
00:35:51.724  00:19:48.472     98  	#verify trace record and trace tool
00:35:51.724  00:19:48.472     99  	#trace entries str in trace-record, like "Trace Size of lcore (0): 4136"
00:35:51.724  00:19:48.472     100 	record_num="$(grep "trace entries for lcore" ${TRACE_RECORD_NOTICE_LOG} | cut -d '\'' '\'' -f 2)"
00:35:51.724  00:19:48.472     101 	
00:35:51.724  00:19:48.472       ...
00:35:51.724  00:19:48.472  
00:35:51.724  00:19:48.472  ========== Backtrace end ==========
00:35:51.724  00:19:48.472  ========== Backtrace start: ==========
00:35:51.724  00:19:48.472  
00:35:51.724  00:19:48.472  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["iscsi_tgt_trace_record"],["/home/vagrant/spdk_repo/spdk/test/iscsi_tgt/trace_record/trace_record.sh"])
00:35:51.724  00:19:48.472       ...
00:35:51.724  00:19:48.472     1106		timing_enter $test_name
00:35:51.724  00:19:48.472     1107		echo "************************************"
00:35:51.724  00:19:48.472     1108		echo "START TEST $test_name"
00:35:51.724  00:19:48.472     1109		echo "************************************"
00:35:51.724  00:19:48.472     1110		xtrace_restore
00:35:51.724  00:19:48.472     1111		time "$@"
00:35:51.725  00:19:48.472     1112		xtrace_disable
00:35:51.725  00:19:48.472     1113		echo "************************************"
00:35:51.725  00:19:48.472     1114		echo "END TEST $test_name"
00:35:51.725  00:19:48.472     1115		echo "************************************"
00:35:51.725  00:19:48.472     1116		timing_exit $test_name
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472  in /home/vagrant/spdk_repo/spdk/test/iscsi_tgt/iscsi_tgt.sh:38 -> main([])
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472     33  	run_test "iscsi_tgt_rpc_config" $rootdir/test/iscsi_tgt/rpc_config/rpc_config.sh
00:35:51.725  00:19:48.472     34  	run_test "iscsi_tgt_iscsi_lvol" $rootdir/test/iscsi_tgt/lvol/iscsi_lvol.sh
00:35:51.725  00:19:48.472     35  	run_test "iscsi_tgt_fio" $rootdir/test/iscsi_tgt/fio/fio.sh
00:35:51.725  00:19:48.472     36  	run_test "iscsi_tgt_qos" $rootdir/test/iscsi_tgt/qos/qos.sh
00:35:51.725  00:19:48.472     37  	run_test "iscsi_tgt_ip_migration" $rootdir/test/iscsi_tgt/ip_migration/ip_migration.sh
00:35:51.725  00:19:48.472  => 38  	run_test "iscsi_tgt_trace_record" $rootdir/test/iscsi_tgt/trace_record/trace_record.sh
00:35:51.725  00:19:48.472     39  	run_test "iscsi_tgt_login_redirection" $rootdir/test/iscsi_tgt/login_redirection/login_redirection.sh
00:35:51.725  00:19:48.472     40  	run_test "iscsi_tgt_digests" $rootdir/test/iscsi_tgt/digests/digests.sh
00:35:51.725  00:19:48.472     41  	run_test "iscsi_tgt_fuzz" $rootdir/test/iscsi_tgt/fuzz/fuzz.sh
00:35:51.725  00:19:48.472     42  	run_test "iscsi_tgt_multiconnection" $rootdir/test/iscsi_tgt/multiconnection/multiconnection.sh
00:35:51.725  00:19:48.472     43  	
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472  
00:35:51.725  00:19:48.472  ========== Backtrace end ==========
00:35:51.725  00:19:48.472  ========== Backtrace start: ==========
00:35:51.725  00:19:48.472  
00:35:51.725  00:19:48.472  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["iscsi_tgt"],["/home/vagrant/spdk_repo/spdk/test/iscsi_tgt/iscsi_tgt.sh"])
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472     1106		timing_enter $test_name
00:35:51.725  00:19:48.472     1107		echo "************************************"
00:35:51.725  00:19:48.472     1108		echo "START TEST $test_name"
00:35:51.725  00:19:48.472     1109		echo "************************************"
00:35:51.725  00:19:48.472     1110		xtrace_restore
00:35:51.725  00:19:48.472     1111		time "$@"
00:35:51.725  00:19:48.472     1112		xtrace_disable
00:35:51.725  00:19:48.472     1113		echo "************************************"
00:35:51.725  00:19:48.472     1114		echo "END TEST $test_name"
00:35:51.725  00:19:48.472     1115		echo "************************************"
00:35:51.725  00:19:48.472     1116		timing_exit $test_name
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472  in /home/vagrant/spdk_repo/spdk/autotest.sh:261 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472     256 		fi
00:35:51.725  00:19:48.472     257 	
00:35:51.725  00:19:48.472     258 		timing_exit lib
00:35:51.725  00:19:48.472     259 	
00:35:51.725  00:19:48.472     260 		if [ $SPDK_TEST_ISCSI -eq 1 ]; then
00:35:51.725  00:19:48.472  => 261 			run_test "iscsi_tgt" $rootdir/test/iscsi_tgt/iscsi_tgt.sh
00:35:51.725  00:19:48.472     262 			run_test "spdkcli_iscsi" $rootdir/test/spdkcli/iscsi.sh
00:35:51.725  00:19:48.472     263 	
00:35:51.725  00:19:48.472     264 			# Run raid spdkcli test under iSCSI since blockdev tests run on systems that can'\''t run spdkcli yet
00:35:51.725  00:19:48.472     265 			run_test "spdkcli_raid" $rootdir/test/spdkcli/raid.sh
00:35:51.725  00:19:48.472     266 		fi
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472  
00:35:51.725  00:19:48.472  ========== Backtrace end =========='
00:35:51.725  + [[ -n 00:19:48.472  ========== Backtrace start: ==========
00:35:51.725  00:19:48.472  
00:35:51.725  00:19:48.472  in /home/vagrant/spdk_repo/spdk/test/iscsi_tgt/trace_record/trace_record.sh:96 -> main([])
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472     91  	
00:35:51.725  00:19:48.472     92  	trap 'delete_tmp_files; iscsitestfini; exit 1' SIGINT SIGTERM EXIT
00:35:51.725  00:19:48.472     93  	
00:35:51.725  00:19:48.472     94  	killprocess $iscsi_pid
00:35:51.725  00:19:48.472     95  	killprocess $record_pid
00:35:51.725  00:19:48.472  => 96  	$rootdir/build/bin/spdk_trace -f ${TRACE_RECORD_OUTPUT} > ${TRACE_TOOL_LOG}
00:35:51.725  00:19:48.472     97  	
00:35:51.725  00:19:48.472     98  	#verify trace record and trace tool
00:35:51.725  00:19:48.472     99  	#trace entries str in trace-record, like "Trace Size of lcore (0): 4136"
00:35:51.725  00:19:48.472     100 	record_num="$(grep "trace entries for lcore" ${TRACE_RECORD_NOTICE_LOG} | cut -d ' ' -f 2)"
00:35:51.725  00:19:48.472     101 	
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472  
00:35:51.725  00:19:48.472  ========== Backtrace end ==========
00:35:51.725  00:19:48.472  ========== Backtrace start: ==========
00:35:51.725  00:19:48.472  
00:35:51.725  00:19:48.472  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["iscsi_tgt_trace_record"],["/home/vagrant/spdk_repo/spdk/test/iscsi_tgt/trace_record/trace_record.sh"])
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472     1106		timing_enter $test_name
00:35:51.725  00:19:48.472     1107		echo "************************************"
00:35:51.725  00:19:48.472     1108		echo "START TEST $test_name"
00:35:51.725  00:19:48.472     1109		echo "************************************"
00:35:51.725  00:19:48.472     1110		xtrace_restore
00:35:51.725  00:19:48.472     1111		time "$@"
00:35:51.725  00:19:48.472     1112		xtrace_disable
00:35:51.725  00:19:48.472     1113		echo "************************************"
00:35:51.725  00:19:48.472     1114		echo "END TEST $test_name"
00:35:51.725  00:19:48.472     1115		echo "************************************"
00:35:51.725  00:19:48.472     1116		timing_exit $test_name
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472  in /home/vagrant/spdk_repo/spdk/test/iscsi_tgt/iscsi_tgt.sh:38 -> main([])
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472     33  	run_test "iscsi_tgt_rpc_config" $rootdir/test/iscsi_tgt/rpc_config/rpc_config.sh
00:35:51.725  00:19:48.472     34  	run_test "iscsi_tgt_iscsi_lvol" $rootdir/test/iscsi_tgt/lvol/iscsi_lvol.sh
00:35:51.725  00:19:48.472     35  	run_test "iscsi_tgt_fio" $rootdir/test/iscsi_tgt/fio/fio.sh
00:35:51.725  00:19:48.472     36  	run_test "iscsi_tgt_qos" $rootdir/test/iscsi_tgt/qos/qos.sh
00:35:51.725  00:19:48.472     37  	run_test "iscsi_tgt_ip_migration" $rootdir/test/iscsi_tgt/ip_migration/ip_migration.sh
00:35:51.725  00:19:48.472  => 38  	run_test "iscsi_tgt_trace_record" $rootdir/test/iscsi_tgt/trace_record/trace_record.sh
00:35:51.725  00:19:48.472     39  	run_test "iscsi_tgt_login_redirection" $rootdir/test/iscsi_tgt/login_redirection/login_redirection.sh
00:35:51.725  00:19:48.472     40  	run_test "iscsi_tgt_digests" $rootdir/test/iscsi_tgt/digests/digests.sh
00:35:51.725  00:19:48.472     41  	run_test "iscsi_tgt_fuzz" $rootdir/test/iscsi_tgt/fuzz/fuzz.sh
00:35:51.725  00:19:48.472     42  	run_test "iscsi_tgt_multiconnection" $rootdir/test/iscsi_tgt/multiconnection/multiconnection.sh
00:35:51.725  00:19:48.472     43  	
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472  
00:35:51.725  00:19:48.472  ========== Backtrace end ==========
00:35:51.725  00:19:48.472  ========== Backtrace start: ==========
00:35:51.725  00:19:48.472  
00:35:51.725  00:19:48.472  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["iscsi_tgt"],["/home/vagrant/spdk_repo/spdk/test/iscsi_tgt/iscsi_tgt.sh"])
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472     1106		timing_enter $test_name
00:35:51.725  00:19:48.472     1107		echo "************************************"
00:35:51.725  00:19:48.472     1108		echo "START TEST $test_name"
00:35:51.725  00:19:48.472     1109		echo "************************************"
00:35:51.725  00:19:48.472     1110		xtrace_restore
00:35:51.725  00:19:48.472     1111		time "$@"
00:35:51.725  00:19:48.472     1112		xtrace_disable
00:35:51.725  00:19:48.472     1113		echo "************************************"
00:35:51.725  00:19:48.472     1114		echo "END TEST $test_name"
00:35:51.725  00:19:48.472     1115		echo "************************************"
00:35:51.725  00:19:48.472     1116		timing_exit $test_name
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472  in /home/vagrant/spdk_repo/spdk/autotest.sh:261 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472     256 		fi
00:35:51.725  00:19:48.472     257 	
00:35:51.725  00:19:48.472     258 		timing_exit lib
00:35:51.725  00:19:48.472     259 	
00:35:51.725  00:19:48.472     260 		if [ $SPDK_TEST_ISCSI -eq 1 ]; then
00:35:51.725  00:19:48.472  => 261 			run_test "iscsi_tgt" $rootdir/test/iscsi_tgt/iscsi_tgt.sh
00:35:51.725  00:19:48.472     262 			run_test "spdkcli_iscsi" $rootdir/test/spdkcli/iscsi.sh
00:35:51.725  00:19:48.472     263 	
00:35:51.725  00:19:48.472     264 			# Run raid spdkcli test under iSCSI since blockdev tests run on systems that can't run spdkcli yet
00:35:51.725  00:19:48.472     265 			run_test "spdkcli_raid" $rootdir/test/spdkcli/raid.sh
00:35:51.725  00:19:48.472     266 		fi
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472  
00:35:51.725  00:19:48.472  ========== Backtrace end ========== ]]
00:35:51.725  + echo '00:19:48.472  ========== Backtrace start: ==========
00:35:51.725  00:19:48.472  
00:35:51.725  00:19:48.472  in /home/vagrant/spdk_repo/spdk/test/iscsi_tgt/trace_record/trace_record.sh:96 -> main([])
00:35:51.725  00:19:48.472       ...
00:35:51.725  00:19:48.472     91  	
00:35:51.726  00:19:48.472     92  	trap '\''delete_tmp_files; iscsitestfini; exit 1'\'' SIGINT SIGTERM EXIT
00:35:51.726  00:19:48.472     93  	
00:35:51.726  00:19:48.472     94  	killprocess $iscsi_pid
00:35:51.726  00:19:48.472     95  	killprocess $record_pid
00:35:51.726  00:19:48.472  => 96  	$rootdir/build/bin/spdk_trace -f ${TRACE_RECORD_OUTPUT} > ${TRACE_TOOL_LOG}
00:35:51.726  00:19:48.472     97  	
00:35:51.726  00:19:48.472     98  	#verify trace record and trace tool
00:35:51.726  00:19:48.472     99  	#trace entries str in trace-record, like "Trace Size of lcore (0): 4136"
00:35:51.726  00:19:48.472     100 	record_num="$(grep "trace entries for lcore" ${TRACE_RECORD_NOTICE_LOG} | cut -d '\'' '\'' -f 2)"
00:35:51.726  00:19:48.472     101 	
00:35:51.726  00:19:48.472       ...
00:35:51.726  00:19:48.472  
00:35:51.726  00:19:48.472  ========== Backtrace end ==========
00:35:51.726  00:19:48.472  ========== Backtrace start: ==========
00:35:51.726  00:19:48.472  
00:35:51.726  00:19:48.472  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["iscsi_tgt_trace_record"],["/home/vagrant/spdk_repo/spdk/test/iscsi_tgt/trace_record/trace_record.sh"])
00:35:51.726  00:19:48.472       ...
00:35:51.726  00:19:48.472     1106		timing_enter $test_name
00:35:51.726  00:19:48.472     1107		echo "************************************"
00:35:51.726  00:19:48.472     1108		echo "START TEST $test_name"
00:35:51.726  00:19:48.472     1109		echo "************************************"
00:35:51.726  00:19:48.472     1110		xtrace_restore
00:35:51.726  00:19:48.472     1111		time "$@"
00:35:51.726  00:19:48.472     1112		xtrace_disable
00:35:51.726  00:19:48.472     1113		echo "************************************"
00:35:51.726  00:19:48.472     1114		echo "END TEST $test_name"
00:35:51.726  00:19:48.472     1115		echo "************************************"
00:35:51.726  00:19:48.472     1116		timing_exit $test_name
00:35:51.726  00:19:48.472       ...
00:35:51.726  00:19:48.472  in /home/vagrant/spdk_repo/spdk/test/iscsi_tgt/iscsi_tgt.sh:38 -> main([])
00:35:51.726  00:19:48.472       ...
00:35:51.726  00:19:48.472     33  	run_test "iscsi_tgt_rpc_config" $rootdir/test/iscsi_tgt/rpc_config/rpc_config.sh
00:35:51.726  00:19:48.472     34  	run_test "iscsi_tgt_iscsi_lvol" $rootdir/test/iscsi_tgt/lvol/iscsi_lvol.sh
00:35:51.726  00:19:48.472     35  	run_test "iscsi_tgt_fio" $rootdir/test/iscsi_tgt/fio/fio.sh
00:35:51.726  00:19:48.472     36  	run_test "iscsi_tgt_qos" $rootdir/test/iscsi_tgt/qos/qos.sh
00:35:51.726  00:19:48.472     37  	run_test "iscsi_tgt_ip_migration" $rootdir/test/iscsi_tgt/ip_migration/ip_migration.sh
00:35:51.726  00:19:48.472  => 38  	run_test "iscsi_tgt_trace_record" $rootdir/test/iscsi_tgt/trace_record/trace_record.sh
00:35:51.726  00:19:48.472     39  	run_test "iscsi_tgt_login_redirection" $rootdir/test/iscsi_tgt/login_redirection/login_redirection.sh
00:35:51.726  00:19:48.472     40  	run_test "iscsi_tgt_digests" $rootdir/test/iscsi_tgt/digests/digests.sh
00:35:51.726  00:19:48.472     41  	run_test "iscsi_tgt_fuzz" $rootdir/test/iscsi_tgt/fuzz/fuzz.sh
00:35:51.726  00:19:48.472     42  	run_test "iscsi_tgt_multiconnection" $rootdir/test/iscsi_tgt/multiconnection/multiconnection.sh
00:35:51.726  00:19:48.472     43  	
00:35:51.726  00:19:48.472       ...
00:35:51.726  00:19:48.472  
00:35:51.726  00:19:48.472  ========== Backtrace end ==========
00:35:51.726  00:19:48.472  ========== Backtrace start: ==========
00:35:51.726  00:19:48.472  
00:35:51.726  00:19:48.472  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["iscsi_tgt"],["/home/vagrant/spdk_repo/spdk/test/iscsi_tgt/iscsi_tgt.sh"])
00:35:51.726  00:19:48.472       ...
00:35:51.726  00:19:48.472     1106		timing_enter $test_name
00:35:51.726  00:19:48.472     1107		echo "************************************"
00:35:51.726  00:19:48.472     1108		echo "START TEST $test_name"
00:35:51.726  00:19:48.472     1109		echo "************************************"
00:35:51.726  00:19:48.472     1110		xtrace_restore
00:35:51.726  00:19:48.472     1111		time "$@"
00:35:51.726  00:19:48.472     1112		xtrace_disable
00:35:51.726  00:19:48.472     1113		echo "************************************"
00:35:51.726  00:19:48.472     1114		echo "END TEST $test_name"
00:35:51.726  00:19:48.472     1115		echo "************************************"
00:35:51.726  00:19:48.472     1116		timing_exit $test_name
00:35:51.726  00:19:48.472       ...
00:35:51.726  00:19:48.472  in /home/vagrant/spdk_repo/spdk/autotest.sh:261 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.726  00:19:48.472       ...
00:35:51.726  00:19:48.472     256 		fi
00:35:51.726  00:19:48.472     257 	
00:35:51.726  00:19:48.472     258 		timing_exit lib
00:35:51.726  00:19:48.472     259 	
00:35:51.726  00:19:48.472     260 		if [ $SPDK_TEST_ISCSI -eq 1 ]; then
00:35:51.726  00:19:48.472  => 261 			run_test "iscsi_tgt" $rootdir/test/iscsi_tgt/iscsi_tgt.sh
00:35:51.726  00:19:48.472     262 			run_test "spdkcli_iscsi" $rootdir/test/spdkcli/iscsi.sh
00:35:51.726  00:19:48.472     263 	
00:35:51.726  00:19:48.472     264 			# Run raid spdkcli test under iSCSI since blockdev tests run on systems that can'\''t run spdkcli yet
00:35:51.726  00:19:48.472     265 			run_test "spdkcli_raid" $rootdir/test/spdkcli/raid.sh
00:35:51.726  00:19:48.472     266 		fi
00:35:51.726  00:19:48.472       ...
00:35:51.726  00:19:48.472  
00:35:51.726  00:19:48.472  ========== Backtrace end =========='
00:35:51.726  + for build in "${build_log_list[@]}"
00:35:51.726  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_17621/build.log
00:35:51.726  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.726  + build_log=/var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_17621/build.log
00:35:51.726  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_17621/build.log
00:35:51.726  ++ basename /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_17621
00:35:51.726  + job_name=unittest-main-docker-autotest_17621
00:35:51.726  + start_pattern='Backtrace start'
00:35:51.726  + end_pattern='Backtrace end'
00:35:51.726  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_17621/build.log ]]
00:35:51.726  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_17621/build.log
00:35:51.726  + backtraces=
00:35:51.726  + [[ -n '' ]]
00:35:51.726  + for build in "${build_log_list[@]}"
00:35:51.726  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_19665/build.log
00:35:51.726  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.726  + build_log=/var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_19665/build.log
00:35:51.726  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_19665/build.log
00:35:51.726  ++ basename /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_19665
00:35:51.726  + job_name=doc-docker-autotest_19665
00:35:51.726  + start_pattern='Backtrace start'
00:35:51.726  + end_pattern='Backtrace end'
00:35:51.726  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_19665/build.log ]]
00:35:51.726  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_19665/build.log
00:35:51.726  + backtraces=
00:35:51.726  + [[ -n '' ]]
00:35:51.726  + for build in "${build_log_list[@]}"
00:35:51.726  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_69577/build.log
00:35:51.726  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.726  + build_log=/var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_69577/build.log
00:35:51.726  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_69577/build.log
00:35:51.726  ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_69577
00:35:51.726  + job_name=nvme-phy-autotest_69577
00:35:51.726  + start_pattern='Backtrace start'
00:35:51.726  + end_pattern='Backtrace end'
00:35:51.726  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_69577/build.log ]]
00:35:51.726  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_69577/build.log
00:35:51.726  + backtraces=
00:35:51.726  + [[ -n '' ]]
00:35:51.726  + for build in "${build_log_list[@]}"
00:35:51.726  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_17622/build.log
00:35:51.726  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.726  + build_log=/var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_17622/build.log
00:35:51.726  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_17622/build.log
00:35:51.726  ++ basename /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_17622
00:35:51.726  + job_name=packaging-docker-autotest_17622
00:35:51.726  + start_pattern='Backtrace start'
00:35:51.726  + end_pattern='Backtrace end'
00:35:51.726  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_17622/build.log ]]
00:35:51.726  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_17622/build.log
00:35:51.726  + backtraces=
00:35:51.726  + [[ -n '' ]]
00:35:51.726  + for build in "${build_log_list[@]}"
00:35:51.726  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_71172/build.log
00:35:51.726  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.726  + build_log=/var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_71172/build.log
00:35:51.726  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_71172/build.log
00:35:51.726  ++ basename /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_71172
00:35:51.726  + job_name=ubuntu20-vg-autotest_71172
00:35:51.726  + start_pattern='Backtrace start'
00:35:51.726  + end_pattern='Backtrace end'
00:35:51.727  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_71172/build.log ]]
00:35:51.727  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_71172/build.log
00:35:51.727  + backtraces='00:18:00.833  ========== Backtrace start: ==========
00:35:51.727  00:18:00.833  
00:35:51.727  00:18:00.833  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:960 -> killprocess(["111694"])
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833     955 				kill $1
00:35:51.727  00:18:00.833     956 			fi
00:35:51.727  00:18:00.833     957 	
00:35:51.727  00:18:00.833     958 			# wait for the process regardless if its the dummy sudo one
00:35:51.727  00:18:00.833     959 			# or the actual app - it should terminate anyway
00:35:51.727  00:18:00.833  => 960 			wait $1
00:35:51.727  00:18:00.833     961 		else
00:35:51.727  00:18:00.833     962 			# the process is not there anymore
00:35:51.727  00:18:00.833     963 			echo "Process with pid $1 is not found"
00:35:51.727  00:18:00.833     964 		fi
00:35:51.727  00:18:00.833     965 	}
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:51.727  00:18:00.833     35  			sleep 3
00:35:51.727  00:18:00.833     36  		done
00:35:51.727  00:18:00.833     37  	
00:35:51.727  00:18:00.833     38  		waitforlisten $repeat_pid $rpc_server
00:35:51.727  00:18:00.833  => 39  		killprocess $repeat_pid
00:35:51.727  00:18:00.833     40  		trap - SIGINT SIGTERM EXIT
00:35:51.727  00:18:00.833     41  	
00:35:51.727  00:18:00.833     42  		return 0
00:35:51.727  00:18:00.833     43  	}
00:35:51.727  00:18:00.833     44  	
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833     1106		timing_enter $test_name
00:35:51.727  00:18:00.833     1107		echo "************************************"
00:35:51.727  00:18:00.833     1108		echo "START TEST $test_name"
00:35:51.727  00:18:00.833     1109		echo "************************************"
00:35:51.727  00:18:00.833     1110		xtrace_restore
00:35:51.727  00:18:00.833     1111		time "$@"
00:35:51.727  00:18:00.833     1112		xtrace_disable
00:35:51.727  00:18:00.833     1113		echo "************************************"
00:35:51.727  00:18:00.833     1114		echo "END TEST $test_name"
00:35:51.727  00:18:00.833     1115		echo "************************************"
00:35:51.727  00:18:00.833     1116		timing_exit $test_name
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833  Process with pid 111694 is not found
00:35:51.727  00:18:00.833  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:52 -> main([])
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:51.727  00:18:00.833     48  	
00:35:51.727  00:18:00.833     49  	if [ $(uname -s) = Linux ]; then
00:35:51.727  00:18:00.833     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:51.727  00:18:00.833     51  		if modprobe -n nbd; then
00:35:51.727  00:18:00.833  => 52  			run_test "app_repeat" app_repeat_test
00:35:51.727  00:18:00.833     53  		fi
00:35:51.727  00:18:00.833     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:51.727  00:18:00.833     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:51.727  00:18:00.833     56  		fi
00:35:51.727  00:18:00.833     57  	fi
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833  
00:35:51.727  00:18:00.833  ========== Backtrace end ==========
00:35:51.727  00:18:00.833  ========== Backtrace start: ==========
00:35:51.727  00:18:00.833  
00:35:51.727  00:18:00.833  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/home/vagrant/spdk_repo/spdk/test/event/event.sh"])
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833     1106		timing_enter $test_name
00:35:51.727  00:18:00.833     1107		echo "************************************"
00:35:51.727  00:18:00.833     1108		echo "START TEST $test_name"
00:35:51.727  00:18:00.833     1109		echo "************************************"
00:35:51.727  00:18:00.833     1110		xtrace_restore
00:35:51.727  00:18:00.833     1111		time "$@"
00:35:51.727  00:18:00.833     1112		xtrace_disable
00:35:51.727  00:18:00.833     1113		echo "************************************"
00:35:51.727  00:18:00.833     1114		echo "END TEST $test_name"
00:35:51.727  00:18:00.833     1115		echo "************************************"
00:35:51.727  00:18:00.833     1116		timing_exit $test_name
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833  in /home/vagrant/spdk_repo/spdk/autotest.sh:177 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:51.727  00:18:00.833     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:51.727  00:18:00.833     174 		fi
00:35:51.727  00:18:00.833     175 	
00:35:51.727  00:18:00.833     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:51.727  00:18:00.833  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:51.727  00:18:00.833     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:51.727  00:18:00.833     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:51.727  00:18:00.833     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:51.727  00:18:00.833     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:51.727  00:18:00.833     182 		run_test "version" $rootdir/test/app/version.sh
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833  
00:35:51.727  00:18:00.833  ========== Backtrace end =========='
00:35:51.727  + [[ -n 00:18:00.833  ========== Backtrace start: ==========
00:35:51.727  00:18:00.833  
00:35:51.727  00:18:00.833  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:960 -> killprocess(["111694"])
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833     955 				kill $1
00:35:51.727  00:18:00.833     956 			fi
00:35:51.727  00:18:00.833     957 	
00:35:51.727  00:18:00.833     958 			# wait for the process regardless if its the dummy sudo one
00:35:51.727  00:18:00.833     959 			# or the actual app - it should terminate anyway
00:35:51.727  00:18:00.833  => 960 			wait $1
00:35:51.727  00:18:00.833     961 		else
00:35:51.727  00:18:00.833     962 			# the process is not there anymore
00:35:51.727  00:18:00.833     963 			echo "Process with pid $1 is not found"
00:35:51.727  00:18:00.833     964 		fi
00:35:51.727  00:18:00.833     965 	}
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:51.727  00:18:00.833     35  			sleep 3
00:35:51.727  00:18:00.833     36  		done
00:35:51.727  00:18:00.833     37  	
00:35:51.727  00:18:00.833     38  		waitforlisten $repeat_pid $rpc_server
00:35:51.727  00:18:00.833  => 39  		killprocess $repeat_pid
00:35:51.727  00:18:00.833     40  		trap - SIGINT SIGTERM EXIT
00:35:51.727  00:18:00.833     41  	
00:35:51.727  00:18:00.833     42  		return 0
00:35:51.727  00:18:00.833     43  	}
00:35:51.727  00:18:00.833     44  	
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833     1106		timing_enter $test_name
00:35:51.727  00:18:00.833     1107		echo "************************************"
00:35:51.727  00:18:00.833     1108		echo "START TEST $test_name"
00:35:51.727  00:18:00.833     1109		echo "************************************"
00:35:51.727  00:18:00.833     1110		xtrace_restore
00:35:51.727  00:18:00.833     1111		time "$@"
00:35:51.727  00:18:00.833     1112		xtrace_disable
00:35:51.727  00:18:00.833     1113		echo "************************************"
00:35:51.727  00:18:00.833     1114		echo "END TEST $test_name"
00:35:51.727  00:18:00.833     1115		echo "************************************"
00:35:51.727  00:18:00.833     1116		timing_exit $test_name
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833  Process with pid 111694 is not found
00:35:51.727  00:18:00.833  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:52 -> main([])
00:35:51.727  00:18:00.833       ...
00:35:51.727  00:18:00.833     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:51.727  00:18:00.833     48  	
00:35:51.727  00:18:00.833     49  	if [ $(uname -s) = Linux ]; then
00:35:51.727  00:18:00.833     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:51.727  00:18:00.833     51  		if modprobe -n nbd; then
00:35:51.727  00:18:00.833  => 52  			run_test "app_repeat" app_repeat_test
00:35:51.727  00:18:00.833     53  		fi
00:35:51.727  00:18:00.833     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:51.727  00:18:00.833     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:51.728  00:18:00.833     56  		fi
00:35:51.728  00:18:00.833     57  	fi
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833  
00:35:51.728  00:18:00.833  ========== Backtrace end ==========
00:35:51.728  00:18:00.833  ========== Backtrace start: ==========
00:35:51.728  00:18:00.833  
00:35:51.728  00:18:00.833  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/home/vagrant/spdk_repo/spdk/test/event/event.sh"])
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833     1106		timing_enter $test_name
00:35:51.728  00:18:00.833     1107		echo "************************************"
00:35:51.728  00:18:00.833     1108		echo "START TEST $test_name"
00:35:51.728  00:18:00.833     1109		echo "************************************"
00:35:51.728  00:18:00.833     1110		xtrace_restore
00:35:51.728  00:18:00.833     1111		time "$@"
00:35:51.728  00:18:00.833     1112		xtrace_disable
00:35:51.728  00:18:00.833     1113		echo "************************************"
00:35:51.728  00:18:00.833     1114		echo "END TEST $test_name"
00:35:51.728  00:18:00.833     1115		echo "************************************"
00:35:51.728  00:18:00.833     1116		timing_exit $test_name
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833  in /home/vagrant/spdk_repo/spdk/autotest.sh:177 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:51.728  00:18:00.833     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:51.728  00:18:00.833     174 		fi
00:35:51.728  00:18:00.833     175 	
00:35:51.728  00:18:00.833     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:51.728  00:18:00.833  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:51.728  00:18:00.833     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:51.728  00:18:00.833     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:51.728  00:18:00.833     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:51.728  00:18:00.833     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:51.728  00:18:00.833     182 		run_test "version" $rootdir/test/app/version.sh
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833  
00:35:51.728  00:18:00.833  ========== Backtrace end ========== ]]
00:35:51.728  + echo '00:18:00.833  ========== Backtrace start: ==========
00:35:51.728  00:18:00.833  
00:35:51.728  00:18:00.833  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:960 -> killprocess(["111694"])
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833     955 				kill $1
00:35:51.728  00:18:00.833     956 			fi
00:35:51.728  00:18:00.833     957 	
00:35:51.728  00:18:00.833     958 			# wait for the process regardless if its the dummy sudo one
00:35:51.728  00:18:00.833     959 			# or the actual app - it should terminate anyway
00:35:51.728  00:18:00.833  => 960 			wait $1
00:35:51.728  00:18:00.833     961 		else
00:35:51.728  00:18:00.833     962 			# the process is not there anymore
00:35:51.728  00:18:00.833     963 			echo "Process with pid $1 is not found"
00:35:51.728  00:18:00.833     964 		fi
00:35:51.728  00:18:00.833     965 	}
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:51.728  00:18:00.833     35  			sleep 3
00:35:51.728  00:18:00.833     36  		done
00:35:51.728  00:18:00.833     37  	
00:35:51.728  00:18:00.833     38  		waitforlisten $repeat_pid $rpc_server
00:35:51.728  00:18:00.833  => 39  		killprocess $repeat_pid
00:35:51.728  00:18:00.833     40  		trap - SIGINT SIGTERM EXIT
00:35:51.728  00:18:00.833     41  	
00:35:51.728  00:18:00.833     42  		return 0
00:35:51.728  00:18:00.833     43  	}
00:35:51.728  00:18:00.833     44  	
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833     1106		timing_enter $test_name
00:35:51.728  00:18:00.833     1107		echo "************************************"
00:35:51.728  00:18:00.833     1108		echo "START TEST $test_name"
00:35:51.728  00:18:00.833     1109		echo "************************************"
00:35:51.728  00:18:00.833     1110		xtrace_restore
00:35:51.728  00:18:00.833     1111		time "$@"
00:35:51.728  00:18:00.833     1112		xtrace_disable
00:35:51.728  00:18:00.833     1113		echo "************************************"
00:35:51.728  00:18:00.833     1114		echo "END TEST $test_name"
00:35:51.728  00:18:00.833     1115		echo "************************************"
00:35:51.728  00:18:00.833     1116		timing_exit $test_name
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833  Process with pid 111694 is not found
00:35:51.728  00:18:00.833  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:52 -> main([])
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:51.728  00:18:00.833     48  	
00:35:51.728  00:18:00.833     49  	if [ $(uname -s) = Linux ]; then
00:35:51.728  00:18:00.833     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:51.728  00:18:00.833     51  		if modprobe -n nbd; then
00:35:51.728  00:18:00.833  => 52  			run_test "app_repeat" app_repeat_test
00:35:51.728  00:18:00.833     53  		fi
00:35:51.728  00:18:00.833     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:51.728  00:18:00.833     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:51.728  00:18:00.833     56  		fi
00:35:51.728  00:18:00.833     57  	fi
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833  
00:35:51.728  00:18:00.833  ========== Backtrace end ==========
00:35:51.728  00:18:00.833  ========== Backtrace start: ==========
00:35:51.728  00:18:00.833  
00:35:51.728  00:18:00.833  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/home/vagrant/spdk_repo/spdk/test/event/event.sh"])
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833     1106		timing_enter $test_name
00:35:51.728  00:18:00.833     1107		echo "************************************"
00:35:51.728  00:18:00.833     1108		echo "START TEST $test_name"
00:35:51.728  00:18:00.833     1109		echo "************************************"
00:35:51.728  00:18:00.833     1110		xtrace_restore
00:35:51.728  00:18:00.833     1111		time "$@"
00:35:51.728  00:18:00.833     1112		xtrace_disable
00:35:51.728  00:18:00.833     1113		echo "************************************"
00:35:51.728  00:18:00.833     1114		echo "END TEST $test_name"
00:35:51.728  00:18:00.833     1115		echo "************************************"
00:35:51.728  00:18:00.833     1116		timing_exit $test_name
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833  in /home/vagrant/spdk_repo/spdk/autotest.sh:177 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:51.728  00:18:00.833     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:51.728  00:18:00.833     174 		fi
00:35:51.728  00:18:00.833     175 	
00:35:51.728  00:18:00.833     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:51.728  00:18:00.833  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:51.728  00:18:00.833     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:51.728  00:18:00.833     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:51.728  00:18:00.833     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:51.728  00:18:00.833     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:51.728  00:18:00.833     182 		run_test "version" $rootdir/test/app/version.sh
00:35:51.728  00:18:00.833       ...
00:35:51.728  00:18:00.833  
00:35:51.728  00:18:00.833  ========== Backtrace end =========='
00:35:51.728  + for build in "${build_log_list[@]}"
00:35:51.728  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_23405/build.log
00:35:51.728  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.728  + build_log=/var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_23405/build.log
00:35:51.728  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_23405/build.log
00:35:51.728  ++ basename /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_23405
00:35:51.728  + job_name=short-fuzz-phy-autotest_23405
00:35:51.728  + start_pattern='Backtrace start'
00:35:51.728  + end_pattern='Backtrace end'
00:35:51.728  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_23405/build.log ]]
00:35:51.728  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_23405/build.log
00:35:51.728  + backtraces='00:02:08.983  ========== Backtrace start: ==========
00:35:51.728  00:02:08.983  
00:35:51.728  00:02:08.983  in /var/jenkins/workspace/short-fuzz-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["make"],["make"],["-j72"])
00:35:51.728  00:02:08.983       ...
00:35:51.728  00:02:08.983     1106		timing_enter $test_name
00:35:51.728  00:02:08.983     1107		echo "************************************"
00:35:51.728  00:02:08.983     1108		echo "START TEST $test_name"
00:35:51.728  00:02:08.983     1109		echo "************************************"
00:35:51.729  00:02:08.983     1110		xtrace_restore
00:35:51.729  00:02:08.983     1111		time "$@"
00:35:51.729  00:02:08.983     1112		xtrace_disable
00:35:51.729  00:02:08.983     1113		echo "************************************"
00:35:51.729  00:02:08.983     1114		echo "END TEST $test_name"
00:35:51.729  00:02:08.983     1115		echo "************************************"
00:35:51.729  00:02:08.983     1116		timing_exit $test_name
00:35:51.729  00:02:08.983       ...
00:35:51.729  00:02:08.983  in /var/jenkins/workspace/short-fuzz-phy-autotest/spdk/autobuild.sh:69 -> main(["/var/jenkins/workspace/short-fuzz-phy-autotest/autorun-spdk.conf"])
00:35:51.729  00:02:08.983       ...
00:35:51.729  00:02:08.983     64  			$rootdir/configure $config_params
00:35:51.729  00:02:08.983     65  		else
00:35:51.729  00:02:08.983     66  			# if we aren'\''t testing the unittests, build with shared objects.
00:35:51.729  00:02:08.984     67  			$rootdir/configure $config_params --with-shared
00:35:51.729  00:02:08.984     68  		fi
00:35:51.729  00:02:08.984  => 69  		run_test "make" $MAKE $MAKEFLAGS
00:35:51.729  00:02:08.984     70  	fi
00:35:51.729  00:02:08.984       ...
00:35:51.729  00:02:08.984  
00:35:51.729  00:02:08.984  ========== Backtrace end =========='
00:35:51.729  + [[ -n 00:02:08.983  ========== Backtrace start: ==========
00:35:51.729  00:02:08.983  
00:35:51.729  00:02:08.983  in /var/jenkins/workspace/short-fuzz-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["make"],["make"],["-j72"])
00:35:51.729  00:02:08.983       ...
00:35:51.729  00:02:08.983     1106		timing_enter $test_name
00:35:51.729  00:02:08.983     1107		echo "************************************"
00:35:51.729  00:02:08.983     1108		echo "START TEST $test_name"
00:35:51.729  00:02:08.983     1109		echo "************************************"
00:35:51.729  00:02:08.983     1110		xtrace_restore
00:35:51.729  00:02:08.983     1111		time "$@"
00:35:51.729  00:02:08.983     1112		xtrace_disable
00:35:51.729  00:02:08.983     1113		echo "************************************"
00:35:51.729  00:02:08.983     1114		echo "END TEST $test_name"
00:35:51.729  00:02:08.983     1115		echo "************************************"
00:35:51.729  00:02:08.983     1116		timing_exit $test_name
00:35:51.729  00:02:08.983       ...
00:35:51.729  00:02:08.983  in /var/jenkins/workspace/short-fuzz-phy-autotest/spdk/autobuild.sh:69 -> main(["/var/jenkins/workspace/short-fuzz-phy-autotest/autorun-spdk.conf"])
00:35:51.729  00:02:08.983       ...
00:35:51.729  00:02:08.983     64  			$rootdir/configure $config_params
00:35:51.729  00:02:08.983     65  		else
00:35:51.729  00:02:08.983     66  			# if we aren't testing the unittests, build with shared objects.
00:35:51.729  00:02:08.984     67  			$rootdir/configure $config_params --with-shared
00:35:51.729  00:02:08.984     68  		fi
00:35:51.729  00:02:08.984  => 69  		run_test "make" $MAKE $MAKEFLAGS
00:35:51.729  00:02:08.984     70  	fi
00:35:51.729  00:02:08.984       ...
00:35:51.729  00:02:08.984  
00:35:51.729  00:02:08.984  ========== Backtrace end ========== ]]
00:35:51.729  + echo '00:02:08.983  ========== Backtrace start: ==========
00:35:51.729  00:02:08.983  
00:35:51.729  00:02:08.983  in /var/jenkins/workspace/short-fuzz-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["make"],["make"],["-j72"])
00:35:51.729  00:02:08.983       ...
00:35:51.729  00:02:08.983     1106		timing_enter $test_name
00:35:51.729  00:02:08.983     1107		echo "************************************"
00:35:51.729  00:02:08.983     1108		echo "START TEST $test_name"
00:35:51.729  00:02:08.983     1109		echo "************************************"
00:35:51.729  00:02:08.983     1110		xtrace_restore
00:35:51.729  00:02:08.983     1111		time "$@"
00:35:51.729  00:02:08.983     1112		xtrace_disable
00:35:51.729  00:02:08.983     1113		echo "************************************"
00:35:51.729  00:02:08.983     1114		echo "END TEST $test_name"
00:35:51.729  00:02:08.983     1115		echo "************************************"
00:35:51.729  00:02:08.983     1116		timing_exit $test_name
00:35:51.729  00:02:08.983       ...
00:35:51.729  00:02:08.983  in /var/jenkins/workspace/short-fuzz-phy-autotest/spdk/autobuild.sh:69 -> main(["/var/jenkins/workspace/short-fuzz-phy-autotest/autorun-spdk.conf"])
00:35:51.729  00:02:08.983       ...
00:35:51.729  00:02:08.983     64  			$rootdir/configure $config_params
00:35:51.729  00:02:08.983     65  		else
00:35:51.729  00:02:08.983     66  			# if we aren'\''t testing the unittests, build with shared objects.
00:35:51.729  00:02:08.984     67  			$rootdir/configure $config_params --with-shared
00:35:51.729  00:02:08.984     68  		fi
00:35:51.729  00:02:08.984  => 69  		run_test "make" $MAKE $MAKEFLAGS
00:35:51.729  00:02:08.984     70  	fi
00:35:51.729  00:02:08.984       ...
00:35:51.729  00:02:08.984  
00:35:51.729  00:02:08.984  ========== Backtrace end =========='
00:35:51.729  + for build in "${build_log_list[@]}"
00:35:51.729  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_17624/build.log
00:35:51.729  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.729  + build_log=/var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_17624/build.log
00:35:51.729  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_17624/build.log
00:35:51.729  ++ basename /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_17624
00:35:51.729  + job_name=unittest-valgrind-docker-autotest_17624
00:35:51.729  + start_pattern='Backtrace start'
00:35:51.729  + end_pattern='Backtrace end'
00:35:51.729  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_17624/build.log ]]
00:35:51.729  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_17624/build.log
00:35:51.729  + backtraces=
00:35:51.729  + [[ -n '' ]]
00:35:51.729  + for build in "${build_log_list[@]}"
00:35:51.729  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_23417/build.log
00:35:51.729  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.729  + build_log=/var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_23417/build.log
00:35:51.729  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_23417/build.log
00:35:51.729  ++ basename /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_23417
00:35:51.729  + job_name=vfio-user-phy-autotest_23417
00:35:51.729  + start_pattern='Backtrace start'
00:35:51.729  + end_pattern='Backtrace end'
00:35:51.729  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_23417/build.log ]]
00:35:51.729  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_23417/build.log
00:35:51.729  + backtraces='00:06:10.998  ========== Backtrace start: ==========
00:35:51.729  00:06:10.998  
00:35:51.729  00:06:10.998  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/common/autotest_common.sh:960 -> killprocess(["115657"])
00:35:51.729  00:06:10.998       ...
00:35:51.729  00:06:10.998     955 				kill $1
00:35:51.729  00:06:10.998     956 			fi
00:35:51.729  00:06:10.998     957 	
00:35:51.729  00:06:10.998     958 			# wait for the process regardless if its the dummy sudo one
00:35:51.729  00:06:10.998     959 			# or the actual app - it should terminate anyway
00:35:51.729  00:06:10.998  => 960 			wait $1
00:35:51.729  00:06:10.998     961 		else
00:35:51.729  00:06:10.998     962 			# the process is not there anymore
00:35:51.729  00:06:10.998     963 			echo "Process with pid $1 is not found"
00:35:51.729  00:06:10.998     964 		fi
00:35:51.729  00:06:10.998     965 	}
00:35:51.729  00:06:10.998       ...
00:35:51.729  00:06:10.998  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:51.729  00:06:10.998       ...
00:35:51.729  00:06:10.998     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:51.729  00:06:10.998     35  			sleep 3
00:35:51.729  00:06:10.998     36  		done
00:35:51.729  00:06:10.998     37  	
00:35:51.729  00:06:10.998     38  		waitforlisten $repeat_pid $rpc_server
00:35:51.729  00:06:10.998  => 39  		killprocess $repeat_pid
00:35:51.729  00:06:10.998     40  		trap - SIGINT SIGTERM EXIT
00:35:51.729  00:06:10.998     41  	
00:35:51.729  00:06:10.998     42  		return 0
00:35:51.729  00:06:10.998     43  	}
00:35:51.729  00:06:10.998     44  	
00:35:51.729  00:06:10.998       ...
00:35:51.729  00:06:10.998  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:51.729  00:06:10.998       ...
00:35:51.729  00:06:10.998     1106		timing_enter $test_name
00:35:51.729  00:06:10.998     1107		echo "************************************"
00:35:51.729  00:06:10.998     1108		echo "START TEST $test_name"
00:35:51.729  00:06:10.998     1109		echo "************************************"
00:35:51.729  00:06:10.998     1110		xtrace_restore
00:35:51.729  00:06:10.998     1111		time "$@"
00:35:51.729  00:06:10.998     1112		xtrace_disable
00:35:51.729  00:06:10.998     1113		echo "************************************"
00:35:51.729  00:06:10.998     1114		echo "END TEST $test_name"
00:35:51.729  00:06:10.998     1115		echo "************************************"
00:35:51.729  00:06:10.998     1116		timing_exit $test_name
00:35:51.729  00:06:10.998       ...
00:35:51.729  00:06:10.998  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/event/event.sh:52 -> main([])
00:35:51.729  00:06:10.998       ...
00:35:51.730  00:06:10.998     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:51.730  00:06:10.998     48  	
00:35:51.730  00:06:10.998     49  	if [ $(uname -s) = Linux ]; then
00:35:51.730  00:06:10.998     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:51.730  00:06:10.998     51  		if modprobe -n nbd; then
00:35:51.730  00:06:10.998  => 52  			run_test "app_repeat" app_repeat_test
00:35:51.730  00:06:10.998     53  		fi
00:35:51.730  00:06:10.998     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:51.730  00:06:10.998     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:51.730  00:06:10.998     56  		fi
00:35:51.730  00:06:10.998     57  	fi
00:35:51.730  00:06:10.998       ...
00:35:51.730  00:06:10.998  
00:35:51.730  00:06:10.998  ========== Backtrace end ==========
00:35:51.730  00:06:10.999  ========== Backtrace start: ==========
00:35:51.730  00:06:10.999  
00:35:51.730  00:06:10.999  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/event/event.sh"])
00:35:51.730  00:06:10.999       ...
00:35:51.730  00:06:10.999     1106		timing_enter $test_name
00:35:51.730  00:06:10.999     1107		echo "************************************"
00:35:51.730  00:06:10.999     1108		echo "START TEST $test_name"
00:35:51.730  00:06:10.999     1109		echo "************************************"
00:35:51.730  00:06:10.999     1110		xtrace_restore
00:35:51.730  00:06:10.999     1111		time "$@"
00:35:51.730  00:06:10.999     1112		xtrace_disable
00:35:51.730  00:06:10.999     1113		echo "************************************"
00:35:51.730  00:06:10.999     1114		echo "END TEST $test_name"
00:35:51.730  00:06:10.999     1115		echo "************************************"
00:35:51.730  00:06:10.999     1116		timing_exit $test_name
00:35:51.730  00:06:10.999       ...
00:35:51.730  00:06:10.999  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/autotest.sh:177 -> main(["/var/jenkins/workspace/vfio-user-phy-autotest/autorun-spdk.conf"])
00:35:51.730  00:06:10.999       ...
00:35:51.730  00:06:10.999     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:51.730  00:06:10.999     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:51.730  00:06:10.999     174 		fi
00:35:51.730  00:06:10.999     175 	
00:35:51.730  00:06:10.999     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:51.730  00:06:10.999  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:51.730  00:06:10.999     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:51.730  00:06:10.999     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:51.730  00:06:10.999     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:51.730  00:06:10.999     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:51.730  00:06:10.999     182 		run_test "version" $rootdir/test/app/version.sh
00:35:51.730  00:06:10.999       ...
00:35:51.730  00:06:10.999  
00:35:51.730  00:06:10.999  ========== Backtrace end =========='
00:35:51.730  + [[ -n 00:06:10.998  ========== Backtrace start: ==========
00:35:51.730  00:06:10.998  
00:35:51.730  00:06:10.998  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/common/autotest_common.sh:960 -> killprocess(["115657"])
00:35:51.730  00:06:10.998       ...
00:35:51.730  00:06:10.998     955 				kill $1
00:35:51.730  00:06:10.998     956 			fi
00:35:51.730  00:06:10.998     957 	
00:35:51.730  00:06:10.998     958 			# wait for the process regardless if its the dummy sudo one
00:35:51.730  00:06:10.998     959 			# or the actual app - it should terminate anyway
00:35:51.730  00:06:10.998  => 960 			wait $1
00:35:51.730  00:06:10.998     961 		else
00:35:51.730  00:06:10.998     962 			# the process is not there anymore
00:35:51.730  00:06:10.998     963 			echo "Process with pid $1 is not found"
00:35:51.730  00:06:10.998     964 		fi
00:35:51.730  00:06:10.998     965 	}
00:35:51.730  00:06:10.998       ...
00:35:51.730  00:06:10.998  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:51.730  00:06:10.998       ...
00:35:51.730  00:06:10.998     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:51.730  00:06:10.998     35  			sleep 3
00:35:51.730  00:06:10.998     36  		done
00:35:51.730  00:06:10.998     37  	
00:35:51.730  00:06:10.998     38  		waitforlisten $repeat_pid $rpc_server
00:35:51.730  00:06:10.998  => 39  		killprocess $repeat_pid
00:35:51.730  00:06:10.998     40  		trap - SIGINT SIGTERM EXIT
00:35:51.730  00:06:10.998     41  	
00:35:51.730  00:06:10.998     42  		return 0
00:35:51.730  00:06:10.998     43  	}
00:35:51.730  00:06:10.998     44  	
00:35:51.730  00:06:10.998       ...
00:35:51.730  00:06:10.998  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:51.730  00:06:10.998       ...
00:35:51.730  00:06:10.998     1106		timing_enter $test_name
00:35:51.730  00:06:10.998     1107		echo "************************************"
00:35:51.730  00:06:10.998     1108		echo "START TEST $test_name"
00:35:51.730  00:06:10.998     1109		echo "************************************"
00:35:51.730  00:06:10.998     1110		xtrace_restore
00:35:51.730  00:06:10.998     1111		time "$@"
00:35:51.730  00:06:10.998     1112		xtrace_disable
00:35:51.730  00:06:10.998     1113		echo "************************************"
00:35:51.730  00:06:10.998     1114		echo "END TEST $test_name"
00:35:51.730  00:06:10.998     1115		echo "************************************"
00:35:51.730  00:06:10.998     1116		timing_exit $test_name
00:35:51.730  00:06:10.998       ...
00:35:51.730  00:06:10.998  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/event/event.sh:52 -> main([])
00:35:51.730  00:06:10.998       ...
00:35:51.730  00:06:10.998     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:51.730  00:06:10.998     48  	
00:35:51.730  00:06:10.998     49  	if [ $(uname -s) = Linux ]; then
00:35:51.730  00:06:10.998     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:51.730  00:06:10.998     51  		if modprobe -n nbd; then
00:35:51.730  00:06:10.998  => 52  			run_test "app_repeat" app_repeat_test
00:35:51.730  00:06:10.998     53  		fi
00:35:51.730  00:06:10.998     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:51.730  00:06:10.998     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:51.730  00:06:10.998     56  		fi
00:35:51.730  00:06:10.998     57  	fi
00:35:51.730  00:06:10.998       ...
00:35:51.730  00:06:10.998  
00:35:51.730  00:06:10.998  ========== Backtrace end ==========
00:35:51.730  00:06:10.999  ========== Backtrace start: ==========
00:35:51.730  00:06:10.999  
00:35:51.730  00:06:10.999  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/event/event.sh"])
00:35:51.730  00:06:10.999       ...
00:35:51.730  00:06:10.999     1106		timing_enter $test_name
00:35:51.730  00:06:10.999     1107		echo "************************************"
00:35:51.730  00:06:10.999     1108		echo "START TEST $test_name"
00:35:51.730  00:06:10.999     1109		echo "************************************"
00:35:51.730  00:06:10.999     1110		xtrace_restore
00:35:51.730  00:06:10.999     1111		time "$@"
00:35:51.730  00:06:10.999     1112		xtrace_disable
00:35:51.730  00:06:10.999     1113		echo "************************************"
00:35:51.730  00:06:10.999     1114		echo "END TEST $test_name"
00:35:51.730  00:06:10.999     1115		echo "************************************"
00:35:51.730  00:06:10.999     1116		timing_exit $test_name
00:35:51.730  00:06:10.999       ...
00:35:51.730  00:06:10.999  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/autotest.sh:177 -> main(["/var/jenkins/workspace/vfio-user-phy-autotest/autorun-spdk.conf"])
00:35:51.730  00:06:10.999       ...
00:35:51.731  00:06:10.999     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:51.731  00:06:10.999     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:51.731  00:06:10.999     174 		fi
00:35:51.731  00:06:10.999     175 	
00:35:51.731  00:06:10.999     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:51.731  00:06:10.999  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:51.731  00:06:10.999     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:51.731  00:06:10.999     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:51.731  00:06:10.999     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:51.731  00:06:10.999     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:51.731  00:06:10.999     182 		run_test "version" $rootdir/test/app/version.sh
00:35:51.731  00:06:10.999       ...
00:35:51.731  00:06:10.999  
00:35:51.731  00:06:10.999  ========== Backtrace end ========== ]]
00:35:51.731  + echo '00:06:10.998  ========== Backtrace start: ==========
00:35:51.731  00:06:10.998  
00:35:51.731  00:06:10.998  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/common/autotest_common.sh:960 -> killprocess(["115657"])
00:35:51.731  00:06:10.998       ...
00:35:51.731  00:06:10.998     955 				kill $1
00:35:51.731  00:06:10.998     956 			fi
00:35:51.731  00:06:10.998     957 	
00:35:51.731  00:06:10.998     958 			# wait for the process regardless if its the dummy sudo one
00:35:51.731  00:06:10.998     959 			# or the actual app - it should terminate anyway
00:35:51.731  00:06:10.998  => 960 			wait $1
00:35:51.731  00:06:10.998     961 		else
00:35:51.731  00:06:10.998     962 			# the process is not there anymore
00:35:51.731  00:06:10.998     963 			echo "Process with pid $1 is not found"
00:35:51.731  00:06:10.998     964 		fi
00:35:51.731  00:06:10.998     965 	}
00:35:51.731  00:06:10.998       ...
00:35:51.731  00:06:10.998  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:51.731  00:06:10.998       ...
00:35:51.731  00:06:10.998     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:51.731  00:06:10.998     35  			sleep 3
00:35:51.731  00:06:10.998     36  		done
00:35:51.731  00:06:10.998     37  	
00:35:51.731  00:06:10.998     38  		waitforlisten $repeat_pid $rpc_server
00:35:51.731  00:06:10.998  => 39  		killprocess $repeat_pid
00:35:51.731  00:06:10.998     40  		trap - SIGINT SIGTERM EXIT
00:35:51.731  00:06:10.998     41  	
00:35:51.731  00:06:10.998     42  		return 0
00:35:51.731  00:06:10.998     43  	}
00:35:51.731  00:06:10.998     44  	
00:35:51.731  00:06:10.998       ...
00:35:51.731  00:06:10.998  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:51.731  00:06:10.998       ...
00:35:51.731  00:06:10.998     1106		timing_enter $test_name
00:35:51.731  00:06:10.998     1107		echo "************************************"
00:35:51.731  00:06:10.998     1108		echo "START TEST $test_name"
00:35:51.731  00:06:10.998     1109		echo "************************************"
00:35:51.731  00:06:10.998     1110		xtrace_restore
00:35:51.731  00:06:10.998     1111		time "$@"
00:35:51.731  00:06:10.998     1112		xtrace_disable
00:35:51.731  00:06:10.998     1113		echo "************************************"
00:35:51.731  00:06:10.998     1114		echo "END TEST $test_name"
00:35:51.731  00:06:10.998     1115		echo "************************************"
00:35:51.731  00:06:10.998     1116		timing_exit $test_name
00:35:51.731  00:06:10.998       ...
00:35:51.731  00:06:10.998  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/event/event.sh:52 -> main([])
00:35:51.731  00:06:10.998       ...
00:35:51.731  00:06:10.998     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:51.731  00:06:10.998     48  	
00:35:51.731  00:06:10.998     49  	if [ $(uname -s) = Linux ]; then
00:35:51.731  00:06:10.998     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:51.731  00:06:10.998     51  		if modprobe -n nbd; then
00:35:51.731  00:06:10.998  => 52  			run_test "app_repeat" app_repeat_test
00:35:51.731  00:06:10.998     53  		fi
00:35:51.731  00:06:10.998     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:51.731  00:06:10.998     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:51.731  00:06:10.998     56  		fi
00:35:51.731  00:06:10.998     57  	fi
00:35:51.731  00:06:10.998       ...
00:35:51.731  00:06:10.998  
00:35:51.731  00:06:10.998  ========== Backtrace end ==========
00:35:51.731  00:06:10.999  ========== Backtrace start: ==========
00:35:51.731  00:06:10.999  
00:35:51.731  00:06:10.999  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/var/jenkins/workspace/vfio-user-phy-autotest/spdk/test/event/event.sh"])
00:35:51.731  00:06:10.999       ...
00:35:51.731  00:06:10.999     1106		timing_enter $test_name
00:35:51.731  00:06:10.999     1107		echo "************************************"
00:35:51.731  00:06:10.999     1108		echo "START TEST $test_name"
00:35:51.731  00:06:10.999     1109		echo "************************************"
00:35:51.731  00:06:10.999     1110		xtrace_restore
00:35:51.731  00:06:10.999     1111		time "$@"
00:35:51.731  00:06:10.999     1112		xtrace_disable
00:35:51.731  00:06:10.999     1113		echo "************************************"
00:35:51.731  00:06:10.999     1114		echo "END TEST $test_name"
00:35:51.731  00:06:10.999     1115		echo "************************************"
00:35:51.731  00:06:10.999     1116		timing_exit $test_name
00:35:51.731  00:06:10.999       ...
00:35:51.731  00:06:10.999  in /var/jenkins/workspace/vfio-user-phy-autotest/spdk/autotest.sh:177 -> main(["/var/jenkins/workspace/vfio-user-phy-autotest/autorun-spdk.conf"])
00:35:51.731  00:06:10.999       ...
00:35:51.731  00:06:10.999     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:51.731  00:06:10.999     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:51.731  00:06:10.999     174 		fi
00:35:51.731  00:06:10.999     175 	
00:35:51.731  00:06:10.999     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:51.731  00:06:10.999  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:51.731  00:06:10.999     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:51.731  00:06:10.999     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:51.731  00:06:10.999     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:51.731  00:06:10.999     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:51.731  00:06:10.999     182 		run_test "version" $rootdir/test/app/version.sh
00:35:51.731  00:06:10.999       ...
00:35:51.731  00:06:10.999  
00:35:51.731  00:06:10.999  ========== Backtrace end =========='
00:35:51.731  + for build in "${build_log_list[@]}"
00:35:51.731  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_19494/build.log
00:35:51.731  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.731  + build_log=/var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_19494/build.log
00:35:51.731  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_19494/build.log
00:35:51.731  ++ basename /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_19494
00:35:51.731  + job_name=build-files-docker-autotest_19494
00:35:51.731  + start_pattern='Backtrace start'
00:35:51.731  + end_pattern='Backtrace end'
00:35:51.731  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_19494/build.log ]]
00:35:51.731  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_19494/build.log
00:35:51.731  + backtraces=
00:35:51.731  + [[ -n '' ]]
00:35:51.731  + for build in "${build_log_list[@]}"
00:35:51.731  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_23432/build.log
00:35:51.731  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.731  + build_log=/var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_23432/build.log
00:35:51.731  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_23432/build.log
00:35:51.731  ++ basename /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_23432
00:35:51.731  + job_name=vhost-phy-autotest_23432
00:35:51.731  + start_pattern='Backtrace start'
00:35:51.731  + end_pattern='Backtrace end'
00:35:51.731  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_23432/build.log ]]
00:35:51.731  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_23432/build.log
00:35:51.731  + backtraces='00:07:50.388  ========== Backtrace start: ==========
00:35:51.731  00:07:50.388  
00:35:51.731  00:07:50.388  in /var/jenkins/workspace/vhost-phy-autotest/spdk/test/common/autotest_common.sh:960 -> killprocess(["127720"])
00:35:51.731  00:07:50.388       ...
00:35:51.731  00:07:50.388     955 				kill $1
00:35:51.731  00:07:50.388     956 			fi
00:35:51.731  00:07:50.388     957 	
00:35:51.731  00:07:50.388     958 			# wait for the process regardless if its the dummy sudo one
00:35:51.731  00:07:50.388     959 			# or the actual app - it should terminate anyway
00:35:51.731  00:07:50.388  => 960 			wait $1
00:35:51.731  00:07:50.388     961 		else
00:35:51.731  00:07:50.388     962 			# the process is not there anymore
00:35:51.731  00:07:50.388     963 			echo "Process with pid $1 is not found"
00:35:51.731  00:07:50.388     964 		fi
00:35:51.731  00:07:50.388     965 	}
00:35:51.731  00:07:50.388       ...
00:35:51.732  00:07:50.388  in /var/jenkins/workspace/vhost-phy-autotest/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:51.732  00:07:50.388       ...
00:35:51.732  00:07:50.388     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:51.732  00:07:50.388     35  			sleep 3
00:35:51.732  00:07:50.388     36  		done
00:35:51.732  00:07:50.388     37  	
00:35:51.732  00:07:50.388     38  		waitforlisten $repeat_pid $rpc_server
00:35:51.732  00:07:50.388  => 39  		killprocess $repeat_pid
00:35:51.732  00:07:50.388     40  		trap - SIGINT SIGTERM EXIT
00:35:51.732  00:07:50.388     41  	
00:35:51.732  00:07:50.388     42  		return 0
00:35:51.732  00:07:50.388     43  	}
00:35:51.732  00:07:50.388     44  	
00:35:51.732  00:07:50.388       ...
00:35:51.732  00:07:50.389  in /var/jenkins/workspace/vhost-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:51.732  00:07:50.389       ...
00:35:51.732  00:07:50.389     1106		timing_enter $test_name
00:35:51.732  00:07:50.389     1107		echo "************************************"
00:35:51.732  00:07:50.389     1108		echo "START TEST $test_name"
00:35:51.732  00:07:50.389     1109		echo "************************************"
00:35:51.732  00:07:50.389     1110		xtrace_restore
00:35:51.732  00:07:50.389     1111		time "$@"
00:35:51.732  00:07:50.389     1112		xtrace_disable
00:35:51.732  00:07:50.389     1113		echo "************************************"
00:35:51.732  00:07:50.389     1114		echo "END TEST $test_name"
00:35:51.732  00:07:50.389     1115		echo "************************************"
00:35:51.732  00:07:50.389     1116		timing_exit $test_name
00:35:51.732  00:07:50.389       ...
00:35:51.732  00:07:50.389  in /var/jenkins/workspace/vhost-phy-autotest/spdk/test/event/event.sh:52 -> main([])
00:35:51.732  00:07:50.389       ...
00:35:51.732  00:07:50.389     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:51.732  00:07:50.389     48  	
00:35:51.732  00:07:50.389     49  	if [ $(uname -s) = Linux ]; then
00:35:51.732  00:07:50.389     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:51.732  00:07:50.389     51  		if modprobe -n nbd; then
00:35:51.732  00:07:50.389  => 52  			run_test "app_repeat" app_repeat_test
00:35:51.732  00:07:50.389     53  		fi
00:35:51.732  00:07:50.389     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:51.732  00:07:50.389     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:51.732  00:07:50.389     56  		fi
00:35:51.732  00:07:50.389     57  	fi
00:35:51.732  00:07:50.389       ...
00:35:51.732  00:07:50.389  
00:35:51.732  00:07:50.389  ========== Backtrace end ==========
00:35:51.732  00:07:50.389  ========== Backtrace start: ==========
00:35:51.732  00:07:50.389  
00:35:51.732  00:07:50.389  in /var/jenkins/workspace/vhost-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/var/jenkins/workspace/vhost-phy-autotest/spdk/test/event/event.sh"])
00:35:51.732  00:07:50.389       ...
00:35:51.732  00:07:50.389     1106		timing_enter $test_name
00:35:51.732  00:07:50.389     1107		echo "************************************"
00:35:51.732  00:07:50.389     1108		echo "START TEST $test_name"
00:35:51.732  00:07:50.389     1109		echo "************************************"
00:35:51.732  00:07:50.389     1110		xtrace_restore
00:35:51.732  00:07:50.389     1111		time "$@"
00:35:51.732  00:07:50.389     1112		xtrace_disable
00:35:51.732  00:07:50.389     1113		echo "************************************"
00:35:51.732  00:07:50.389     1114		echo "END TEST $test_name"
00:35:51.732  00:07:50.389     1115		echo "************************************"
00:35:51.732  00:07:50.389     1116		timing_exit $test_name
00:35:51.732  00:07:50.389       ...
00:35:51.732  00:07:50.389  in /var/jenkins/workspace/vhost-phy-autotest/spdk/autotest.sh:177 -> main(["/var/jenkins/workspace/vhost-phy-autotest/autorun-spdk.conf"])
00:35:51.732  00:07:50.389       ...
00:35:51.732  00:07:50.389     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:51.732  00:07:50.389     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:51.732  00:07:50.389     174 		fi
00:35:51.732  00:07:50.389     175 	
00:35:51.732  00:07:50.389     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:51.732  00:07:50.389  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:51.732  00:07:50.389     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:51.732  00:07:50.389     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:51.732  00:07:50.389     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:51.732  00:07:50.389     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:51.732  00:07:50.389     182 		run_test "version" $rootdir/test/app/version.sh
00:35:51.732  00:07:50.389       ...
00:35:51.732  00:07:50.389  
00:35:51.732  00:07:50.389  ========== Backtrace end =========='
00:35:51.732  + [[ -n 00:07:50.388  ========== Backtrace start: ==========
00:35:51.732  00:07:50.388  
00:35:51.732  00:07:50.388  in /var/jenkins/workspace/vhost-phy-autotest/spdk/test/common/autotest_common.sh:960 -> killprocess(["127720"])
00:35:51.732  00:07:50.388       ...
00:35:51.732  00:07:50.388     955 				kill $1
00:35:51.732  00:07:50.388     956 			fi
00:35:51.732  00:07:50.388     957 	
00:35:51.732  00:07:50.388     958 			# wait for the process regardless if its the dummy sudo one
00:35:51.732  00:07:50.388     959 			# or the actual app - it should terminate anyway
00:35:51.732  00:07:50.388  => 960 			wait $1
00:35:51.732  00:07:50.388     961 		else
00:35:51.732  00:07:50.388     962 			# the process is not there anymore
00:35:51.732  00:07:50.388     963 			echo "Process with pid $1 is not found"
00:35:51.732  00:07:50.388     964 		fi
00:35:51.732  00:07:50.388     965 	}
00:35:51.732  00:07:50.388       ...
00:35:51.732  00:07:50.388  in /var/jenkins/workspace/vhost-phy-autotest/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:51.732  00:07:50.388       ...
00:35:51.732  00:07:50.388     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:51.732  00:07:50.388     35  			sleep 3
00:35:51.732  00:07:50.388     36  		done
00:35:51.732  00:07:50.388     37  	
00:35:51.732  00:07:50.388     38  		waitforlisten $repeat_pid $rpc_server
00:35:51.732  00:07:50.388  => 39  		killprocess $repeat_pid
00:35:51.732  00:07:50.388     40  		trap - SIGINT SIGTERM EXIT
00:35:51.732  00:07:50.388     41  	
00:35:51.732  00:07:50.388     42  		return 0
00:35:51.732  00:07:50.388     43  	}
00:35:51.732  00:07:50.388     44  	
00:35:51.732  00:07:50.388       ...
00:35:51.732  00:07:50.389  in /var/jenkins/workspace/vhost-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:51.732  00:07:50.389       ...
00:35:51.732  00:07:50.389     1106		timing_enter $test_name
00:35:51.732  00:07:50.389     1107		echo "************************************"
00:35:51.732  00:07:50.389     1108		echo "START TEST $test_name"
00:35:51.732  00:07:50.389     1109		echo "************************************"
00:35:51.732  00:07:50.389     1110		xtrace_restore
00:35:51.732  00:07:50.389     1111		time "$@"
00:35:51.732  00:07:50.389     1112		xtrace_disable
00:35:51.732  00:07:50.389     1113		echo "************************************"
00:35:51.732  00:07:50.389     1114		echo "END TEST $test_name"
00:35:51.732  00:07:50.389     1115		echo "************************************"
00:35:51.732  00:07:50.389     1116		timing_exit $test_name
00:35:51.732  00:07:50.389       ...
00:35:51.732  00:07:50.389  in /var/jenkins/workspace/vhost-phy-autotest/spdk/test/event/event.sh:52 -> main([])
00:35:51.732  00:07:50.389       ...
00:35:51.732  00:07:50.389     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:51.732  00:07:50.389     48  	
00:35:51.732  00:07:50.389     49  	if [ $(uname -s) = Linux ]; then
00:35:51.732  00:07:50.389     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:51.732  00:07:50.389     51  		if modprobe -n nbd; then
00:35:51.732  00:07:50.389  => 52  			run_test "app_repeat" app_repeat_test
00:35:51.732  00:07:50.389     53  		fi
00:35:51.732  00:07:50.389     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:51.732  00:07:50.389     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:51.732  00:07:50.389     56  		fi
00:35:51.732  00:07:50.389     57  	fi
00:35:51.732  00:07:50.389       ...
00:35:51.732  00:07:50.389  
00:35:51.732  00:07:50.389  ========== Backtrace end ==========
00:35:51.732  00:07:50.389  ========== Backtrace start: ==========
00:35:51.732  00:07:50.389  
00:35:51.732  00:07:50.389  in /var/jenkins/workspace/vhost-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/var/jenkins/workspace/vhost-phy-autotest/spdk/test/event/event.sh"])
00:35:51.732  00:07:50.389       ...
00:35:51.732  00:07:50.389     1106		timing_enter $test_name
00:35:51.732  00:07:50.389     1107		echo "************************************"
00:35:51.732  00:07:50.389     1108		echo "START TEST $test_name"
00:35:51.732  00:07:50.389     1109		echo "************************************"
00:35:51.732  00:07:50.389     1110		xtrace_restore
00:35:51.732  00:07:50.389     1111		time "$@"
00:35:51.732  00:07:50.389     1112		xtrace_disable
00:35:51.732  00:07:50.389     1113		echo "************************************"
00:35:51.732  00:07:50.389     1114		echo "END TEST $test_name"
00:35:51.732  00:07:50.389     1115		echo "************************************"
00:35:51.732  00:07:50.389     1116		timing_exit $test_name
00:35:51.732  00:07:50.389       ...
00:35:51.732  00:07:50.389  in /var/jenkins/workspace/vhost-phy-autotest/spdk/autotest.sh:177 -> main(["/var/jenkins/workspace/vhost-phy-autotest/autorun-spdk.conf"])
00:35:51.733  00:07:50.389       ...
00:35:51.733  00:07:50.389     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:51.733  00:07:50.389     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:51.733  00:07:50.389     174 		fi
00:35:51.733  00:07:50.389     175 	
00:35:51.733  00:07:50.389     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:51.733  00:07:50.389  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:51.733  00:07:50.389     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:51.733  00:07:50.389     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:51.733  00:07:50.389     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:51.733  00:07:50.389     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:51.733  00:07:50.389     182 		run_test "version" $rootdir/test/app/version.sh
00:35:51.733  00:07:50.389       ...
00:35:51.733  00:07:50.389  
00:35:51.733  00:07:50.389  ========== Backtrace end ========== ]]
00:35:51.733  + echo '00:07:50.388  ========== Backtrace start: ==========
00:35:51.733  00:07:50.388  
00:35:51.733  00:07:50.388  in /var/jenkins/workspace/vhost-phy-autotest/spdk/test/common/autotest_common.sh:960 -> killprocess(["127720"])
00:35:51.733  00:07:50.388       ...
00:35:51.733  00:07:50.388     955 				kill $1
00:35:51.733  00:07:50.388     956 			fi
00:35:51.733  00:07:50.388     957 	
00:35:51.733  00:07:50.388     958 			# wait for the process regardless if its the dummy sudo one
00:35:51.733  00:07:50.388     959 			# or the actual app - it should terminate anyway
00:35:51.733  00:07:50.388  => 960 			wait $1
00:35:51.733  00:07:50.388     961 		else
00:35:51.733  00:07:50.388     962 			# the process is not there anymore
00:35:51.733  00:07:50.388     963 			echo "Process with pid $1 is not found"
00:35:51.733  00:07:50.388     964 		fi
00:35:51.733  00:07:50.388     965 	}
00:35:51.733  00:07:50.388       ...
00:35:51.733  00:07:50.388  in /var/jenkins/workspace/vhost-phy-autotest/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:51.733  00:07:50.388       ...
00:35:51.733  00:07:50.388     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:51.733  00:07:50.388     35  			sleep 3
00:35:51.733  00:07:50.388     36  		done
00:35:51.733  00:07:50.388     37  	
00:35:51.733  00:07:50.388     38  		waitforlisten $repeat_pid $rpc_server
00:35:51.733  00:07:50.388  => 39  		killprocess $repeat_pid
00:35:51.733  00:07:50.388     40  		trap - SIGINT SIGTERM EXIT
00:35:51.733  00:07:50.388     41  	
00:35:51.733  00:07:50.388     42  		return 0
00:35:51.733  00:07:50.388     43  	}
00:35:51.733  00:07:50.388     44  	
00:35:51.733  00:07:50.388       ...
00:35:51.733  00:07:50.389  in /var/jenkins/workspace/vhost-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:51.733  00:07:50.389       ...
00:35:51.733  00:07:50.389     1106		timing_enter $test_name
00:35:51.733  00:07:50.389     1107		echo "************************************"
00:35:51.733  00:07:50.389     1108		echo "START TEST $test_name"
00:35:51.733  00:07:50.389     1109		echo "************************************"
00:35:51.733  00:07:50.389     1110		xtrace_restore
00:35:51.733  00:07:50.389     1111		time "$@"
00:35:51.733  00:07:50.389     1112		xtrace_disable
00:35:51.733  00:07:50.389     1113		echo "************************************"
00:35:51.733  00:07:50.389     1114		echo "END TEST $test_name"
00:35:51.733  00:07:50.389     1115		echo "************************************"
00:35:51.733  00:07:50.389     1116		timing_exit $test_name
00:35:51.733  00:07:50.389       ...
00:35:51.733  00:07:50.389  in /var/jenkins/workspace/vhost-phy-autotest/spdk/test/event/event.sh:52 -> main([])
00:35:51.733  00:07:50.389       ...
00:35:51.733  00:07:50.389     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:51.733  00:07:50.389     48  	
00:35:51.733  00:07:50.389     49  	if [ $(uname -s) = Linux ]; then
00:35:51.733  00:07:50.389     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:51.733  00:07:50.389     51  		if modprobe -n nbd; then
00:35:51.733  00:07:50.389  => 52  			run_test "app_repeat" app_repeat_test
00:35:51.733  00:07:50.389     53  		fi
00:35:51.733  00:07:50.389     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:51.733  00:07:50.389     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:51.733  00:07:50.389     56  		fi
00:35:51.733  00:07:50.389     57  	fi
00:35:51.733  00:07:50.389       ...
00:35:51.733  00:07:50.389  
00:35:51.733  00:07:50.389  ========== Backtrace end ==========
00:35:51.733  00:07:50.389  ========== Backtrace start: ==========
00:35:51.733  00:07:50.389  
00:35:51.733  00:07:50.389  in /var/jenkins/workspace/vhost-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/var/jenkins/workspace/vhost-phy-autotest/spdk/test/event/event.sh"])
00:35:51.733  00:07:50.389       ...
00:35:51.733  00:07:50.389     1106		timing_enter $test_name
00:35:51.733  00:07:50.389     1107		echo "************************************"
00:35:51.733  00:07:50.389     1108		echo "START TEST $test_name"
00:35:51.733  00:07:50.389     1109		echo "************************************"
00:35:51.733  00:07:50.389     1110		xtrace_restore
00:35:51.733  00:07:50.389     1111		time "$@"
00:35:51.733  00:07:50.389     1112		xtrace_disable
00:35:51.733  00:07:50.389     1113		echo "************************************"
00:35:51.733  00:07:50.389     1114		echo "END TEST $test_name"
00:35:51.733  00:07:50.389     1115		echo "************************************"
00:35:51.733  00:07:50.389     1116		timing_exit $test_name
00:35:51.733  00:07:50.389       ...
00:35:51.733  00:07:50.389  in /var/jenkins/workspace/vhost-phy-autotest/spdk/autotest.sh:177 -> main(["/var/jenkins/workspace/vhost-phy-autotest/autorun-spdk.conf"])
00:35:51.733  00:07:50.389       ...
00:35:51.733  00:07:50.389     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:51.733  00:07:50.389     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:51.733  00:07:50.389     174 		fi
00:35:51.733  00:07:50.389     175 	
00:35:51.733  00:07:50.389     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:51.733  00:07:50.389  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:51.733  00:07:50.389     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:51.733  00:07:50.389     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:51.733  00:07:50.389     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:51.733  00:07:50.389     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:51.733  00:07:50.389     182 		run_test "version" $rootdir/test/app/version.sh
00:35:51.733  00:07:50.389       ...
00:35:51.733  00:07:50.389  
00:35:51.733  00:07:50.389  ========== Backtrace end =========='
00:35:51.733  + for build in "${build_log_list[@]}"
00:35:51.733  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_69651/build.log
00:35:51.733  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.733  + build_log=/var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_69651/build.log
00:35:51.733  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_69651/build.log
00:35:51.733  ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_69651
00:35:51.733  + job_name=nvmf-phy-autotest_69651
00:35:51.733  + start_pattern='Backtrace start'
00:35:51.733  + end_pattern='Backtrace end'
00:35:51.733  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_69651/build.log ]]
00:35:51.733  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_69651/build.log
00:35:51.733  + backtraces=
00:35:51.733  + [[ -n '' ]]
00:35:51.733  + for build in "${build_log_list[@]}"
00:35:51.733  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_17621/build.log
00:35:51.733  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.733  + build_log=/var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_17621/build.log
00:35:51.733  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_17621/build.log
00:35:51.733  ++ basename /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_17621
00:35:51.733  + job_name=scan-build-docker-autotest_17621
00:35:51.733  + start_pattern='Backtrace start'
00:35:51.733  + end_pattern='Backtrace end'
00:35:51.733  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_17621/build.log ]]
00:35:51.733  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_17621/build.log
00:35:51.733  + backtraces='00:04:05.031  ========== Backtrace start: ==========
00:35:51.733  00:04:05.031  
00:35:51.733  00:04:05.031  in /var/jenkins/workspace/scan-build-docker-autotest/spdk/test/common/autobuild_common.sh:225 -> make_fail_cleanup([])
00:35:51.733  00:04:05.031       ...
00:35:51.733  00:04:05.031     220 			scanoutput=$(ls -1 $out/scan-build-tmp/)
00:35:51.733  00:04:05.031     221 			mv $out/scan-build-tmp/$scanoutput $out/scan-build
00:35:51.733  00:04:05.031     222 			rm -rf $out/scan-build-tmp
00:35:51.733  00:04:05.031     223 			chmod -R a+rX $out/scan-build
00:35:51.733  00:04:05.031     224 		fi
00:35:51.733  00:04:05.031  => 225 		false
00:35:51.733  00:04:05.031     226 	}
00:35:51.733  00:04:05.031     227 	
00:35:51.733  00:04:05.031     228 	_scanbuild_make() {
00:35:51.733  00:04:05.031     229 		pass=true
00:35:51.733  00:04:05.031     230 		"$rootdir/configure" $config_params --without-shared
00:35:51.733  00:04:05.031       ...
00:35:51.733  00:04:05.031  in /var/jenkins/workspace/scan-build-docker-autotest/spdk/test/common/autobuild_common.sh:231 -> _scanbuild_make([])
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031     226 	}
00:35:51.734  00:04:05.031     227 	
00:35:51.734  00:04:05.031     228 	_scanbuild_make() {
00:35:51.734  00:04:05.031     229 		pass=true
00:35:51.734  00:04:05.031     230 		"$rootdir/configure" $config_params --without-shared
00:35:51.734  00:04:05.031  => 231 		$scanbuild $MAKE $MAKEFLAGS > $out/build_output.txt && rm -rf $out/scan-build-tmp || make_fail_cleanup
00:35:51.734  00:04:05.031     232 		xtrace_disable
00:35:51.734  00:04:05.031     233 	
00:35:51.734  00:04:05.031     234 		rm -f $out/*files.txt
00:35:51.734  00:04:05.031     235 		for ent in $(find app examples lib module test -type f | grep -vF ".h"); do
00:35:51.734  00:04:05.031     236 			if [[ $ent == lib/env_ocf* ]]; then continue; fi
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031  in /var/jenkins/workspace/scan-build-docker-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["scanbuild_make"],["_scanbuild_make"])
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031     1106		timing_enter $test_name
00:35:51.734  00:04:05.031     1107		echo "************************************"
00:35:51.734  00:04:05.031     1108		echo "START TEST $test_name"
00:35:51.734  00:04:05.031     1109		echo "************************************"
00:35:51.734  00:04:05.031     1110		xtrace_restore
00:35:51.734  00:04:05.031     1111		time "$@"
00:35:51.734  00:04:05.031     1112		xtrace_disable
00:35:51.734  00:04:05.031     1113		echo "************************************"
00:35:51.734  00:04:05.031     1114		echo "END TEST $test_name"
00:35:51.734  00:04:05.031     1115		echo "************************************"
00:35:51.734  00:04:05.031     1116		timing_exit $test_name
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031  in /var/jenkins/workspace/scan-build-docker-autotest/spdk/test/common/autobuild_common.sh:415 -> scanbuild_make([])
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031     410 	unittest_build() {
00:35:51.734  00:04:05.031     411 		run_test "unittest_build" _unittest_build
00:35:51.734  00:04:05.031     412 	}
00:35:51.734  00:04:05.031     413 	
00:35:51.734  00:04:05.031     414 	scanbuild_make() {
00:35:51.734  00:04:05.031  => 415 		run_test "scanbuild_make" _scanbuild_make
00:35:51.734  00:04:05.031     416 	}
00:35:51.734  00:04:05.031     417 	
00:35:51.734  00:04:05.031     418 	ocf_precompile() {
00:35:51.734  00:04:05.031     419 		run_test "autobuild_ocf_precompile" _ocf_precompile
00:35:51.734  00:04:05.031     420 	}
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031  in jbp/jenkins/jjb-config/jobs/scripts/docker-autoruner.sh:49 -> main([])
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031  backtrace unavailable
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031  
00:35:51.734  00:04:05.031  ========== Backtrace end =========='
00:35:51.734  + [[ -n 00:04:05.031  ========== Backtrace start: ==========
00:35:51.734  00:04:05.031  
00:35:51.734  00:04:05.031  in /var/jenkins/workspace/scan-build-docker-autotest/spdk/test/common/autobuild_common.sh:225 -> make_fail_cleanup([])
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031     220 			scanoutput=$(ls -1 $out/scan-build-tmp/)
00:35:51.734  00:04:05.031     221 			mv $out/scan-build-tmp/$scanoutput $out/scan-build
00:35:51.734  00:04:05.031     222 			rm -rf $out/scan-build-tmp
00:35:51.734  00:04:05.031     223 			chmod -R a+rX $out/scan-build
00:35:51.734  00:04:05.031     224 		fi
00:35:51.734  00:04:05.031  => 225 		false
00:35:51.734  00:04:05.031     226 	}
00:35:51.734  00:04:05.031     227 	
00:35:51.734  00:04:05.031     228 	_scanbuild_make() {
00:35:51.734  00:04:05.031     229 		pass=true
00:35:51.734  00:04:05.031     230 		"$rootdir/configure" $config_params --without-shared
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031  in /var/jenkins/workspace/scan-build-docker-autotest/spdk/test/common/autobuild_common.sh:231 -> _scanbuild_make([])
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031     226 	}
00:35:51.734  00:04:05.031     227 	
00:35:51.734  00:04:05.031     228 	_scanbuild_make() {
00:35:51.734  00:04:05.031     229 		pass=true
00:35:51.734  00:04:05.031     230 		"$rootdir/configure" $config_params --without-shared
00:35:51.734  00:04:05.031  => 231 		$scanbuild $MAKE $MAKEFLAGS > $out/build_output.txt && rm -rf $out/scan-build-tmp || make_fail_cleanup
00:35:51.734  00:04:05.031     232 		xtrace_disable
00:35:51.734  00:04:05.031     233 	
00:35:51.734  00:04:05.031     234 		rm -f $out/*files.txt
00:35:51.734  00:04:05.031     235 		for ent in $(find app examples lib module test -type f | grep -vF ".h"); do
00:35:51.734  00:04:05.031     236 			if [[ $ent == lib/env_ocf* ]]; then continue; fi
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031  in /var/jenkins/workspace/scan-build-docker-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["scanbuild_make"],["_scanbuild_make"])
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031     1106		timing_enter $test_name
00:35:51.734  00:04:05.031     1107		echo "************************************"
00:35:51.734  00:04:05.031     1108		echo "START TEST $test_name"
00:35:51.734  00:04:05.031     1109		echo "************************************"
00:35:51.734  00:04:05.031     1110		xtrace_restore
00:35:51.734  00:04:05.031     1111		time "$@"
00:35:51.734  00:04:05.031     1112		xtrace_disable
00:35:51.734  00:04:05.031     1113		echo "************************************"
00:35:51.734  00:04:05.031     1114		echo "END TEST $test_name"
00:35:51.734  00:04:05.031     1115		echo "************************************"
00:35:51.734  00:04:05.031     1116		timing_exit $test_name
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031  in /var/jenkins/workspace/scan-build-docker-autotest/spdk/test/common/autobuild_common.sh:415 -> scanbuild_make([])
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031     410 	unittest_build() {
00:35:51.734  00:04:05.031     411 		run_test "unittest_build" _unittest_build
00:35:51.734  00:04:05.031     412 	}
00:35:51.734  00:04:05.031     413 	
00:35:51.734  00:04:05.031     414 	scanbuild_make() {
00:35:51.734  00:04:05.031  => 415 		run_test "scanbuild_make" _scanbuild_make
00:35:51.734  00:04:05.031     416 	}
00:35:51.734  00:04:05.031     417 	
00:35:51.734  00:04:05.031     418 	ocf_precompile() {
00:35:51.734  00:04:05.031     419 		run_test "autobuild_ocf_precompile" _ocf_precompile
00:35:51.734  00:04:05.031     420 	}
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031  in jbp/jenkins/jjb-config/jobs/scripts/docker-autoruner.sh:49 -> main([])
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031  backtrace unavailable
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031  
00:35:51.734  00:04:05.031  ========== Backtrace end ========== ]]
00:35:51.734  + echo '00:04:05.031  ========== Backtrace start: ==========
00:35:51.734  00:04:05.031  
00:35:51.734  00:04:05.031  in /var/jenkins/workspace/scan-build-docker-autotest/spdk/test/common/autobuild_common.sh:225 -> make_fail_cleanup([])
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031     220 			scanoutput=$(ls -1 $out/scan-build-tmp/)
00:35:51.734  00:04:05.031     221 			mv $out/scan-build-tmp/$scanoutput $out/scan-build
00:35:51.734  00:04:05.031     222 			rm -rf $out/scan-build-tmp
00:35:51.734  00:04:05.031     223 			chmod -R a+rX $out/scan-build
00:35:51.734  00:04:05.031     224 		fi
00:35:51.734  00:04:05.031  => 225 		false
00:35:51.734  00:04:05.031     226 	}
00:35:51.734  00:04:05.031     227 	
00:35:51.734  00:04:05.031     228 	_scanbuild_make() {
00:35:51.734  00:04:05.031     229 		pass=true
00:35:51.734  00:04:05.031     230 		"$rootdir/configure" $config_params --without-shared
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031  in /var/jenkins/workspace/scan-build-docker-autotest/spdk/test/common/autobuild_common.sh:231 -> _scanbuild_make([])
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031     226 	}
00:35:51.734  00:04:05.031     227 	
00:35:51.734  00:04:05.031     228 	_scanbuild_make() {
00:35:51.734  00:04:05.031     229 		pass=true
00:35:51.734  00:04:05.031     230 		"$rootdir/configure" $config_params --without-shared
00:35:51.734  00:04:05.031  => 231 		$scanbuild $MAKE $MAKEFLAGS > $out/build_output.txt && rm -rf $out/scan-build-tmp || make_fail_cleanup
00:35:51.734  00:04:05.031     232 		xtrace_disable
00:35:51.734  00:04:05.031     233 	
00:35:51.734  00:04:05.031     234 		rm -f $out/*files.txt
00:35:51.734  00:04:05.031     235 		for ent in $(find app examples lib module test -type f | grep -vF ".h"); do
00:35:51.734  00:04:05.031     236 			if [[ $ent == lib/env_ocf* ]]; then continue; fi
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031  in /var/jenkins/workspace/scan-build-docker-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["scanbuild_make"],["_scanbuild_make"])
00:35:51.734  00:04:05.031       ...
00:35:51.734  00:04:05.031     1106		timing_enter $test_name
00:35:51.734  00:04:05.031     1107		echo "************************************"
00:35:51.734  00:04:05.031     1108		echo "START TEST $test_name"
00:35:51.734  00:04:05.031     1109		echo "************************************"
00:35:51.734  00:04:05.031     1110		xtrace_restore
00:35:51.734  00:04:05.031     1111		time "$@"
00:35:51.734  00:04:05.031     1112		xtrace_disable
00:35:51.734  00:04:05.031     1113		echo "************************************"
00:35:51.734  00:04:05.031     1114		echo "END TEST $test_name"
00:35:51.734  00:04:05.031     1115		echo "************************************"
00:35:51.734  00:04:05.031     1116		timing_exit $test_name
00:35:51.734  00:04:05.031       ...
00:35:51.735  00:04:05.031  in /var/jenkins/workspace/scan-build-docker-autotest/spdk/test/common/autobuild_common.sh:415 -> scanbuild_make([])
00:35:51.735  00:04:05.031       ...
00:35:51.735  00:04:05.031     410 	unittest_build() {
00:35:51.735  00:04:05.031     411 		run_test "unittest_build" _unittest_build
00:35:51.735  00:04:05.031     412 	}
00:35:51.735  00:04:05.031     413 	
00:35:51.735  00:04:05.031     414 	scanbuild_make() {
00:35:51.735  00:04:05.031  => 415 		run_test "scanbuild_make" _scanbuild_make
00:35:51.735  00:04:05.031     416 	}
00:35:51.735  00:04:05.031     417 	
00:35:51.735  00:04:05.031     418 	ocf_precompile() {
00:35:51.735  00:04:05.031     419 		run_test "autobuild_ocf_precompile" _ocf_precompile
00:35:51.735  00:04:05.031     420 	}
00:35:51.735  00:04:05.031       ...
00:35:51.735  00:04:05.031  in jbp/jenkins/jjb-config/jobs/scripts/docker-autoruner.sh:49 -> main([])
00:35:51.735  00:04:05.031       ...
00:35:51.735  00:04:05.031  backtrace unavailable
00:35:51.735  00:04:05.031       ...
00:35:51.735  00:04:05.031  
00:35:51.735  00:04:05.031  ========== Backtrace end =========='
00:35:51.735  + for build in "${build_log_list[@]}"
00:35:51.735  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_74582/build.log
00:35:51.735  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.735  + build_log=/var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_74582/build.log
00:35:51.735  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_74582/build.log
00:35:51.735  ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_74582
00:35:51.735  + job_name=nvmf-tcp-vg-autotest_74582
00:35:51.735  + start_pattern='Backtrace start'
00:35:51.735  + end_pattern='Backtrace end'
00:35:51.735  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_74582/build.log ]]
00:35:51.735  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_74582/build.log
00:35:51.735  + backtraces=
00:35:51.735  + [[ -n '' ]]
00:35:51.735  + for build in "${build_log_list[@]}"
00:35:51.735  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_13462/build.log
00:35:51.735  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.735  + build_log=/var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_13462/build.log
00:35:51.735  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_13462/build.log
00:35:51.735  ++ basename /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_13462
00:35:51.735  + job_name=rocky9-vg-autotest_13462
00:35:51.735  + start_pattern='Backtrace start'
00:35:51.735  + end_pattern='Backtrace end'
00:35:51.735  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_13462/build.log ]]
00:35:51.735  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_13462/build.log
00:35:51.735  + backtraces='00:14:01.450  ========== Backtrace start: ==========
00:35:51.735  00:14:01.450  
00:35:51.735  00:14:01.450  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:960 -> killprocess(["179993"])
00:35:51.735  00:14:01.450       ...
00:35:51.735  00:14:01.450     955 				kill $1
00:35:51.735  00:14:01.450     956 			fi
00:35:51.735  00:14:01.450     957 	
00:35:51.735  00:14:01.450     958 			# wait for the process regardless if its the dummy sudo one
00:35:51.735  00:14:01.450     959 			# or the actual app - it should terminate anyway
00:35:51.735  00:14:01.450  => 960 			wait $1
00:35:51.735  00:14:01.450     961 		else
00:35:51.735  00:14:01.450     962 			# the process is not there anymore
00:35:51.735  00:14:01.450     963 			echo "Process with pid $1 is not found"
00:35:51.735  00:14:01.450     964 		fi
00:35:51.735  00:14:01.450     965 	}
00:35:51.735  00:14:01.450       ...
00:35:51.735  00:14:01.450  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:51.735  00:14:01.450       ...
00:35:51.735  00:14:01.450     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:51.735  00:14:01.450     35  			sleep 3
00:35:51.735  00:14:01.450     36  		done
00:35:51.735  00:14:01.450     37  	
00:35:51.735  00:14:01.450     38  		waitforlisten $repeat_pid $rpc_server
00:35:51.735  00:14:01.450  => 39  		killprocess $repeat_pid
00:35:51.735  00:14:01.450     40  		trap - SIGINT SIGTERM EXIT
00:35:51.735  00:14:01.450     41  	
00:35:51.735  00:14:01.450     42  		return 0
00:35:51.735  00:14:01.450     43  	}
00:35:51.735  00:14:01.450     44  	
00:35:51.735  00:14:01.450       ...
00:35:51.735  00:14:01.450  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:51.735  00:14:01.450       ...
00:35:51.735  00:14:01.450     1106		timing_enter $test_name
00:35:51.735  00:14:01.450     1107		echo "************************************"
00:35:51.735  00:14:01.450     1108		echo "START TEST $test_name"
00:35:51.735  00:14:01.450     1109		echo "************************************"
00:35:51.735  00:14:01.450     1110		xtrace_restore
00:35:51.735  00:14:01.450     1111		time "$@"
00:35:51.735  00:14:01.450     1112		xtrace_disable
00:35:51.735  00:14:01.450     1113		echo "************************************"
00:35:51.735  00:14:01.450     1114		echo "END TEST $test_name"
00:35:51.735  00:14:01.450     1115		echo "************************************"
00:35:51.735  00:14:01.450     1116		timing_exit $test_name
00:35:51.735  00:14:01.450       ...
00:35:51.735  00:14:01.450  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:52 -> main([])
00:35:51.735  00:14:01.450       ...
00:35:51.735  00:14:01.450     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:51.735  00:14:01.450     48  	
00:35:51.735  00:14:01.450     49  	if [ $(uname -s) = Linux ]; then
00:35:51.735  00:14:01.450     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:51.735  00:14:01.450     51  		if modprobe -n nbd; then
00:35:51.735  00:14:01.450  => 52  			run_test "app_repeat" app_repeat_test
00:35:51.735  00:14:01.450     53  		fi
00:35:51.735  00:14:01.450     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:51.735  00:14:01.450     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:51.735  00:14:01.450     56  		fi
00:35:51.735  00:14:01.450     57  	fi
00:35:51.735  00:14:01.450       ...
00:35:51.735  00:14:01.450  
00:35:51.735  00:14:01.450  ========== Backtrace end ==========
00:35:51.735  00:14:01.450  ========== Backtrace start: ==========
00:35:51.735  00:14:01.450  
00:35:51.735  00:14:01.450  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/home/vagrant/spdk_repo/spdk/test/event/event.sh"])
00:35:51.735  00:14:01.450       ...
00:35:51.735  00:14:01.450     1106		timing_enter $test_name
00:35:51.735  00:14:01.450     1107		echo "************************************"
00:35:51.735  00:14:01.450     1108		echo "START TEST $test_name"
00:35:51.735  00:14:01.450     1109		echo "************************************"
00:35:51.735  00:14:01.450     1110		xtrace_restore
00:35:51.735  00:14:01.450     1111		time "$@"
00:35:51.735  00:14:01.450     1112		xtrace_disable
00:35:51.735  00:14:01.450     1113		echo "************************************"
00:35:51.735  00:14:01.450     1114		echo "END TEST $test_name"
00:35:51.735  00:14:01.450     1115		echo "************************************"
00:35:51.735  00:14:01.450     1116		timing_exit $test_name
00:35:51.735  00:14:01.450       ...
00:35:51.735  00:14:01.450  in /home/vagrant/spdk_repo/spdk/autotest.sh:177 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.735  00:14:01.450       ...
00:35:51.735  00:14:01.450     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:51.735  00:14:01.450     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:51.735  00:14:01.450     174 		fi
00:35:51.735  00:14:01.450     175 	
00:35:51.735  00:14:01.450     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:51.735  00:14:01.450  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:51.735  00:14:01.450     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:51.735  00:14:01.450     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:51.735  00:14:01.450     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:51.736  00:14:01.450     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:51.736  00:14:01.450     182 		run_test "version" $rootdir/test/app/version.sh
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450  
00:35:51.736  00:14:01.450  ========== Backtrace end =========='
00:35:51.736  + [[ -n 00:14:01.450  ========== Backtrace start: ==========
00:35:51.736  00:14:01.450  
00:35:51.736  00:14:01.450  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:960 -> killprocess(["179993"])
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450     955 				kill $1
00:35:51.736  00:14:01.450     956 			fi
00:35:51.736  00:14:01.450     957 	
00:35:51.736  00:14:01.450     958 			# wait for the process regardless if its the dummy sudo one
00:35:51.736  00:14:01.450     959 			# or the actual app - it should terminate anyway
00:35:51.736  00:14:01.450  => 960 			wait $1
00:35:51.736  00:14:01.450     961 		else
00:35:51.736  00:14:01.450     962 			# the process is not there anymore
00:35:51.736  00:14:01.450     963 			echo "Process with pid $1 is not found"
00:35:51.736  00:14:01.450     964 		fi
00:35:51.736  00:14:01.450     965 	}
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:51.736  00:14:01.450     35  			sleep 3
00:35:51.736  00:14:01.450     36  		done
00:35:51.736  00:14:01.450     37  	
00:35:51.736  00:14:01.450     38  		waitforlisten $repeat_pid $rpc_server
00:35:51.736  00:14:01.450  => 39  		killprocess $repeat_pid
00:35:51.736  00:14:01.450     40  		trap - SIGINT SIGTERM EXIT
00:35:51.736  00:14:01.450     41  	
00:35:51.736  00:14:01.450     42  		return 0
00:35:51.736  00:14:01.450     43  	}
00:35:51.736  00:14:01.450     44  	
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450     1106		timing_enter $test_name
00:35:51.736  00:14:01.450     1107		echo "************************************"
00:35:51.736  00:14:01.450     1108		echo "START TEST $test_name"
00:35:51.736  00:14:01.450     1109		echo "************************************"
00:35:51.736  00:14:01.450     1110		xtrace_restore
00:35:51.736  00:14:01.450     1111		time "$@"
00:35:51.736  00:14:01.450     1112		xtrace_disable
00:35:51.736  00:14:01.450     1113		echo "************************************"
00:35:51.736  00:14:01.450     1114		echo "END TEST $test_name"
00:35:51.736  00:14:01.450     1115		echo "************************************"
00:35:51.736  00:14:01.450     1116		timing_exit $test_name
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:52 -> main([])
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:51.736  00:14:01.450     48  	
00:35:51.736  00:14:01.450     49  	if [ $(uname -s) = Linux ]; then
00:35:51.736  00:14:01.450     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:51.736  00:14:01.450     51  		if modprobe -n nbd; then
00:35:51.736  00:14:01.450  => 52  			run_test "app_repeat" app_repeat_test
00:35:51.736  00:14:01.450     53  		fi
00:35:51.736  00:14:01.450     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:51.736  00:14:01.450     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:51.736  00:14:01.450     56  		fi
00:35:51.736  00:14:01.450     57  	fi
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450  
00:35:51.736  00:14:01.450  ========== Backtrace end ==========
00:35:51.736  00:14:01.450  ========== Backtrace start: ==========
00:35:51.736  00:14:01.450  
00:35:51.736  00:14:01.450  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/home/vagrant/spdk_repo/spdk/test/event/event.sh"])
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450     1106		timing_enter $test_name
00:35:51.736  00:14:01.450     1107		echo "************************************"
00:35:51.736  00:14:01.450     1108		echo "START TEST $test_name"
00:35:51.736  00:14:01.450     1109		echo "************************************"
00:35:51.736  00:14:01.450     1110		xtrace_restore
00:35:51.736  00:14:01.450     1111		time "$@"
00:35:51.736  00:14:01.450     1112		xtrace_disable
00:35:51.736  00:14:01.450     1113		echo "************************************"
00:35:51.736  00:14:01.450     1114		echo "END TEST $test_name"
00:35:51.736  00:14:01.450     1115		echo "************************************"
00:35:51.736  00:14:01.450     1116		timing_exit $test_name
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450  in /home/vagrant/spdk_repo/spdk/autotest.sh:177 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:51.736  00:14:01.450     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:51.736  00:14:01.450     174 		fi
00:35:51.736  00:14:01.450     175 	
00:35:51.736  00:14:01.450     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:51.736  00:14:01.450  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:51.736  00:14:01.450     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:51.736  00:14:01.450     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:51.736  00:14:01.450     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:51.736  00:14:01.450     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:51.736  00:14:01.450     182 		run_test "version" $rootdir/test/app/version.sh
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450  
00:35:51.736  00:14:01.450  ========== Backtrace end ========== ]]
00:35:51.736  + echo '00:14:01.450  ========== Backtrace start: ==========
00:35:51.736  00:14:01.450  
00:35:51.736  00:14:01.450  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:960 -> killprocess(["179993"])
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450     955 				kill $1
00:35:51.736  00:14:01.450     956 			fi
00:35:51.736  00:14:01.450     957 	
00:35:51.736  00:14:01.450     958 			# wait for the process regardless if its the dummy sudo one
00:35:51.736  00:14:01.450     959 			# or the actual app - it should terminate anyway
00:35:51.736  00:14:01.450  => 960 			wait $1
00:35:51.736  00:14:01.450     961 		else
00:35:51.736  00:14:01.450     962 			# the process is not there anymore
00:35:51.736  00:14:01.450     963 			echo "Process with pid $1 is not found"
00:35:51.736  00:14:01.450     964 		fi
00:35:51.736  00:14:01.450     965 	}
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:51.736  00:14:01.450     35  			sleep 3
00:35:51.736  00:14:01.450     36  		done
00:35:51.736  00:14:01.450     37  	
00:35:51.736  00:14:01.450     38  		waitforlisten $repeat_pid $rpc_server
00:35:51.736  00:14:01.450  => 39  		killprocess $repeat_pid
00:35:51.736  00:14:01.450     40  		trap - SIGINT SIGTERM EXIT
00:35:51.736  00:14:01.450     41  	
00:35:51.736  00:14:01.450     42  		return 0
00:35:51.736  00:14:01.450     43  	}
00:35:51.736  00:14:01.450     44  	
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450     1106		timing_enter $test_name
00:35:51.736  00:14:01.450     1107		echo "************************************"
00:35:51.736  00:14:01.450     1108		echo "START TEST $test_name"
00:35:51.736  00:14:01.450     1109		echo "************************************"
00:35:51.736  00:14:01.450     1110		xtrace_restore
00:35:51.736  00:14:01.450     1111		time "$@"
00:35:51.736  00:14:01.450     1112		xtrace_disable
00:35:51.736  00:14:01.450     1113		echo "************************************"
00:35:51.736  00:14:01.450     1114		echo "END TEST $test_name"
00:35:51.736  00:14:01.450     1115		echo "************************************"
00:35:51.736  00:14:01.450     1116		timing_exit $test_name
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:52 -> main([])
00:35:51.736  00:14:01.450       ...
00:35:51.736  00:14:01.450     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:51.736  00:14:01.450     48  	
00:35:51.736  00:14:01.450     49  	if [ $(uname -s) = Linux ]; then
00:35:51.736  00:14:01.450     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:51.736  00:14:01.450     51  		if modprobe -n nbd; then
00:35:51.736  00:14:01.450  => 52  			run_test "app_repeat" app_repeat_test
00:35:51.736  00:14:01.450     53  		fi
00:35:51.737  00:14:01.450     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:51.737  00:14:01.450     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:51.737  00:14:01.450     56  		fi
00:35:51.737  00:14:01.450     57  	fi
00:35:51.737  00:14:01.450       ...
00:35:51.737  00:14:01.450  
00:35:51.737  00:14:01.450  ========== Backtrace end ==========
00:35:51.737  00:14:01.450  ========== Backtrace start: ==========
00:35:51.737  00:14:01.450  
00:35:51.737  00:14:01.450  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/home/vagrant/spdk_repo/spdk/test/event/event.sh"])
00:35:51.737  00:14:01.450       ...
00:35:51.737  00:14:01.450     1106		timing_enter $test_name
00:35:51.737  00:14:01.450     1107		echo "************************************"
00:35:51.737  00:14:01.450     1108		echo "START TEST $test_name"
00:35:51.737  00:14:01.450     1109		echo "************************************"
00:35:51.737  00:14:01.450     1110		xtrace_restore
00:35:51.737  00:14:01.450     1111		time "$@"
00:35:51.737  00:14:01.450     1112		xtrace_disable
00:35:51.737  00:14:01.450     1113		echo "************************************"
00:35:51.737  00:14:01.450     1114		echo "END TEST $test_name"
00:35:51.737  00:14:01.450     1115		echo "************************************"
00:35:51.737  00:14:01.450     1116		timing_exit $test_name
00:35:51.737  00:14:01.450       ...
00:35:51.737  00:14:01.450  in /home/vagrant/spdk_repo/spdk/autotest.sh:177 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.737  00:14:01.450       ...
00:35:51.737  00:14:01.450     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:51.737  00:14:01.450     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:51.737  00:14:01.450     174 		fi
00:35:51.737  00:14:01.450     175 	
00:35:51.737  00:14:01.450     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:51.737  00:14:01.450  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:51.737  00:14:01.450     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:51.737  00:14:01.450     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:51.737  00:14:01.450     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:51.737  00:14:01.450     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:51.737  00:14:01.450     182 		run_test "version" $rootdir/test/app/version.sh
00:35:51.737  00:14:01.450       ...
00:35:51.737  00:14:01.450  
00:35:51.737  00:14:01.450  ========== Backtrace end =========='
00:35:51.737  + for build in "${build_log_list[@]}"
00:35:51.737  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_74615/build.log
00:35:51.737  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.737  + build_log=/var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_74615/build.log
00:35:51.737  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_74615/build.log
00:35:51.737  ++ basename /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_74615
00:35:51.737  + job_name=vhost-initiator-vg-autotest_74615
00:35:51.737  + start_pattern='Backtrace start'
00:35:51.737  + end_pattern='Backtrace end'
00:35:51.737  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_74615/build.log ]]
00:35:51.737  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_74615/build.log
00:35:51.737  + backtraces=
00:35:51.737  + [[ -n '' ]]
00:35:51.737  + for build in "${build_log_list[@]}"
00:35:51.737  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_23409/build.log
00:35:51.737  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.737  + build_log=/var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_23409/build.log
00:35:51.737  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_23409/build.log
00:35:51.737  ++ basename /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_23409
00:35:51.737  + job_name=crypto-phy-autotest_23409
00:35:51.737  + start_pattern='Backtrace start'
00:35:51.737  + end_pattern='Backtrace end'
00:35:51.737  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_23409/build.log ]]
00:35:51.737  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_23409/build.log
00:35:51.737  + backtraces=
00:35:51.737  + [[ -n '' ]]
00:35:51.737  + for build in "${build_log_list[@]}"
00:35:51.737  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_74631/build.log
00:35:51.737  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.737  + build_log=/var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_74631/build.log
00:35:51.737  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_74631/build.log
00:35:51.737  ++ basename /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_74631
00:35:51.737  + job_name=iscsi-vg-autotest_74631
00:35:51.737  + start_pattern='Backtrace start'
00:35:51.737  + end_pattern='Backtrace end'
00:35:51.737  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_74631/build.log ]]
00:35:51.737  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_74631/build.log
00:35:51.737  + backtraces='00:16:10.932  ========== Backtrace start: ==========
00:35:51.737  00:16:10.932  
00:35:51.737  00:16:10.932  in /home/vagrant/spdk_repo/spdk/test/iscsi_tgt/trace_record/trace_record.sh:96 -> main([])
00:35:51.737  00:16:10.932       ...
00:35:51.737  00:16:10.932     91  	
00:35:51.737  00:16:10.932     92  	trap '\''delete_tmp_files; iscsitestfini; exit 1'\'' SIGINT SIGTERM EXIT
00:35:51.737  00:16:10.932     93  	
00:35:51.737  00:16:10.932     94  	killprocess $iscsi_pid
00:35:51.737  00:16:10.932     95  	killprocess $record_pid
00:35:51.737  00:16:10.932  => 96  	$rootdir/build/bin/spdk_trace -f ${TRACE_RECORD_OUTPUT} > ${TRACE_TOOL_LOG}
00:35:51.737  00:16:10.932     97  	
00:35:51.737  00:16:10.932     98  	#verify trace record and trace tool
00:35:51.737  00:16:10.932     99  	#trace entries str in trace-record, like "Trace Size of lcore (0): 4136"
00:35:51.737  00:16:10.932     100 	record_num="$(grep "trace entries for lcore" ${TRACE_RECORD_NOTICE_LOG} | cut -d '\'' '\'' -f 2)"
00:35:51.737  00:16:10.932     101 	
00:35:51.737  00:16:10.932       ...
00:35:51.737  00:16:10.932  
00:35:51.737  00:16:10.932  ========== Backtrace end ==========
00:35:51.737  00:16:10.932  ========== Backtrace start: ==========
00:35:51.737  00:16:10.932  
00:35:51.737  00:16:10.932  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["iscsi_tgt_trace_record"],["/home/vagrant/spdk_repo/spdk/test/iscsi_tgt/trace_record/trace_record.sh"])
00:35:51.737  00:16:10.932       ...
00:35:51.737  00:16:10.932     1106		timing_enter $test_name
00:35:51.737  00:16:10.932     1107		echo "************************************"
00:35:51.737  00:16:10.932     1108		echo "START TEST $test_name"
00:35:51.737  00:16:10.932     1109		echo "************************************"
00:35:51.737  00:16:10.932     1110		xtrace_restore
00:35:51.737  00:16:10.932     1111		time "$@"
00:35:51.737  00:16:10.932     1112		xtrace_disable
00:35:51.737  00:16:10.932     1113		echo "************************************"
00:35:51.737  00:16:10.932     1114		echo "END TEST $test_name"
00:35:51.737  00:16:10.932     1115		echo "************************************"
00:35:51.737  00:16:10.932     1116		timing_exit $test_name
00:35:51.737  00:16:10.932       ...
00:35:51.737  00:16:10.932  in /home/vagrant/spdk_repo/spdk/test/iscsi_tgt/iscsi_tgt.sh:38 -> main([])
00:35:51.737  00:16:10.932       ...
00:35:51.737  00:16:10.932     33  	run_test "iscsi_tgt_rpc_config" $rootdir/test/iscsi_tgt/rpc_config/rpc_config.sh
00:35:51.737  00:16:10.932     34  	run_test "iscsi_tgt_iscsi_lvol" $rootdir/test/iscsi_tgt/lvol/iscsi_lvol.sh
00:35:51.737  00:16:10.932     35  	run_test "iscsi_tgt_fio" $rootdir/test/iscsi_tgt/fio/fio.sh
00:35:51.737  00:16:10.932     36  	run_test "iscsi_tgt_qos" $rootdir/test/iscsi_tgt/qos/qos.sh
00:35:51.737  00:16:10.932     37  	run_test "iscsi_tgt_ip_migration" $rootdir/test/iscsi_tgt/ip_migration/ip_migration.sh
00:35:51.737  00:16:10.932  => 38  	run_test "iscsi_tgt_trace_record" $rootdir/test/iscsi_tgt/trace_record/trace_record.sh
00:35:51.737  00:16:10.932     39  	run_test "iscsi_tgt_login_redirection" $rootdir/test/iscsi_tgt/login_redirection/login_redirection.sh
00:35:51.737  00:16:10.932     40  	run_test "iscsi_tgt_digests" $rootdir/test/iscsi_tgt/digests/digests.sh
00:35:51.737  00:16:10.932     41  	run_test "iscsi_tgt_fuzz" $rootdir/test/iscsi_tgt/fuzz/fuzz.sh
00:35:51.737  00:16:10.932     42  	run_test "iscsi_tgt_multiconnection" $rootdir/test/iscsi_tgt/multiconnection/multiconnection.sh
00:35:51.737  00:16:10.932     43  	
00:35:51.737  00:16:10.932       ...
00:35:51.737  00:16:10.932  
00:35:51.737  00:16:10.932  ========== Backtrace end ==========
00:35:51.737  00:16:10.932  ========== Backtrace start: ==========
00:35:51.737  00:16:10.932  
00:35:51.737  00:16:10.932  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["iscsi_tgt"],["/home/vagrant/spdk_repo/spdk/test/iscsi_tgt/iscsi_tgt.sh"])
00:35:51.737  00:16:10.932       ...
00:35:51.737  00:16:10.932     1106		timing_enter $test_name
00:35:51.737  00:16:10.932     1107		echo "************************************"
00:35:51.737  00:16:10.932     1108		echo "START TEST $test_name"
00:35:51.737  00:16:10.932     1109		echo "************************************"
00:35:51.737  00:16:10.932     1110		xtrace_restore
00:35:51.737  00:16:10.932     1111		time "$@"
00:35:51.737  00:16:10.932     1112		xtrace_disable
00:35:51.737  00:16:10.932     1113		echo "************************************"
00:35:51.737  00:16:10.932     1114		echo "END TEST $test_name"
00:35:51.737  00:16:10.932     1115		echo "************************************"
00:35:51.738  00:16:10.932     1116		timing_exit $test_name
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932  in /home/vagrant/spdk_repo/spdk/autotest.sh:261 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932     256 		fi
00:35:51.738  00:16:10.932     257 	
00:35:51.738  00:16:10.932     258 		timing_exit lib
00:35:51.738  00:16:10.932     259 	
00:35:51.738  00:16:10.932     260 		if [ $SPDK_TEST_ISCSI -eq 1 ]; then
00:35:51.738  00:16:10.932  => 261 			run_test "iscsi_tgt" $rootdir/test/iscsi_tgt/iscsi_tgt.sh
00:35:51.738  00:16:10.932     262 			run_test "spdkcli_iscsi" $rootdir/test/spdkcli/iscsi.sh
00:35:51.738  00:16:10.932     263 	
00:35:51.738  00:16:10.932     264 			# Run raid spdkcli test under iSCSI since blockdev tests run on systems that can'\''t run spdkcli yet
00:35:51.738  00:16:10.932     265 			run_test "spdkcli_raid" $rootdir/test/spdkcli/raid.sh
00:35:51.738  00:16:10.932     266 		fi
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932  
00:35:51.738  00:16:10.932  ========== Backtrace end =========='
00:35:51.738  + [[ -n 00:16:10.932  ========== Backtrace start: ==========
00:35:51.738  00:16:10.932  
00:35:51.738  00:16:10.932  in /home/vagrant/spdk_repo/spdk/test/iscsi_tgt/trace_record/trace_record.sh:96 -> main([])
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932     91  	
00:35:51.738  00:16:10.932     92  	trap 'delete_tmp_files; iscsitestfini; exit 1' SIGINT SIGTERM EXIT
00:35:51.738  00:16:10.932     93  	
00:35:51.738  00:16:10.932     94  	killprocess $iscsi_pid
00:35:51.738  00:16:10.932     95  	killprocess $record_pid
00:35:51.738  00:16:10.932  => 96  	$rootdir/build/bin/spdk_trace -f ${TRACE_RECORD_OUTPUT} > ${TRACE_TOOL_LOG}
00:35:51.738  00:16:10.932     97  	
00:35:51.738  00:16:10.932     98  	#verify trace record and trace tool
00:35:51.738  00:16:10.932     99  	#trace entries str in trace-record, like "Trace Size of lcore (0): 4136"
00:35:51.738  00:16:10.932     100 	record_num="$(grep "trace entries for lcore" ${TRACE_RECORD_NOTICE_LOG} | cut -d ' ' -f 2)"
00:35:51.738  00:16:10.932     101 	
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932  
00:35:51.738  00:16:10.932  ========== Backtrace end ==========
00:35:51.738  00:16:10.932  ========== Backtrace start: ==========
00:35:51.738  00:16:10.932  
00:35:51.738  00:16:10.932  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["iscsi_tgt_trace_record"],["/home/vagrant/spdk_repo/spdk/test/iscsi_tgt/trace_record/trace_record.sh"])
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932     1106		timing_enter $test_name
00:35:51.738  00:16:10.932     1107		echo "************************************"
00:35:51.738  00:16:10.932     1108		echo "START TEST $test_name"
00:35:51.738  00:16:10.932     1109		echo "************************************"
00:35:51.738  00:16:10.932     1110		xtrace_restore
00:35:51.738  00:16:10.932     1111		time "$@"
00:35:51.738  00:16:10.932     1112		xtrace_disable
00:35:51.738  00:16:10.932     1113		echo "************************************"
00:35:51.738  00:16:10.932     1114		echo "END TEST $test_name"
00:35:51.738  00:16:10.932     1115		echo "************************************"
00:35:51.738  00:16:10.932     1116		timing_exit $test_name
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932  in /home/vagrant/spdk_repo/spdk/test/iscsi_tgt/iscsi_tgt.sh:38 -> main([])
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932     33  	run_test "iscsi_tgt_rpc_config" $rootdir/test/iscsi_tgt/rpc_config/rpc_config.sh
00:35:51.738  00:16:10.932     34  	run_test "iscsi_tgt_iscsi_lvol" $rootdir/test/iscsi_tgt/lvol/iscsi_lvol.sh
00:35:51.738  00:16:10.932     35  	run_test "iscsi_tgt_fio" $rootdir/test/iscsi_tgt/fio/fio.sh
00:35:51.738  00:16:10.932     36  	run_test "iscsi_tgt_qos" $rootdir/test/iscsi_tgt/qos/qos.sh
00:35:51.738  00:16:10.932     37  	run_test "iscsi_tgt_ip_migration" $rootdir/test/iscsi_tgt/ip_migration/ip_migration.sh
00:35:51.738  00:16:10.932  => 38  	run_test "iscsi_tgt_trace_record" $rootdir/test/iscsi_tgt/trace_record/trace_record.sh
00:35:51.738  00:16:10.932     39  	run_test "iscsi_tgt_login_redirection" $rootdir/test/iscsi_tgt/login_redirection/login_redirection.sh
00:35:51.738  00:16:10.932     40  	run_test "iscsi_tgt_digests" $rootdir/test/iscsi_tgt/digests/digests.sh
00:35:51.738  00:16:10.932     41  	run_test "iscsi_tgt_fuzz" $rootdir/test/iscsi_tgt/fuzz/fuzz.sh
00:35:51.738  00:16:10.932     42  	run_test "iscsi_tgt_multiconnection" $rootdir/test/iscsi_tgt/multiconnection/multiconnection.sh
00:35:51.738  00:16:10.932     43  	
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932  
00:35:51.738  00:16:10.932  ========== Backtrace end ==========
00:35:51.738  00:16:10.932  ========== Backtrace start: ==========
00:35:51.738  00:16:10.932  
00:35:51.738  00:16:10.932  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["iscsi_tgt"],["/home/vagrant/spdk_repo/spdk/test/iscsi_tgt/iscsi_tgt.sh"])
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932     1106		timing_enter $test_name
00:35:51.738  00:16:10.932     1107		echo "************************************"
00:35:51.738  00:16:10.932     1108		echo "START TEST $test_name"
00:35:51.738  00:16:10.932     1109		echo "************************************"
00:35:51.738  00:16:10.932     1110		xtrace_restore
00:35:51.738  00:16:10.932     1111		time "$@"
00:35:51.738  00:16:10.932     1112		xtrace_disable
00:35:51.738  00:16:10.932     1113		echo "************************************"
00:35:51.738  00:16:10.932     1114		echo "END TEST $test_name"
00:35:51.738  00:16:10.932     1115		echo "************************************"
00:35:51.738  00:16:10.932     1116		timing_exit $test_name
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932  in /home/vagrant/spdk_repo/spdk/autotest.sh:261 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932     256 		fi
00:35:51.738  00:16:10.932     257 	
00:35:51.738  00:16:10.932     258 		timing_exit lib
00:35:51.738  00:16:10.932     259 	
00:35:51.738  00:16:10.932     260 		if [ $SPDK_TEST_ISCSI -eq 1 ]; then
00:35:51.738  00:16:10.932  => 261 			run_test "iscsi_tgt" $rootdir/test/iscsi_tgt/iscsi_tgt.sh
00:35:51.738  00:16:10.932     262 			run_test "spdkcli_iscsi" $rootdir/test/spdkcli/iscsi.sh
00:35:51.738  00:16:10.932     263 	
00:35:51.738  00:16:10.932     264 			# Run raid spdkcli test under iSCSI since blockdev tests run on systems that can't run spdkcli yet
00:35:51.738  00:16:10.932     265 			run_test "spdkcli_raid" $rootdir/test/spdkcli/raid.sh
00:35:51.738  00:16:10.932     266 		fi
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932  
00:35:51.738  00:16:10.932  ========== Backtrace end ========== ]]
00:35:51.738  + echo '00:16:10.932  ========== Backtrace start: ==========
00:35:51.738  00:16:10.932  
00:35:51.738  00:16:10.932  in /home/vagrant/spdk_repo/spdk/test/iscsi_tgt/trace_record/trace_record.sh:96 -> main([])
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932     91  	
00:35:51.738  00:16:10.932     92  	trap '\''delete_tmp_files; iscsitestfini; exit 1'\'' SIGINT SIGTERM EXIT
00:35:51.738  00:16:10.932     93  	
00:35:51.738  00:16:10.932     94  	killprocess $iscsi_pid
00:35:51.738  00:16:10.932     95  	killprocess $record_pid
00:35:51.738  00:16:10.932  => 96  	$rootdir/build/bin/spdk_trace -f ${TRACE_RECORD_OUTPUT} > ${TRACE_TOOL_LOG}
00:35:51.738  00:16:10.932     97  	
00:35:51.738  00:16:10.932     98  	#verify trace record and trace tool
00:35:51.738  00:16:10.932     99  	#trace entries str in trace-record, like "Trace Size of lcore (0): 4136"
00:35:51.738  00:16:10.932     100 	record_num="$(grep "trace entries for lcore" ${TRACE_RECORD_NOTICE_LOG} | cut -d '\'' '\'' -f 2)"
00:35:51.738  00:16:10.932     101 	
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932  
00:35:51.738  00:16:10.932  ========== Backtrace end ==========
00:35:51.738  00:16:10.932  ========== Backtrace start: ==========
00:35:51.738  00:16:10.932  
00:35:51.738  00:16:10.932  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["iscsi_tgt_trace_record"],["/home/vagrant/spdk_repo/spdk/test/iscsi_tgt/trace_record/trace_record.sh"])
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932     1106		timing_enter $test_name
00:35:51.738  00:16:10.932     1107		echo "************************************"
00:35:51.738  00:16:10.932     1108		echo "START TEST $test_name"
00:35:51.738  00:16:10.932     1109		echo "************************************"
00:35:51.738  00:16:10.932     1110		xtrace_restore
00:35:51.738  00:16:10.932     1111		time "$@"
00:35:51.738  00:16:10.932     1112		xtrace_disable
00:35:51.738  00:16:10.932     1113		echo "************************************"
00:35:51.738  00:16:10.932     1114		echo "END TEST $test_name"
00:35:51.738  00:16:10.932     1115		echo "************************************"
00:35:51.738  00:16:10.932     1116		timing_exit $test_name
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932  in /home/vagrant/spdk_repo/spdk/test/iscsi_tgt/iscsi_tgt.sh:38 -> main([])
00:35:51.738  00:16:10.932       ...
00:35:51.738  00:16:10.932     33  	run_test "iscsi_tgt_rpc_config" $rootdir/test/iscsi_tgt/rpc_config/rpc_config.sh
00:35:51.738  00:16:10.932     34  	run_test "iscsi_tgt_iscsi_lvol" $rootdir/test/iscsi_tgt/lvol/iscsi_lvol.sh
00:35:51.738  00:16:10.932     35  	run_test "iscsi_tgt_fio" $rootdir/test/iscsi_tgt/fio/fio.sh
00:35:51.738  00:16:10.932     36  	run_test "iscsi_tgt_qos" $rootdir/test/iscsi_tgt/qos/qos.sh
00:35:51.738  00:16:10.932     37  	run_test "iscsi_tgt_ip_migration" $rootdir/test/iscsi_tgt/ip_migration/ip_migration.sh
00:35:51.738  00:16:10.932  => 38  	run_test "iscsi_tgt_trace_record" $rootdir/test/iscsi_tgt/trace_record/trace_record.sh
00:35:51.738  00:16:10.932     39  	run_test "iscsi_tgt_login_redirection" $rootdir/test/iscsi_tgt/login_redirection/login_redirection.sh
00:35:51.739  00:16:10.932     40  	run_test "iscsi_tgt_digests" $rootdir/test/iscsi_tgt/digests/digests.sh
00:35:51.739  00:16:10.932     41  	run_test "iscsi_tgt_fuzz" $rootdir/test/iscsi_tgt/fuzz/fuzz.sh
00:35:51.739  00:16:10.932     42  	run_test "iscsi_tgt_multiconnection" $rootdir/test/iscsi_tgt/multiconnection/multiconnection.sh
00:35:51.739  00:16:10.932     43  	
00:35:51.739  00:16:10.932       ...
00:35:51.739  00:16:10.932  
00:35:51.739  00:16:10.932  ========== Backtrace end ==========
00:35:51.739  00:16:10.932  ========== Backtrace start: ==========
00:35:51.739  00:16:10.932  
00:35:51.739  00:16:10.932  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["iscsi_tgt"],["/home/vagrant/spdk_repo/spdk/test/iscsi_tgt/iscsi_tgt.sh"])
00:35:51.739  00:16:10.932       ...
00:35:51.739  00:16:10.932     1106		timing_enter $test_name
00:35:51.739  00:16:10.932     1107		echo "************************************"
00:35:51.739  00:16:10.932     1108		echo "START TEST $test_name"
00:35:51.739  00:16:10.932     1109		echo "************************************"
00:35:51.739  00:16:10.932     1110		xtrace_restore
00:35:51.739  00:16:10.932     1111		time "$@"
00:35:51.739  00:16:10.932     1112		xtrace_disable
00:35:51.739  00:16:10.932     1113		echo "************************************"
00:35:51.739  00:16:10.932     1114		echo "END TEST $test_name"
00:35:51.739  00:16:10.932     1115		echo "************************************"
00:35:51.739  00:16:10.932     1116		timing_exit $test_name
00:35:51.739  00:16:10.932       ...
00:35:51.739  00:16:10.932  in /home/vagrant/spdk_repo/spdk/autotest.sh:261 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.739  00:16:10.932       ...
00:35:51.739  00:16:10.932     256 		fi
00:35:51.739  00:16:10.932     257 	
00:35:51.739  00:16:10.932     258 		timing_exit lib
00:35:51.739  00:16:10.932     259 	
00:35:51.739  00:16:10.932     260 		if [ $SPDK_TEST_ISCSI -eq 1 ]; then
00:35:51.739  00:16:10.932  => 261 			run_test "iscsi_tgt" $rootdir/test/iscsi_tgt/iscsi_tgt.sh
00:35:51.739  00:16:10.932     262 			run_test "spdkcli_iscsi" $rootdir/test/spdkcli/iscsi.sh
00:35:51.739  00:16:10.932     263 	
00:35:51.739  00:16:10.932     264 			# Run raid spdkcli test under iSCSI since blockdev tests run on systems that can'\''t run spdkcli yet
00:35:51.739  00:16:10.932     265 			run_test "spdkcli_raid" $rootdir/test/spdkcli/raid.sh
00:35:51.739  00:16:10.932     266 		fi
00:35:51.739  00:16:10.932       ...
00:35:51.739  00:16:10.932  
00:35:51.739  00:16:10.932  ========== Backtrace end =========='
00:35:51.739  + for build in "${build_log_list[@]}"
00:35:51.739  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_56903/build.log
00:35:51.739  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.739  + build_log=/var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_56903/build.log
00:35:51.739  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_56903/build.log
00:35:51.739  ++ basename /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_56903
00:35:51.739  + job_name=freebsd-vg-autotest_56903
00:35:51.739  + start_pattern='Backtrace start'
00:35:51.739  + end_pattern='Backtrace end'
00:35:51.739  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_56903/build.log ]]
00:35:51.739  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_56903/build.log
00:35:51.739  + backtraces='00:14:53.913  ========== Backtrace start: ==========
00:35:51.739  00:14:53.913  
00:35:51.739  00:14:53.913  in /usr/home/vagrant/spdk_repo/spdk/test/common/autobuild_common.sh:403 -> _unittest_build([])
00:35:51.739  00:14:53.913       ...
00:35:51.739  00:14:53.913     398 		esac
00:35:51.739  00:14:53.913     399 	}
00:35:51.739  00:14:53.913     400 	
00:35:51.739  00:14:53.913     401 	_unittest_build() {
00:35:51.739  00:14:53.913     402 		"$rootdir/configure" $config_params --without-shared
00:35:51.739  00:14:53.913  => 403 		$MAKE $MAKEFLAGS
00:35:51.739  00:14:53.913     404 	}
00:35:51.739  00:14:53.913     405 	
00:35:51.739  00:14:53.913     406 	autobuild_test_suite() {
00:35:51.739  00:14:53.913     407 		run_test "autobuild" _autobuild_test_suite
00:35:51.739  00:14:53.913     408 	}
00:35:51.739  00:14:53.913       ...
00:35:51.739  00:14:53.913  in /usr/home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["unittest_build"],["_unittest_build"])
00:35:51.739  00:14:53.913       ...
00:35:51.739  00:14:53.913     1106		timing_enter $test_name
00:35:51.739  00:14:53.913     1107		echo "************************************"
00:35:51.739  00:14:53.913     1108		echo "START TEST $test_name"
00:35:51.739  00:14:53.913     1109		echo "************************************"
00:35:51.739  00:14:53.913     1110		xtrace_restore
00:35:51.739  00:14:53.913     1111		time "$@"
00:35:51.739  00:14:53.913     1112		xtrace_disable
00:35:51.739  00:14:53.913     1113		echo "************************************"
00:35:51.739  00:14:53.913     1114		echo "END TEST $test_name"
00:35:51.739  00:14:53.913     1115		echo "************************************"
00:35:51.739  00:14:53.913     1116		timing_exit $test_name
00:35:51.739  00:14:53.913       ...
00:35:51.739  00:14:53.913  in /usr/home/vagrant/spdk_repo/spdk/test/common/autobuild_common.sh:411 -> unittest_build([])
00:35:51.739  00:14:53.913       ...
00:35:51.739  00:14:53.913     406 	autobuild_test_suite() {
00:35:51.739  00:14:53.913     407 		run_test "autobuild" _autobuild_test_suite
00:35:51.739  00:14:53.913     408 	}
00:35:51.739  00:14:53.913     409 	
00:35:51.739  00:14:53.913     410 	unittest_build() {
00:35:51.739  00:14:53.913  => 411 		run_test "unittest_build" _unittest_build
00:35:51.739  00:14:53.913     412 	}
00:35:51.739  00:14:53.913     413 	
00:35:51.739  00:14:53.913     414 	scanbuild_make() {
00:35:51.739  00:14:53.913     415 		run_test "scanbuild_make" _scanbuild_make
00:35:51.739  00:14:53.913     416 	}
00:35:51.739  00:14:53.913       ...
00:35:51.739  00:14:53.913  in /usr/home/vagrant/spdk_repo/spdk/autobuild.sh:58 -> main(["/usr/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.739  00:14:53.913       ...
00:35:51.739  00:14:53.913     53  	fi
00:35:51.739  00:14:53.913     54  	
00:35:51.739  00:14:53.913     55  	if [[ -n $SPDK_TEST_AUTOBUILD ]]; then
00:35:51.739  00:14:53.913     56  		autobuild_test_suite
00:35:51.739  00:14:53.914     57  	elif [[ $SPDK_TEST_UNITTEST -eq 1 ]]; then
00:35:51.739  00:14:53.914  => 58  		unittest_build
00:35:51.739  00:14:53.914     59  	elif [[ $SPDK_TEST_SCANBUILD -eq 1 ]]; then
00:35:51.739  00:14:53.914     60  		scanbuild_make
00:35:51.739  00:14:53.914     61  	else
00:35:51.739  00:14:53.914     62  		if [[ $SPDK_TEST_FUZZER -eq 1 ]]; then
00:35:51.739  00:14:53.914     63  			# if we are testing nvmf fuzz with llvm lib, --with-shared will cause lib link fail
00:35:51.739  00:14:53.914       ...
00:35:51.739  00:14:53.914  
00:35:51.739  00:14:53.914  ========== Backtrace end =========='
00:35:51.739  + [[ -n 00:14:53.913  ========== Backtrace start: ==========
00:35:51.739  00:14:53.913  
00:35:51.739  00:14:53.913  in /usr/home/vagrant/spdk_repo/spdk/test/common/autobuild_common.sh:403 -> _unittest_build([])
00:35:51.739  00:14:53.913       ...
00:35:51.739  00:14:53.913     398 		esac
00:35:51.739  00:14:53.913     399 	}
00:35:51.739  00:14:53.913     400 	
00:35:51.739  00:14:53.913     401 	_unittest_build() {
00:35:51.739  00:14:53.913     402 		"$rootdir/configure" $config_params --without-shared
00:35:51.739  00:14:53.913  => 403 		$MAKE $MAKEFLAGS
00:35:51.739  00:14:53.913     404 	}
00:35:51.739  00:14:53.913     405 	
00:35:51.739  00:14:53.913     406 	autobuild_test_suite() {
00:35:51.739  00:14:53.913     407 		run_test "autobuild" _autobuild_test_suite
00:35:51.739  00:14:53.913     408 	}
00:35:51.739  00:14:53.913       ...
00:35:51.739  00:14:53.913  in /usr/home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["unittest_build"],["_unittest_build"])
00:35:51.739  00:14:53.913       ...
00:35:51.739  00:14:53.913     1106		timing_enter $test_name
00:35:51.739  00:14:53.913     1107		echo "************************************"
00:35:51.739  00:14:53.913     1108		echo "START TEST $test_name"
00:35:51.739  00:14:53.913     1109		echo "************************************"
00:35:51.739  00:14:53.913     1110		xtrace_restore
00:35:51.739  00:14:53.913     1111		time "$@"
00:35:51.739  00:14:53.913     1112		xtrace_disable
00:35:51.739  00:14:53.913     1113		echo "************************************"
00:35:51.739  00:14:53.913     1114		echo "END TEST $test_name"
00:35:51.739  00:14:53.913     1115		echo "************************************"
00:35:51.739  00:14:53.913     1116		timing_exit $test_name
00:35:51.739  00:14:53.913       ...
00:35:51.739  00:14:53.913  in /usr/home/vagrant/spdk_repo/spdk/test/common/autobuild_common.sh:411 -> unittest_build([])
00:35:51.739  00:14:53.913       ...
00:35:51.739  00:14:53.913     406 	autobuild_test_suite() {
00:35:51.739  00:14:53.913     407 		run_test "autobuild" _autobuild_test_suite
00:35:51.739  00:14:53.913     408 	}
00:35:51.739  00:14:53.913     409 	
00:35:51.739  00:14:53.913     410 	unittest_build() {
00:35:51.739  00:14:53.913  => 411 		run_test "unittest_build" _unittest_build
00:35:51.739  00:14:53.913     412 	}
00:35:51.739  00:14:53.913     413 	
00:35:51.740  00:14:53.913     414 	scanbuild_make() {
00:35:51.740  00:14:53.913     415 		run_test "scanbuild_make" _scanbuild_make
00:35:51.740  00:14:53.913     416 	}
00:35:51.740  00:14:53.913       ...
00:35:51.740  00:14:53.913  in /usr/home/vagrant/spdk_repo/spdk/autobuild.sh:58 -> main(["/usr/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.740  00:14:53.913       ...
00:35:51.740  00:14:53.913     53  	fi
00:35:51.740  00:14:53.913     54  	
00:35:51.740  00:14:53.913     55  	if [[ -n $SPDK_TEST_AUTOBUILD ]]; then
00:35:51.740  00:14:53.913     56  		autobuild_test_suite
00:35:51.740  00:14:53.914     57  	elif [[ $SPDK_TEST_UNITTEST -eq 1 ]]; then
00:35:51.740  00:14:53.914  => 58  		unittest_build
00:35:51.740  00:14:53.914     59  	elif [[ $SPDK_TEST_SCANBUILD -eq 1 ]]; then
00:35:51.740  00:14:53.914     60  		scanbuild_make
00:35:51.740  00:14:53.914     61  	else
00:35:51.740  00:14:53.914     62  		if [[ $SPDK_TEST_FUZZER -eq 1 ]]; then
00:35:51.740  00:14:53.914     63  			# if we are testing nvmf fuzz with llvm lib, --with-shared will cause lib link fail
00:35:51.740  00:14:53.914       ...
00:35:51.740  00:14:53.914  
00:35:51.740  00:14:53.914  ========== Backtrace end ========== ]]
00:35:51.740  + echo '00:14:53.913  ========== Backtrace start: ==========
00:35:51.740  00:14:53.913  
00:35:51.740  00:14:53.913  in /usr/home/vagrant/spdk_repo/spdk/test/common/autobuild_common.sh:403 -> _unittest_build([])
00:35:51.740  00:14:53.913       ...
00:35:51.740  00:14:53.913     398 		esac
00:35:51.740  00:14:53.913     399 	}
00:35:51.740  00:14:53.913     400 	
00:35:51.740  00:14:53.913     401 	_unittest_build() {
00:35:51.740  00:14:53.913     402 		"$rootdir/configure" $config_params --without-shared
00:35:51.740  00:14:53.913  => 403 		$MAKE $MAKEFLAGS
00:35:51.740  00:14:53.913     404 	}
00:35:51.740  00:14:53.913     405 	
00:35:51.740  00:14:53.913     406 	autobuild_test_suite() {
00:35:51.740  00:14:53.913     407 		run_test "autobuild" _autobuild_test_suite
00:35:51.740  00:14:53.913     408 	}
00:35:51.740  00:14:53.913       ...
00:35:51.740  00:14:53.913  in /usr/home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["unittest_build"],["_unittest_build"])
00:35:51.740  00:14:53.913       ...
00:35:51.740  00:14:53.913     1106		timing_enter $test_name
00:35:51.740  00:14:53.913     1107		echo "************************************"
00:35:51.740  00:14:53.913     1108		echo "START TEST $test_name"
00:35:51.740  00:14:53.913     1109		echo "************************************"
00:35:51.740  00:14:53.913     1110		xtrace_restore
00:35:51.740  00:14:53.913     1111		time "$@"
00:35:51.740  00:14:53.913     1112		xtrace_disable
00:35:51.740  00:14:53.913     1113		echo "************************************"
00:35:51.740  00:14:53.913     1114		echo "END TEST $test_name"
00:35:51.740  00:14:53.913     1115		echo "************************************"
00:35:51.740  00:14:53.913     1116		timing_exit $test_name
00:35:51.740  00:14:53.913       ...
00:35:51.740  00:14:53.913  in /usr/home/vagrant/spdk_repo/spdk/test/common/autobuild_common.sh:411 -> unittest_build([])
00:35:51.740  00:14:53.913       ...
00:35:51.740  00:14:53.913     406 	autobuild_test_suite() {
00:35:51.740  00:14:53.913     407 		run_test "autobuild" _autobuild_test_suite
00:35:51.740  00:14:53.913     408 	}
00:35:51.740  00:14:53.913     409 	
00:35:51.740  00:14:53.913     410 	unittest_build() {
00:35:51.740  00:14:53.913  => 411 		run_test "unittest_build" _unittest_build
00:35:51.740  00:14:53.913     412 	}
00:35:51.740  00:14:53.913     413 	
00:35:51.740  00:14:53.913     414 	scanbuild_make() {
00:35:51.740  00:14:53.913     415 		run_test "scanbuild_make" _scanbuild_make
00:35:51.740  00:14:53.913     416 	}
00:35:51.740  00:14:53.913       ...
00:35:51.740  00:14:53.913  in /usr/home/vagrant/spdk_repo/spdk/autobuild.sh:58 -> main(["/usr/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.740  00:14:53.913       ...
00:35:51.740  00:14:53.913     53  	fi
00:35:51.740  00:14:53.913     54  	
00:35:51.740  00:14:53.913     55  	if [[ -n $SPDK_TEST_AUTOBUILD ]]; then
00:35:51.740  00:14:53.913     56  		autobuild_test_suite
00:35:51.740  00:14:53.914     57  	elif [[ $SPDK_TEST_UNITTEST -eq 1 ]]; then
00:35:51.740  00:14:53.914  => 58  		unittest_build
00:35:51.740  00:14:53.914     59  	elif [[ $SPDK_TEST_SCANBUILD -eq 1 ]]; then
00:35:51.740  00:14:53.914     60  		scanbuild_make
00:35:51.740  00:14:53.914     61  	else
00:35:51.740  00:14:53.914     62  		if [[ $SPDK_TEST_FUZZER -eq 1 ]]; then
00:35:51.740  00:14:53.914     63  			# if we are testing nvmf fuzz with llvm lib, --with-shared will cause lib link fail
00:35:51.740  00:14:53.914       ...
00:35:51.740  00:14:53.914  
00:35:51.740  00:14:53.914  ========== Backtrace end =========='
00:35:51.740  + for build in "${build_log_list[@]}"
00:35:51.740  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_74542/build.log
00:35:51.740  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.740  + build_log=/var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_74542/build.log
00:35:51.740  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_74542/build.log
00:35:51.740  ++ basename /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_74542
00:35:51.740  + job_name=lvol-vg-autotest_74542
00:35:51.740  + start_pattern='Backtrace start'
00:35:51.740  + end_pattern='Backtrace end'
00:35:51.740  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_74542/build.log ]]
00:35:51.740  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_74542/build.log
00:35:51.740  + backtraces='00:15:01.787  ========== Backtrace start: ==========
00:35:51.740  00:15:01.787  
00:35:51.740  00:15:01.787  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:960 -> killprocess(["60209"])
00:35:51.740  00:15:01.787       ...
00:35:51.740  00:15:01.787     955 				kill $1
00:35:51.740  00:15:01.787     956 			fi
00:35:51.740  00:15:01.787     957 	
00:35:51.740  00:15:01.787     958 			# wait for the process regardless if its the dummy sudo one
00:35:51.740  00:15:01.787     959 			# or the actual app - it should terminate anyway
00:35:51.740  00:15:01.787  => 960 			wait $1
00:35:51.740  00:15:01.787     961 		else
00:35:51.740  00:15:01.787     962 			# the process is not there anymore
00:35:51.740  00:15:01.787     963 			echo "Process with pid $1 is not found"
00:35:51.740  00:15:01.787     964 		fi
00:35:51.740  00:15:01.787     965 	}
00:35:51.740  00:15:01.787       ...
00:35:51.740  00:15:01.787  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:51.740  00:15:01.787       ...
00:35:51.740  00:15:01.787     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:51.740  00:15:01.787     35  			sleep 3
00:35:51.740  00:15:01.787     36  		done
00:35:51.740  00:15:01.787     37  	
00:35:51.740  00:15:01.787     38  		waitforlisten $repeat_pid $rpc_server
00:35:51.740  00:15:01.787  => 39  		killprocess $repeat_pid
00:35:51.740  00:15:01.787     40  		trap - SIGINT SIGTERM EXIT
00:35:51.740  00:15:01.787     41  	
00:35:51.740  00:15:01.787     42  		return 0
00:35:51.740  00:15:01.787     43  	}
00:35:51.740  00:15:01.787     44  	
00:35:51.740  00:15:01.787       ...
00:35:51.740  00:15:01.787  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:51.740  00:15:01.787       ...
00:35:51.740  00:15:01.787     1106		timing_enter $test_name
00:35:51.740  00:15:01.787     1107		echo "************************************"
00:35:51.740  00:15:01.787     1108		echo "START TEST $test_name"
00:35:51.740  00:15:01.787     1109		echo "************************************"
00:35:51.740  00:15:01.787     1110		xtrace_restore
00:35:51.740  00:15:01.787     1111		time "$@"
00:35:51.740  00:15:01.787     1112		xtrace_disable
00:35:51.740  00:15:01.787     1113		echo "************************************"
00:35:51.740  00:15:01.787     1114		echo "END TEST $test_name"
00:35:51.740  00:15:01.787     1115		echo "************************************"
00:35:51.740  00:15:01.787     1116		timing_exit $test_name
00:35:51.740  00:15:01.787       ...
00:35:51.740  00:15:01.787  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:52 -> main([])
00:35:51.740  00:15:01.787       ...
00:35:51.740  00:15:01.787     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:51.740  00:15:01.787     48  	
00:35:51.740  00:15:01.787     49  	if [ $(uname -s) = Linux ]; then
00:35:51.740  00:15:01.787     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:51.740  00:15:01.787     51  		if modprobe -n nbd; then
00:35:51.740  00:15:01.787  => 52  			run_test "app_repeat" app_repeat_test
00:35:51.740  00:15:01.787     53  		fi
00:35:51.740  00:15:01.787     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:51.740  00:15:01.787     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:51.740  00:15:01.787     56  		fi
00:35:51.740  00:15:01.787     57  	fi
00:35:51.740  00:15:01.787       ...
00:35:51.740  00:15:01.787  
00:35:51.740  00:15:01.787  ========== Backtrace end ==========
00:35:51.740  00:15:01.787  ========== Backtrace start: ==========
00:35:51.740  00:15:01.787  
00:35:51.740  00:15:01.787  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/home/vagrant/spdk_repo/spdk/test/event/event.sh"])
00:35:51.740  00:15:01.787       ...
00:35:51.740  00:15:01.787     1106		timing_enter $test_name
00:35:51.741  00:15:01.787     1107		echo "************************************"
00:35:51.741  00:15:01.787     1108		echo "START TEST $test_name"
00:35:51.741  00:15:01.787     1109		echo "************************************"
00:35:51.741  00:15:01.787     1110		xtrace_restore
00:35:51.741  00:15:01.787     1111		time "$@"
00:35:51.741  00:15:01.787     1112		xtrace_disable
00:35:51.741  00:15:01.787     1113		echo "************************************"
00:35:51.741  00:15:01.787     1114		echo "END TEST $test_name"
00:35:51.741  00:15:01.787     1115		echo "************************************"
00:35:51.741  00:15:01.787     1116		timing_exit $test_name
00:35:51.741  00:15:01.787       ...
00:35:51.741  00:15:01.787  in /home/vagrant/spdk_repo/spdk/autotest.sh:177 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.741  00:15:01.787       ...
00:35:51.741  00:15:01.787     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:51.741  00:15:01.787     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:51.741  00:15:01.787     174 		fi
00:35:51.741  00:15:01.787     175 	
00:35:51.741  00:15:01.787     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:51.741  00:15:01.787  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:51.741  00:15:01.787     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:51.741  00:15:01.787     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:51.741  00:15:01.787     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:51.741  00:15:01.787     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:51.741  00:15:01.788     182 		run_test "version" $rootdir/test/app/version.sh
00:35:51.741  00:15:01.788       ...
00:35:51.741  00:15:01.788  
00:35:51.741  00:15:01.788  ========== Backtrace end =========='
00:35:51.741  + [[ -n 00:15:01.787  ========== Backtrace start: ==========
00:35:51.741  00:15:01.787  
00:35:51.741  00:15:01.787  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:960 -> killprocess(["60209"])
00:35:51.741  00:15:01.787       ...
00:35:51.741  00:15:01.787     955 				kill $1
00:35:51.741  00:15:01.787     956 			fi
00:35:51.741  00:15:01.787     957 	
00:35:51.741  00:15:01.787     958 			# wait for the process regardless if its the dummy sudo one
00:35:51.741  00:15:01.787     959 			# or the actual app - it should terminate anyway
00:35:51.741  00:15:01.787  => 960 			wait $1
00:35:51.741  00:15:01.787     961 		else
00:35:51.741  00:15:01.787     962 			# the process is not there anymore
00:35:51.741  00:15:01.787     963 			echo "Process with pid $1 is not found"
00:35:51.741  00:15:01.787     964 		fi
00:35:51.741  00:15:01.787     965 	}
00:35:51.741  00:15:01.787       ...
00:35:51.741  00:15:01.787  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:51.741  00:15:01.787       ...
00:35:51.741  00:15:01.787     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:51.741  00:15:01.787     35  			sleep 3
00:35:51.741  00:15:01.787     36  		done
00:35:51.741  00:15:01.787     37  	
00:35:51.741  00:15:01.787     38  		waitforlisten $repeat_pid $rpc_server
00:35:51.741  00:15:01.787  => 39  		killprocess $repeat_pid
00:35:51.741  00:15:01.787     40  		trap - SIGINT SIGTERM EXIT
00:35:51.741  00:15:01.787     41  	
00:35:51.741  00:15:01.787     42  		return 0
00:35:51.741  00:15:01.787     43  	}
00:35:51.741  00:15:01.787     44  	
00:35:51.741  00:15:01.787       ...
00:35:51.741  00:15:01.787  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:51.741  00:15:01.787       ...
00:35:51.741  00:15:01.787     1106		timing_enter $test_name
00:35:51.741  00:15:01.787     1107		echo "************************************"
00:35:51.741  00:15:01.787     1108		echo "START TEST $test_name"
00:35:51.741  00:15:01.787     1109		echo "************************************"
00:35:51.741  00:15:01.787     1110		xtrace_restore
00:35:51.741  00:15:01.787     1111		time "$@"
00:35:51.741  00:15:01.787     1112		xtrace_disable
00:35:51.741  00:15:01.787     1113		echo "************************************"
00:35:51.741  00:15:01.787     1114		echo "END TEST $test_name"
00:35:51.741  00:15:01.787     1115		echo "************************************"
00:35:51.741  00:15:01.787     1116		timing_exit $test_name
00:35:51.741  00:15:01.787       ...
00:35:51.741  00:15:01.787  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:52 -> main([])
00:35:51.741  00:15:01.787       ...
00:35:51.741  00:15:01.787     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:51.741  00:15:01.787     48  	
00:35:51.741  00:15:01.787     49  	if [ $(uname -s) = Linux ]; then
00:35:51.741  00:15:01.787     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:51.741  00:15:01.787     51  		if modprobe -n nbd; then
00:35:51.741  00:15:01.787  => 52  			run_test "app_repeat" app_repeat_test
00:35:51.741  00:15:01.787     53  		fi
00:35:51.741  00:15:01.787     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:51.741  00:15:01.787     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:51.741  00:15:01.787     56  		fi
00:35:51.741  00:15:01.787     57  	fi
00:35:51.741  00:15:01.787       ...
00:35:51.741  00:15:01.787  
00:35:51.741  00:15:01.787  ========== Backtrace end ==========
00:35:51.741  00:15:01.787  ========== Backtrace start: ==========
00:35:51.741  00:15:01.787  
00:35:51.741  00:15:01.787  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/home/vagrant/spdk_repo/spdk/test/event/event.sh"])
00:35:51.741  00:15:01.787       ...
00:35:51.741  00:15:01.787     1106		timing_enter $test_name
00:35:51.741  00:15:01.787     1107		echo "************************************"
00:35:51.741  00:15:01.787     1108		echo "START TEST $test_name"
00:35:51.741  00:15:01.787     1109		echo "************************************"
00:35:51.741  00:15:01.787     1110		xtrace_restore
00:35:51.741  00:15:01.787     1111		time "$@"
00:35:51.741  00:15:01.787     1112		xtrace_disable
00:35:51.741  00:15:01.787     1113		echo "************************************"
00:35:51.741  00:15:01.787     1114		echo "END TEST $test_name"
00:35:51.741  00:15:01.787     1115		echo "************************************"
00:35:51.741  00:15:01.787     1116		timing_exit $test_name
00:35:51.741  00:15:01.787       ...
00:35:51.741  00:15:01.787  in /home/vagrant/spdk_repo/spdk/autotest.sh:177 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.741  00:15:01.787       ...
00:35:51.741  00:15:01.787     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:51.741  00:15:01.787     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:51.741  00:15:01.787     174 		fi
00:35:51.741  00:15:01.787     175 	
00:35:51.741  00:15:01.787     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:51.741  00:15:01.787  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:51.741  00:15:01.787     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:51.741  00:15:01.787     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:51.741  00:15:01.787     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:51.741  00:15:01.787     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:51.741  00:15:01.788     182 		run_test "version" $rootdir/test/app/version.sh
00:35:51.741  00:15:01.788       ...
00:35:51.741  00:15:01.788  
00:35:51.741  00:15:01.788  ========== Backtrace end ========== ]]
00:35:51.741  + echo '00:15:01.787  ========== Backtrace start: ==========
00:35:51.741  00:15:01.787  
00:35:51.741  00:15:01.787  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:960 -> killprocess(["60209"])
00:35:51.741  00:15:01.787       ...
00:35:51.741  00:15:01.787     955 				kill $1
00:35:51.741  00:15:01.787     956 			fi
00:35:51.741  00:15:01.787     957 	
00:35:51.742  00:15:01.787     958 			# wait for the process regardless if its the dummy sudo one
00:35:51.742  00:15:01.787     959 			# or the actual app - it should terminate anyway
00:35:51.742  00:15:01.787  => 960 			wait $1
00:35:51.742  00:15:01.787     961 		else
00:35:51.742  00:15:01.787     962 			# the process is not there anymore
00:35:51.742  00:15:01.787     963 			echo "Process with pid $1 is not found"
00:35:51.742  00:15:01.787     964 		fi
00:35:51.742  00:15:01.787     965 	}
00:35:51.742  00:15:01.787       ...
00:35:51.742  00:15:01.787  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:51.742  00:15:01.787       ...
00:35:51.742  00:15:01.787     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:51.742  00:15:01.787     35  			sleep 3
00:35:51.742  00:15:01.787     36  		done
00:35:51.742  00:15:01.787     37  	
00:35:51.742  00:15:01.787     38  		waitforlisten $repeat_pid $rpc_server
00:35:51.742  00:15:01.787  => 39  		killprocess $repeat_pid
00:35:51.742  00:15:01.787     40  		trap - SIGINT SIGTERM EXIT
00:35:51.742  00:15:01.787     41  	
00:35:51.742  00:15:01.787     42  		return 0
00:35:51.742  00:15:01.787     43  	}
00:35:51.742  00:15:01.787     44  	
00:35:51.742  00:15:01.787       ...
00:35:51.742  00:15:01.787  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:51.742  00:15:01.787       ...
00:35:51.742  00:15:01.787     1106		timing_enter $test_name
00:35:51.742  00:15:01.787     1107		echo "************************************"
00:35:51.742  00:15:01.787     1108		echo "START TEST $test_name"
00:35:51.742  00:15:01.787     1109		echo "************************************"
00:35:51.742  00:15:01.787     1110		xtrace_restore
00:35:51.742  00:15:01.787     1111		time "$@"
00:35:51.742  00:15:01.787     1112		xtrace_disable
00:35:51.742  00:15:01.787     1113		echo "************************************"
00:35:51.742  00:15:01.787     1114		echo "END TEST $test_name"
00:35:51.742  00:15:01.787     1115		echo "************************************"
00:35:51.742  00:15:01.787     1116		timing_exit $test_name
00:35:51.742  00:15:01.787       ...
00:35:51.742  00:15:01.787  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:52 -> main([])
00:35:51.742  00:15:01.787       ...
00:35:51.742  00:15:01.787     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:51.742  00:15:01.787     48  	
00:35:51.742  00:15:01.787     49  	if [ $(uname -s) = Linux ]; then
00:35:51.742  00:15:01.787     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:51.742  00:15:01.787     51  		if modprobe -n nbd; then
00:35:51.742  00:15:01.787  => 52  			run_test "app_repeat" app_repeat_test
00:35:51.742  00:15:01.787     53  		fi
00:35:51.742  00:15:01.787     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:51.742  00:15:01.787     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:51.742  00:15:01.787     56  		fi
00:35:51.742  00:15:01.787     57  	fi
00:35:51.742  00:15:01.787       ...
00:35:51.742  00:15:01.787  
00:35:51.742  00:15:01.787  ========== Backtrace end ==========
00:35:51.742  00:15:01.787  ========== Backtrace start: ==========
00:35:51.742  00:15:01.787  
00:35:51.742  00:15:01.787  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/home/vagrant/spdk_repo/spdk/test/event/event.sh"])
00:35:51.742  00:15:01.787       ...
00:35:51.742  00:15:01.787     1106		timing_enter $test_name
00:35:51.742  00:15:01.787     1107		echo "************************************"
00:35:51.742  00:15:01.787     1108		echo "START TEST $test_name"
00:35:51.742  00:15:01.787     1109		echo "************************************"
00:35:51.742  00:15:01.787     1110		xtrace_restore
00:35:51.742  00:15:01.787     1111		time "$@"
00:35:51.742  00:15:01.787     1112		xtrace_disable
00:35:51.742  00:15:01.787     1113		echo "************************************"
00:35:51.742  00:15:01.787     1114		echo "END TEST $test_name"
00:35:51.742  00:15:01.787     1115		echo "************************************"
00:35:51.742  00:15:01.787     1116		timing_exit $test_name
00:35:51.742  00:15:01.787       ...
00:35:51.742  00:15:01.787  in /home/vagrant/spdk_repo/spdk/autotest.sh:177 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:51.742  00:15:01.787       ...
00:35:51.742  00:15:01.787     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:51.742  00:15:01.787     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:51.742  00:15:01.787     174 		fi
00:35:51.742  00:15:01.787     175 	
00:35:51.742  00:15:01.787     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:51.742  00:15:01.787  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:51.742  00:15:01.787     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:51.742  00:15:01.787     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:51.742  00:15:01.787     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:51.742  00:15:01.787     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:51.742  00:15:01.788     182 		run_test "version" $rootdir/test/app/version.sh
00:35:51.742  00:15:01.788       ...
00:35:51.742  00:15:01.788  
00:35:51.742  00:15:01.788  ========== Backtrace end =========='
00:35:51.742  + for build in "${build_log_list[@]}"
00:35:51.742  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_71090/build.log
00:35:51.742  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.742  + build_log=/var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_71090/build.log
00:35:51.742  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_71090/build.log
00:35:51.742  ++ basename /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_71090
00:35:51.742  + job_name=centos7-vg-autotest_71090
00:35:51.742  + start_pattern='Backtrace start'
00:35:51.742  + end_pattern='Backtrace end'
00:35:51.742  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_71090/build.log ]]
00:35:51.742  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_71090/build.log
00:35:51.742  + backtraces=
00:35:51.742  + [[ -n '' ]]
00:35:51.742  + for build in "${build_log_list[@]}"
00:35:51.742  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_69540/build.log
00:35:51.742  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.742  + build_log=/var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_69540/build.log
00:35:51.742  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_69540/build.log
00:35:51.742  ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_69540
00:35:51.742  + job_name=nvmf-tcp-phy-autotest_69540
00:35:51.742  + start_pattern='Backtrace start'
00:35:51.742  + end_pattern='Backtrace end'
00:35:51.742  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_69540/build.log ]]
00:35:51.742  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_69540/build.log
00:35:51.742  + backtraces=
00:35:51.742  + [[ -n '' ]]
00:35:51.742  + for build in "${build_log_list[@]}"
00:35:51.742  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_74515/build.log
00:35:51.742  + local build_log job_name start_pattern end_pattern backtraces
00:35:51.742  + build_log=/var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_74515/build.log
00:35:51.742  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_74515/build.log
00:35:51.742  ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_74515
00:35:51.742  + job_name=nvmf-tcp-uring-vg-autotest_74515
00:35:51.742  + start_pattern='Backtrace start'
00:35:51.742  + end_pattern='Backtrace end'
00:35:51.742  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_74515/build.log ]]
00:35:51.742  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_74515/build.log
00:35:52.008  + backtraces=
00:35:52.008  + [[ -n '' ]]
00:35:52.008  + for build in "${build_log_list[@]}"
00:35:52.008  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_35184/build.log
00:35:52.008  + local build_log job_name start_pattern end_pattern backtraces
00:35:52.008  + build_log=/var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_35184/build.log
00:35:52.008  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_35184/build.log
00:35:52.008  ++ basename /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_35184
00:35:52.008  + job_name=ubuntu22-vg-autotest_35184
00:35:52.008  + start_pattern='Backtrace start'
00:35:52.008  + end_pattern='Backtrace end'
00:35:52.008  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_35184/build.log ]]
00:35:52.008  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_35184/build.log
00:35:52.008  + backtraces='00:20:25.645  ========== Backtrace start: ==========
00:35:52.008  00:20:25.645  
00:35:52.008  00:20:25.645  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:960 -> killprocess(["112578"])
00:35:52.008  00:20:25.645       ...
00:35:52.008  00:20:25.645     955 				kill $1
00:35:52.008  00:20:25.645     956 			fi
00:35:52.008  00:20:25.645     957 	
00:35:52.008  00:20:25.645     958 			# wait for the process regardless if its the dummy sudo one
00:35:52.008  00:20:25.645     959 			# or the actual app - it should terminate anyway
00:35:52.008  00:20:25.645  => 960 			wait $1
00:35:52.008  00:20:25.645     961 		else
00:35:52.008  00:20:25.645     962 			# the process is not there anymore
00:35:52.008  00:20:25.645     963 			echo "Process with pid $1 is not found"
00:35:52.008  00:20:25.645     964 		fi
00:35:52.008  00:20:25.645     965 	}
00:35:52.008  00:20:25.645       ...
00:35:52.008  00:20:25.645  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:52.008  00:20:25.645       ...
00:35:52.008  00:20:25.645     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:52.008  00:20:25.645     35  			sleep 3
00:35:52.008  00:20:25.645     36  		done
00:35:52.008  00:20:25.645     37  	
00:35:52.008  00:20:25.645     38  		waitforlisten $repeat_pid $rpc_server
00:35:52.008  00:20:25.645  => 39  		killprocess $repeat_pid
00:35:52.008  00:20:25.645     40  		trap - SIGINT SIGTERM EXIT
00:35:52.008  00:20:25.645     41  	
00:35:52.008  00:20:25.645     42  		return 0
00:35:52.008  00:20:25.645     43  	}
00:35:52.008  00:20:25.645     44  	
00:35:52.008  00:20:25.645       ...
00:35:52.008  00:20:25.645  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:52.008  00:20:25.645       ...
00:35:52.008  00:20:25.645     1106		timing_enter $test_name
00:35:52.008  00:20:25.645     1107		echo "************************************"
00:35:52.008  00:20:25.645     1108		echo "START TEST $test_name"
00:35:52.008  00:20:25.645     1109		echo "************************************"
00:35:52.008  00:20:25.645     1110		xtrace_restore
00:35:52.008  00:20:25.645     1111		time "$@"
00:35:52.008  00:20:25.645     1112		xtrace_disable
00:35:52.008  00:20:25.645     1113		echo "************************************"
00:35:52.008  00:20:25.645     1114		echo "END TEST $test_name"
00:35:52.008  00:20:25.645     1115		echo "************************************"
00:35:52.008  00:20:25.645     1116		timing_exit $test_name
00:35:52.008  00:20:25.645       ...
00:35:52.008  00:20:25.645  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:52 -> main([])
00:35:52.009  00:20:25.645       ...
00:35:52.009  00:20:25.645     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:52.009  00:20:25.645     48  	
00:35:52.009  00:20:25.645     49  	if [ $(uname -s) = Linux ]; then
00:35:52.009  00:20:25.645     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:52.009  00:20:25.645     51  		if modprobe -n nbd; then
00:35:52.009  00:20:25.645  => 52  			run_test "app_repeat" app_repeat_test
00:35:52.009  00:20:25.645     53  		fi
00:35:52.009  00:20:25.645     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:52.009  00:20:25.645     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:52.009  00:20:25.645     56  		fi
00:35:52.009  00:20:25.645     57  	fi
00:35:52.009  00:20:25.645       ...
00:35:52.009  00:20:25.645  
00:35:52.009  00:20:25.645  ========== Backtrace end ==========
00:35:52.009  00:20:25.645  ========== Backtrace start: ==========
00:35:52.009  00:20:25.645  
00:35:52.009  00:20:25.645  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/home/vagrant/spdk_repo/spdk/test/event/event.sh"])
00:35:52.009  00:20:25.645       ...
00:35:52.009  00:20:25.645     1106		timing_enter $test_name
00:35:52.009  00:20:25.645     1107		echo "************************************"
00:35:52.009  00:20:25.645     1108		echo "START TEST $test_name"
00:35:52.009  00:20:25.645     1109		echo "************************************"
00:35:52.009  00:20:25.645     1110		xtrace_restore
00:35:52.009  00:20:25.645     1111		time "$@"
00:35:52.009  00:20:25.645     1112		xtrace_disable
00:35:52.009  00:20:25.645     1113		echo "************************************"
00:35:52.009  00:20:25.645     1114		echo "END TEST $test_name"
00:35:52.009  00:20:25.645     1115		echo "************************************"
00:35:52.009  00:20:25.645     1116		timing_exit $test_name
00:35:52.009  00:20:25.645       ...
00:35:52.009  00:20:25.645  in /home/vagrant/spdk_repo/spdk/autotest.sh:177 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:52.009  00:20:25.645       ...
00:35:52.009  00:20:25.645     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:52.009  00:20:25.645     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:52.009  00:20:25.645     174 		fi
00:35:52.009  00:20:25.645     175 	
00:35:52.009  00:20:25.645     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:52.009  00:20:25.645  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:52.009  00:20:25.645     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:52.009  00:20:25.645     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:52.009  00:20:25.645     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:52.009  00:20:25.645     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:52.009  00:20:25.645     182 		run_test "version" $rootdir/test/app/version.sh
00:35:52.009  00:20:25.645       ...
00:35:52.009  00:20:25.645  
00:35:52.009  00:20:25.645  ========== Backtrace end =========='
00:35:52.009  + [[ -n 00:20:25.645  ========== Backtrace start: ==========
00:35:52.009  00:20:25.645  
00:35:52.009  00:20:25.645  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:960 -> killprocess(["112578"])
00:35:52.009  00:20:25.645       ...
00:35:52.009  00:20:25.645     955 				kill $1
00:35:52.009  00:20:25.645     956 			fi
00:35:52.009  00:20:25.645     957 	
00:35:52.009  00:20:25.645     958 			# wait for the process regardless if its the dummy sudo one
00:35:52.009  00:20:25.645     959 			# or the actual app - it should terminate anyway
00:35:52.009  00:20:25.645  => 960 			wait $1
00:35:52.009  00:20:25.645     961 		else
00:35:52.009  00:20:25.645     962 			# the process is not there anymore
00:35:52.009  00:20:25.645     963 			echo "Process with pid $1 is not found"
00:35:52.009  00:20:25.645     964 		fi
00:35:52.009  00:20:25.645     965 	}
00:35:52.009  00:20:25.645       ...
00:35:52.009  00:20:25.645  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:52.009  00:20:25.645       ...
00:35:52.009  00:20:25.645     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:52.009  00:20:25.645     35  			sleep 3
00:35:52.009  00:20:25.645     36  		done
00:35:52.009  00:20:25.645     37  	
00:35:52.009  00:20:25.645     38  		waitforlisten $repeat_pid $rpc_server
00:35:52.009  00:20:25.645  => 39  		killprocess $repeat_pid
00:35:52.009  00:20:25.645     40  		trap - SIGINT SIGTERM EXIT
00:35:52.009  00:20:25.645     41  	
00:35:52.009  00:20:25.645     42  		return 0
00:35:52.009  00:20:25.645     43  	}
00:35:52.009  00:20:25.645     44  	
00:35:52.009  00:20:25.645       ...
00:35:52.009  00:20:25.645  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:52.009  00:20:25.645       ...
00:35:52.009  00:20:25.645     1106		timing_enter $test_name
00:35:52.009  00:20:25.645     1107		echo "************************************"
00:35:52.009  00:20:25.645     1108		echo "START TEST $test_name"
00:35:52.009  00:20:25.645     1109		echo "************************************"
00:35:52.009  00:20:25.645     1110		xtrace_restore
00:35:52.009  00:20:25.645     1111		time "$@"
00:35:52.009  00:20:25.645     1112		xtrace_disable
00:35:52.009  00:20:25.645     1113		echo "************************************"
00:35:52.009  00:20:25.645     1114		echo "END TEST $test_name"
00:35:52.009  00:20:25.645     1115		echo "************************************"
00:35:52.009  00:20:25.645     1116		timing_exit $test_name
00:35:52.009  00:20:25.645       ...
00:35:52.009  00:20:25.645  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:52 -> main([])
00:35:52.009  00:20:25.645       ...
00:35:52.009  00:20:25.645     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:52.009  00:20:25.645     48  	
00:35:52.009  00:20:25.645     49  	if [ $(uname -s) = Linux ]; then
00:35:52.009  00:20:25.645     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:52.009  00:20:25.645     51  		if modprobe -n nbd; then
00:35:52.009  00:20:25.645  => 52  			run_test "app_repeat" app_repeat_test
00:35:52.009  00:20:25.645     53  		fi
00:35:52.009  00:20:25.645     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:52.009  00:20:25.645     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:52.009  00:20:25.645     56  		fi
00:35:52.009  00:20:25.645     57  	fi
00:35:52.009  00:20:25.645       ...
00:35:52.009  00:20:25.645  
00:35:52.009  00:20:25.645  ========== Backtrace end ==========
00:35:52.009  00:20:25.645  ========== Backtrace start: ==========
00:35:52.009  00:20:25.645  
00:35:52.009  00:20:25.645  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/home/vagrant/spdk_repo/spdk/test/event/event.sh"])
00:35:52.009  00:20:25.645       ...
00:35:52.009  00:20:25.645     1106		timing_enter $test_name
00:35:52.009  00:20:25.645     1107		echo "************************************"
00:35:52.009  00:20:25.645     1108		echo "START TEST $test_name"
00:35:52.009  00:20:25.645     1109		echo "************************************"
00:35:52.009  00:20:25.645     1110		xtrace_restore
00:35:52.009  00:20:25.645     1111		time "$@"
00:35:52.009  00:20:25.645     1112		xtrace_disable
00:35:52.009  00:20:25.645     1113		echo "************************************"
00:35:52.009  00:20:25.645     1114		echo "END TEST $test_name"
00:35:52.009  00:20:25.645     1115		echo "************************************"
00:35:52.009  00:20:25.645     1116		timing_exit $test_name
00:35:52.009  00:20:25.645       ...
00:35:52.010  00:20:25.645  in /home/vagrant/spdk_repo/spdk/autotest.sh:177 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:52.010  00:20:25.645       ...
00:35:52.010  00:20:25.645     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:52.010  00:20:25.645     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:52.010  00:20:25.645     174 		fi
00:35:52.010  00:20:25.645     175 	
00:35:52.010  00:20:25.645     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:52.010  00:20:25.645  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:52.010  00:20:25.645     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:52.010  00:20:25.645     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:52.010  00:20:25.645     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:52.010  00:20:25.645     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:52.010  00:20:25.645     182 		run_test "version" $rootdir/test/app/version.sh
00:35:52.010  00:20:25.645       ...
00:35:52.010  00:20:25.645  
00:35:52.010  00:20:25.645  ========== Backtrace end ========== ]]
00:35:52.010  + echo '00:20:25.645  ========== Backtrace start: ==========
00:35:52.010  00:20:25.645  
00:35:52.010  00:20:25.645  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:960 -> killprocess(["112578"])
00:35:52.010  00:20:25.645       ...
00:35:52.010  00:20:25.645     955 				kill $1
00:35:52.010  00:20:25.645     956 			fi
00:35:52.010  00:20:25.645     957 	
00:35:52.010  00:20:25.645     958 			# wait for the process regardless if its the dummy sudo one
00:35:52.010  00:20:25.645     959 			# or the actual app - it should terminate anyway
00:35:52.010  00:20:25.645  => 960 			wait $1
00:35:52.010  00:20:25.645     961 		else
00:35:52.010  00:20:25.645     962 			# the process is not there anymore
00:35:52.010  00:20:25.645     963 			echo "Process with pid $1 is not found"
00:35:52.010  00:20:25.645     964 		fi
00:35:52.010  00:20:25.645     965 	}
00:35:52.010  00:20:25.645       ...
00:35:52.010  00:20:25.645  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:52.010  00:20:25.645       ...
00:35:52.010  00:20:25.645     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:52.010  00:20:25.645     35  			sleep 3
00:35:52.010  00:20:25.645     36  		done
00:35:52.010  00:20:25.645     37  	
00:35:52.010  00:20:25.645     38  		waitforlisten $repeat_pid $rpc_server
00:35:52.010  00:20:25.645  => 39  		killprocess $repeat_pid
00:35:52.010  00:20:25.645     40  		trap - SIGINT SIGTERM EXIT
00:35:52.010  00:20:25.645     41  	
00:35:52.010  00:20:25.645     42  		return 0
00:35:52.010  00:20:25.645     43  	}
00:35:52.010  00:20:25.645     44  	
00:35:52.010  00:20:25.645       ...
00:35:52.010  00:20:25.645  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:52.010  00:20:25.645       ...
00:35:52.010  00:20:25.645     1106		timing_enter $test_name
00:35:52.010  00:20:25.645     1107		echo "************************************"
00:35:52.010  00:20:25.645     1108		echo "START TEST $test_name"
00:35:52.010  00:20:25.645     1109		echo "************************************"
00:35:52.010  00:20:25.645     1110		xtrace_restore
00:35:52.010  00:20:25.645     1111		time "$@"
00:35:52.010  00:20:25.645     1112		xtrace_disable
00:35:52.010  00:20:25.645     1113		echo "************************************"
00:35:52.010  00:20:25.645     1114		echo "END TEST $test_name"
00:35:52.010  00:20:25.645     1115		echo "************************************"
00:35:52.010  00:20:25.645     1116		timing_exit $test_name
00:35:52.010  00:20:25.645       ...
00:35:52.010  00:20:25.645  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:52 -> main([])
00:35:52.010  00:20:25.645       ...
00:35:52.010  00:20:25.645     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:52.010  00:20:25.645     48  	
00:35:52.010  00:20:25.645     49  	if [ $(uname -s) = Linux ]; then
00:35:52.010  00:20:25.645     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:52.010  00:20:25.645     51  		if modprobe -n nbd; then
00:35:52.010  00:20:25.645  => 52  			run_test "app_repeat" app_repeat_test
00:35:52.010  00:20:25.645     53  		fi
00:35:52.010  00:20:25.645     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:52.010  00:20:25.645     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:52.010  00:20:25.645     56  		fi
00:35:52.010  00:20:25.645     57  	fi
00:35:52.010  00:20:25.645       ...
00:35:52.010  00:20:25.645  
00:35:52.010  00:20:25.645  ========== Backtrace end ==========
00:35:52.010  00:20:25.645  ========== Backtrace start: ==========
00:35:52.010  00:20:25.645  
00:35:52.010  00:20:25.645  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/home/vagrant/spdk_repo/spdk/test/event/event.sh"])
00:35:52.010  00:20:25.645       ...
00:35:52.010  00:20:25.645     1106		timing_enter $test_name
00:35:52.010  00:20:25.645     1107		echo "************************************"
00:35:52.010  00:20:25.645     1108		echo "START TEST $test_name"
00:35:52.010  00:20:25.645     1109		echo "************************************"
00:35:52.010  00:20:25.645     1110		xtrace_restore
00:35:52.010  00:20:25.645     1111		time "$@"
00:35:52.010  00:20:25.645     1112		xtrace_disable
00:35:52.010  00:20:25.645     1113		echo "************************************"
00:35:52.010  00:20:25.645     1114		echo "END TEST $test_name"
00:35:52.010  00:20:25.645     1115		echo "************************************"
00:35:52.010  00:20:25.645     1116		timing_exit $test_name
00:35:52.010  00:20:25.645       ...
00:35:52.010  00:20:25.645  in /home/vagrant/spdk_repo/spdk/autotest.sh:177 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:52.010  00:20:25.645       ...
00:35:52.010  00:20:25.645     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:52.010  00:20:25.645     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:52.010  00:20:25.645     174 		fi
00:35:52.010  00:20:25.645     175 	
00:35:52.010  00:20:25.645     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:52.010  00:20:25.645  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:52.010  00:20:25.645     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:52.010  00:20:25.645     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:52.010  00:20:25.645     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:52.010  00:20:25.645     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:52.010  00:20:25.645     182 		run_test "version" $rootdir/test/app/version.sh
00:35:52.010  00:20:25.645       ...
00:35:52.010  00:20:25.645  
00:35:52.010  00:20:25.645  ========== Backtrace end =========='
00:35:52.010  + for build in "${build_log_list[@]}"
00:35:52.010  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_17619/build.log
00:35:52.010  + local build_log job_name start_pattern end_pattern backtraces
00:35:52.010  + build_log=/var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_17619/build.log
00:35:52.010  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_17619/build.log
00:35:52.010  ++ basename /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_17619
00:35:52.010  + job_name=unittest-clang-docker-autotest_17619
00:35:52.010  + start_pattern='Backtrace start'
00:35:52.010  + end_pattern='Backtrace end'
00:35:52.010  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_17619/build.log ]]
00:35:52.010  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_17619/build.log
00:35:52.010  + backtraces='00:02:18.986  ========== Backtrace start: ==========
00:35:52.011  00:02:18.986  
00:35:52.011  00:02:18.986  in /var/jenkins/workspace/unittest-clang-docker-autotest/spdk/test/common/autobuild_common.sh:403 -> _unittest_build([])
00:35:52.011  00:02:18.986       ...
00:35:52.011  00:02:18.986     398 		esac
00:35:52.011  00:02:18.986     399 	}
00:35:52.011  00:02:18.986     400 	
00:35:52.011  00:02:18.986     401 	_unittest_build() {
00:35:52.011  00:02:18.986     402 		"$rootdir/configure" $config_params --without-shared
00:35:52.011  00:02:18.986  => 403 		$MAKE $MAKEFLAGS
00:35:52.011  00:02:18.986     404 	}
00:35:52.011  00:02:18.986     405 	
00:35:52.011  00:02:18.986     406 	autobuild_test_suite() {
00:35:52.011  00:02:18.986     407 		run_test "autobuild" _autobuild_test_suite
00:35:52.011  00:02:18.986     408 	}
00:35:52.011  00:02:18.986       ...
00:35:52.011  00:02:18.986  in /var/jenkins/workspace/unittest-clang-docker-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["unittest_build"],["_unittest_build"])
00:35:52.011  00:02:18.986       ...
00:35:52.011  00:02:18.986     1106		timing_enter $test_name
00:35:52.011  00:02:18.986     1107		echo "************************************"
00:35:52.011  00:02:18.986     1108		echo "START TEST $test_name"
00:35:52.011  00:02:18.986     1109		echo "************************************"
00:35:52.011  00:02:18.986     1110		xtrace_restore
00:35:52.011  00:02:18.987     1111		time "$@"
00:35:52.011  00:02:18.987     1112		xtrace_disable
00:35:52.011  00:02:18.987     1113		echo "************************************"
00:35:52.011  00:02:18.987     1114		echo "END TEST $test_name"
00:35:52.011  00:02:18.987     1115		echo "************************************"
00:35:52.011  00:02:18.987     1116		timing_exit $test_name
00:35:52.011  00:02:18.987       ...
00:35:52.011  00:02:18.987  in /var/jenkins/workspace/unittest-clang-docker-autotest/spdk/test/common/autobuild_common.sh:411 -> unittest_build([])
00:35:52.011  00:02:18.987       ...
00:35:52.011  00:02:18.987     406 	autobuild_test_suite() {
00:35:52.011  00:02:18.987     407 		run_test "autobuild" _autobuild_test_suite
00:35:52.011  00:02:18.987     408 	}
00:35:52.011  00:02:18.987     409 	
00:35:52.011  00:02:18.987     410 	unittest_build() {
00:35:52.011  00:02:18.987  => 411 		run_test "unittest_build" _unittest_build
00:35:52.011  00:02:18.987     412 	}
00:35:52.011  00:02:18.987     413 	
00:35:52.011  00:02:18.987     414 	scanbuild_make() {
00:35:52.011  00:02:18.987     415 		run_test "scanbuild_make" _scanbuild_make
00:35:52.011  00:02:18.987     416 	}
00:35:52.011  00:02:18.987       ...
00:35:52.011  00:02:18.987  in jbp/jenkins/jjb-config/jobs/scripts/docker-autoruner.sh:55 -> main([])
00:35:52.011  00:02:18.987       ...
00:35:52.011  00:02:18.987  backtrace unavailable
00:35:52.011  00:02:18.987       ...
00:35:52.011  00:02:18.987  
00:35:52.011  00:02:18.987  ========== Backtrace end =========='
00:35:52.011  + [[ -n 00:02:18.986  ========== Backtrace start: ==========
00:35:52.011  00:02:18.986  
00:35:52.011  00:02:18.986  in /var/jenkins/workspace/unittest-clang-docker-autotest/spdk/test/common/autobuild_common.sh:403 -> _unittest_build([])
00:35:52.011  00:02:18.986       ...
00:35:52.011  00:02:18.986     398 		esac
00:35:52.011  00:02:18.986     399 	}
00:35:52.011  00:02:18.986     400 	
00:35:52.011  00:02:18.986     401 	_unittest_build() {
00:35:52.011  00:02:18.986     402 		"$rootdir/configure" $config_params --without-shared
00:35:52.011  00:02:18.986  => 403 		$MAKE $MAKEFLAGS
00:35:52.011  00:02:18.986     404 	}
00:35:52.011  00:02:18.986     405 	
00:35:52.011  00:02:18.986     406 	autobuild_test_suite() {
00:35:52.011  00:02:18.986     407 		run_test "autobuild" _autobuild_test_suite
00:35:52.011  00:02:18.986     408 	}
00:35:52.011  00:02:18.986       ...
00:35:52.011  00:02:18.986  in /var/jenkins/workspace/unittest-clang-docker-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["unittest_build"],["_unittest_build"])
00:35:52.011  00:02:18.986       ...
00:35:52.011  00:02:18.986     1106		timing_enter $test_name
00:35:52.011  00:02:18.986     1107		echo "************************************"
00:35:52.011  00:02:18.986     1108		echo "START TEST $test_name"
00:35:52.011  00:02:18.986     1109		echo "************************************"
00:35:52.011  00:02:18.986     1110		xtrace_restore
00:35:52.011  00:02:18.987     1111		time "$@"
00:35:52.011  00:02:18.987     1112		xtrace_disable
00:35:52.011  00:02:18.987     1113		echo "************************************"
00:35:52.011  00:02:18.987     1114		echo "END TEST $test_name"
00:35:52.011  00:02:18.987     1115		echo "************************************"
00:35:52.011  00:02:18.987     1116		timing_exit $test_name
00:35:52.011  00:02:18.987       ...
00:35:52.011  00:02:18.987  in /var/jenkins/workspace/unittest-clang-docker-autotest/spdk/test/common/autobuild_common.sh:411 -> unittest_build([])
00:35:52.011  00:02:18.987       ...
00:35:52.011  00:02:18.987     406 	autobuild_test_suite() {
00:35:52.011  00:02:18.987     407 		run_test "autobuild" _autobuild_test_suite
00:35:52.011  00:02:18.987     408 	}
00:35:52.011  00:02:18.987     409 	
00:35:52.011  00:02:18.987     410 	unittest_build() {
00:35:52.011  00:02:18.987  => 411 		run_test "unittest_build" _unittest_build
00:35:52.011  00:02:18.987     412 	}
00:35:52.011  00:02:18.987     413 	
00:35:52.011  00:02:18.987     414 	scanbuild_make() {
00:35:52.011  00:02:18.987     415 		run_test "scanbuild_make" _scanbuild_make
00:35:52.011  00:02:18.987     416 	}
00:35:52.011  00:02:18.987       ...
00:35:52.011  00:02:18.987  in jbp/jenkins/jjb-config/jobs/scripts/docker-autoruner.sh:55 -> main([])
00:35:52.011  00:02:18.987       ...
00:35:52.011  00:02:18.987  backtrace unavailable
00:35:52.011  00:02:18.987       ...
00:35:52.011  00:02:18.987  
00:35:52.011  00:02:18.987  ========== Backtrace end ========== ]]
00:35:52.011  + echo '00:02:18.986  ========== Backtrace start: ==========
00:35:52.011  00:02:18.986  
00:35:52.011  00:02:18.986  in /var/jenkins/workspace/unittest-clang-docker-autotest/spdk/test/common/autobuild_common.sh:403 -> _unittest_build([])
00:35:52.011  00:02:18.986       ...
00:35:52.011  00:02:18.986     398 		esac
00:35:52.011  00:02:18.986     399 	}
00:35:52.011  00:02:18.986     400 	
00:35:52.011  00:02:18.986     401 	_unittest_build() {
00:35:52.011  00:02:18.986     402 		"$rootdir/configure" $config_params --without-shared
00:35:52.011  00:02:18.986  => 403 		$MAKE $MAKEFLAGS
00:35:52.011  00:02:18.986     404 	}
00:35:52.011  00:02:18.986     405 	
00:35:52.011  00:02:18.986     406 	autobuild_test_suite() {
00:35:52.011  00:02:18.986     407 		run_test "autobuild" _autobuild_test_suite
00:35:52.011  00:02:18.986     408 	}
00:35:52.011  00:02:18.986       ...
00:35:52.011  00:02:18.986  in /var/jenkins/workspace/unittest-clang-docker-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["unittest_build"],["_unittest_build"])
00:35:52.011  00:02:18.986       ...
00:35:52.011  00:02:18.986     1106		timing_enter $test_name
00:35:52.011  00:02:18.986     1107		echo "************************************"
00:35:52.011  00:02:18.986     1108		echo "START TEST $test_name"
00:35:52.011  00:02:18.986     1109		echo "************************************"
00:35:52.012  00:02:18.986     1110		xtrace_restore
00:35:52.012  00:02:18.987     1111		time "$@"
00:35:52.012  00:02:18.987     1112		xtrace_disable
00:35:52.012  00:02:18.987     1113		echo "************************************"
00:35:52.012  00:02:18.987     1114		echo "END TEST $test_name"
00:35:52.012  00:02:18.987     1115		echo "************************************"
00:35:52.012  00:02:18.987     1116		timing_exit $test_name
00:35:52.012  00:02:18.987       ...
00:35:52.012  00:02:18.987  in /var/jenkins/workspace/unittest-clang-docker-autotest/spdk/test/common/autobuild_common.sh:411 -> unittest_build([])
00:35:52.012  00:02:18.987       ...
00:35:52.012  00:02:18.987     406 	autobuild_test_suite() {
00:35:52.012  00:02:18.987     407 		run_test "autobuild" _autobuild_test_suite
00:35:52.012  00:02:18.987     408 	}
00:35:52.012  00:02:18.987     409 	
00:35:52.012  00:02:18.987     410 	unittest_build() {
00:35:52.012  00:02:18.987  => 411 		run_test "unittest_build" _unittest_build
00:35:52.012  00:02:18.987     412 	}
00:35:52.012  00:02:18.987     413 	
00:35:52.012  00:02:18.987     414 	scanbuild_make() {
00:35:52.012  00:02:18.987     415 		run_test "scanbuild_make" _scanbuild_make
00:35:52.012  00:02:18.987     416 	}
00:35:52.012  00:02:18.987       ...
00:35:52.012  00:02:18.987  in jbp/jenkins/jjb-config/jobs/scripts/docker-autoruner.sh:55 -> main([])
00:35:52.012  00:02:18.987       ...
00:35:52.012  00:02:18.987  backtrace unavailable
00:35:52.012  00:02:18.987       ...
00:35:52.012  00:02:18.987  
00:35:52.012  00:02:18.987  ========== Backtrace end =========='
00:35:52.012  + for build in "${build_log_list[@]}"
00:35:52.012  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_75716/build.log
00:35:52.012  + local build_log job_name start_pattern end_pattern backtraces
00:35:52.012  + build_log=/var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_75716/build.log
00:35:52.012  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_75716/build.log
00:35:52.012  ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_75716
00:35:52.012  + job_name=nvme-cmb-pmr-vg-autotest_75716
00:35:52.012  + start_pattern='Backtrace start'
00:35:52.012  + end_pattern='Backtrace end'
00:35:52.012  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_75716/build.log ]]
00:35:52.012  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_75716/build.log
00:35:52.012  + backtraces=
00:35:52.012  + [[ -n '' ]]
00:35:52.012  + for build in "${build_log_list[@]}"
00:35:52.012  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_17454/build.log
00:35:52.012  + local build_log job_name start_pattern end_pattern backtraces
00:35:52.012  + build_log=/var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_17454/build.log
00:35:52.012  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_17454/build.log
00:35:52.012  ++ basename /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_17454
00:35:52.012  + job_name=autobuild-external-vg-autotest_17454
00:35:52.012  + start_pattern='Backtrace start'
00:35:52.012  + end_pattern='Backtrace end'
00:35:52.012  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_17454/build.log ]]
00:35:52.012  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_17454/build.log
00:35:52.012  + backtraces=
00:35:52.012  + [[ -n '' ]]
00:35:52.012  + for build in "${build_log_list[@]}"
00:35:52.012  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_19502/build.log
00:35:52.012  + local build_log job_name start_pattern end_pattern backtraces
00:35:52.012  + build_log=/var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_19502/build.log
00:35:52.012  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_19502/build.log
00:35:52.012  ++ basename /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_19502
00:35:52.012  + job_name=check-format-docker-autotest_19502
00:35:52.012  + start_pattern='Backtrace start'
00:35:52.012  + end_pattern='Backtrace end'
00:35:52.012  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_19502/build.log ]]
00:35:52.012  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_19502/build.log
00:35:52.012  + backtraces=
00:35:52.012  + [[ -n '' ]]
00:35:52.012  + for build in "${build_log_list[@]}"
00:35:52.012  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_23411/build.log
00:35:52.012  + local build_log job_name start_pattern end_pattern backtraces
00:35:52.012  + build_log=/var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_23411/build.log
00:35:52.012  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_23411/build.log
00:35:52.012  ++ basename /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_23411
00:35:52.012  + job_name=BlobFS-phy-autotest_23411
00:35:52.012  + start_pattern='Backtrace start'
00:35:52.012  + end_pattern='Backtrace end'
00:35:52.012  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_23411/build.log ]]
00:35:52.012  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_23411/build.log
00:35:52.012  + backtraces='00:05:39.068  ========== Backtrace start: ==========
00:35:52.012  00:05:39.068  
00:35:52.012  00:05:39.068  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/common/autotest_common.sh:960 -> killprocess(["1081163"])
00:35:52.012  00:05:39.068       ...
00:35:52.012  00:05:39.068     955 				kill $1
00:35:52.012  00:05:39.068     956 			fi
00:35:52.012  00:05:39.068     957 	
00:35:52.012  00:05:39.068     958 			# wait for the process regardless if its the dummy sudo one
00:35:52.012  00:05:39.068     959 			# or the actual app - it should terminate anyway
00:35:52.012  00:05:39.068  => 960 			wait $1
00:35:52.012  00:05:39.068     961 		else
00:35:52.012  00:05:39.068     962 			# the process is not there anymore
00:35:52.012  00:05:39.068     963 			echo "Process with pid $1 is not found"
00:35:52.012  00:05:39.068     964 		fi
00:35:52.012  00:05:39.068     965 	}
00:35:52.012  00:05:39.068       ...
00:35:52.012  00:05:39.068  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:52.012  00:05:39.068       ...
00:35:52.012  00:05:39.068     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:52.013  00:05:39.068     35  			sleep 3
00:35:52.013  00:05:39.068     36  		done
00:35:52.013  00:05:39.068     37  	
00:35:52.013  00:05:39.068     38  		waitforlisten $repeat_pid $rpc_server
00:35:52.013  00:05:39.068  => 39  		killprocess $repeat_pid
00:35:52.013  00:05:39.068     40  		trap - SIGINT SIGTERM EXIT
00:35:52.013  00:05:39.068     41  	
00:35:52.013  00:05:39.068     42  		return 0
00:35:52.013  00:05:39.068     43  	}
00:35:52.013  00:05:39.068     44  	
00:35:52.013  00:05:39.068       ...
00:35:52.013  00:05:39.068  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:52.013  00:05:39.068       ...
00:35:52.013  00:05:39.068     1106		timing_enter $test_name
00:35:52.013  00:05:39.068     1107		echo "************************************"
00:35:52.013  00:05:39.068     1108		echo "START TEST $test_name"
00:35:52.013  00:05:39.068     1109		echo "************************************"
00:35:52.013  00:05:39.068     1110		xtrace_restore
00:35:52.013  00:05:39.068     1111		time "$@"
00:35:52.013  00:05:39.068     1112		xtrace_disable
00:35:52.013  00:05:39.068     1113		echo "************************************"
00:35:52.013  00:05:39.068     1114		echo "END TEST $test_name"
00:35:52.013  00:05:39.068     1115		echo "************************************"
00:35:52.013  00:05:39.068     1116		timing_exit $test_name
00:35:52.013  00:05:39.068       ...
00:35:52.013  00:05:39.068  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/event/event.sh:52 -> main([])
00:35:52.013  00:05:39.068       ...
00:35:52.013  00:05:39.068     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:52.013  00:05:39.068     48  	
00:35:52.013  00:05:39.068     49  	if [ $(uname -s) = Linux ]; then
00:35:52.013  00:05:39.068     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:52.013  00:05:39.068     51  		if modprobe -n nbd; then
00:35:52.013  00:05:39.068  => 52  			run_test "app_repeat" app_repeat_test
00:35:52.013  00:05:39.068     53  		fi
00:35:52.013  00:05:39.068     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:52.013  00:05:39.068     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:52.013  00:05:39.068     56  		fi
00:35:52.013  00:05:39.068     57  	fi
00:35:52.013  00:05:39.068       ...
00:35:52.013  00:05:39.068  
00:35:52.013  00:05:39.068  ========== Backtrace end ==========
00:35:52.013  00:05:39.068  ========== Backtrace start: ==========
00:35:52.013  00:05:39.068  
00:35:52.013  00:05:39.068  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/event/event.sh"])
00:35:52.013  00:05:39.068       ...
00:35:52.013  00:05:39.068     1106		timing_enter $test_name
00:35:52.013  00:05:39.068     1107		echo "************************************"
00:35:52.013  00:05:39.068     1108		echo "START TEST $test_name"
00:35:52.013  00:05:39.068     1109		echo "************************************"
00:35:52.013  00:05:39.068     1110		xtrace_restore
00:35:52.013  00:05:39.068     1111		time "$@"
00:35:52.013  00:05:39.068     1112		xtrace_disable
00:35:52.013  00:05:39.068     1113		echo "************************************"
00:35:52.013  00:05:39.068     1114		echo "END TEST $test_name"
00:35:52.013  00:05:39.068     1115		echo "************************************"
00:35:52.013  00:05:39.068     1116		timing_exit $test_name
00:35:52.013  00:05:39.068       ...
00:35:52.013  00:05:39.069  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/autotest.sh:177 -> main(["/var/jenkins/workspace/BlobFS-phy-autotest/autorun-spdk.conf"])
00:35:52.013  00:05:39.069       ...
00:35:52.013  00:05:39.069     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:52.013  00:05:39.069     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:52.013  00:05:39.069     174 		fi
00:35:52.013  00:05:39.069     175 	
00:35:52.013  00:05:39.069     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:52.013  00:05:39.069  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:52.013  00:05:39.069     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:52.013  00:05:39.069     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:52.013  00:05:39.069     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:52.013  00:05:39.069     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:52.013  00:05:39.069     182 		run_test "version" $rootdir/test/app/version.sh
00:35:52.013  00:05:39.069       ...
00:35:52.013  00:05:39.069  
00:35:52.013  00:05:39.069  ========== Backtrace end =========='
00:35:52.013  + [[ -n 00:05:39.068  ========== Backtrace start: ==========
00:35:52.013  00:05:39.068  
00:35:52.013  00:05:39.068  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/common/autotest_common.sh:960 -> killprocess(["1081163"])
00:35:52.013  00:05:39.068       ...
00:35:52.013  00:05:39.068     955 				kill $1
00:35:52.013  00:05:39.068     956 			fi
00:35:52.013  00:05:39.068     957 	
00:35:52.013  00:05:39.068     958 			# wait for the process regardless if its the dummy sudo one
00:35:52.013  00:05:39.068     959 			# or the actual app - it should terminate anyway
00:35:52.013  00:05:39.068  => 960 			wait $1
00:35:52.013  00:05:39.068     961 		else
00:35:52.013  00:05:39.068     962 			# the process is not there anymore
00:35:52.013  00:05:39.068     963 			echo "Process with pid $1 is not found"
00:35:52.013  00:05:39.068     964 		fi
00:35:52.013  00:05:39.068     965 	}
00:35:52.013  00:05:39.068       ...
00:35:52.013  00:05:39.068  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:52.013  00:05:39.068       ...
00:35:52.013  00:05:39.068     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:52.013  00:05:39.068     35  			sleep 3
00:35:52.013  00:05:39.068     36  		done
00:35:52.013  00:05:39.068     37  	
00:35:52.013  00:05:39.068     38  		waitforlisten $repeat_pid $rpc_server
00:35:52.014  00:05:39.068  => 39  		killprocess $repeat_pid
00:35:52.014  00:05:39.068     40  		trap - SIGINT SIGTERM EXIT
00:35:52.014  00:05:39.068     41  	
00:35:52.014  00:05:39.068     42  		return 0
00:35:52.014  00:05:39.068     43  	}
00:35:52.014  00:05:39.068     44  	
00:35:52.014  00:05:39.068       ...
00:35:52.014  00:05:39.068  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:52.014  00:05:39.068       ...
00:35:52.014  00:05:39.068     1106		timing_enter $test_name
00:35:52.014  00:05:39.068     1107		echo "************************************"
00:35:52.014  00:05:39.068     1108		echo "START TEST $test_name"
00:35:52.014  00:05:39.068     1109		echo "************************************"
00:35:52.014  00:05:39.068     1110		xtrace_restore
00:35:52.014  00:05:39.068     1111		time "$@"
00:35:52.014  00:05:39.068     1112		xtrace_disable
00:35:52.014  00:05:39.068     1113		echo "************************************"
00:35:52.014  00:05:39.068     1114		echo "END TEST $test_name"
00:35:52.014  00:05:39.068     1115		echo "************************************"
00:35:52.014  00:05:39.068     1116		timing_exit $test_name
00:35:52.014  00:05:39.068       ...
00:35:52.014  00:05:39.068  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/event/event.sh:52 -> main([])
00:35:52.014  00:05:39.068       ...
00:35:52.014  00:05:39.068     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:52.014  00:05:39.068     48  	
00:35:52.014  00:05:39.068     49  	if [ $(uname -s) = Linux ]; then
00:35:52.014  00:05:39.068     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:52.014  00:05:39.068     51  		if modprobe -n nbd; then
00:35:52.014  00:05:39.068  => 52  			run_test "app_repeat" app_repeat_test
00:35:52.014  00:05:39.068     53  		fi
00:35:52.014  00:05:39.068     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:52.014  00:05:39.068     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:52.014  00:05:39.068     56  		fi
00:35:52.014  00:05:39.068     57  	fi
00:35:52.014  00:05:39.068       ...
00:35:52.014  00:05:39.068  
00:35:52.014  00:05:39.068  ========== Backtrace end ==========
00:35:52.014  00:05:39.068  ========== Backtrace start: ==========
00:35:52.014  00:05:39.068  
00:35:52.014  00:05:39.068  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/event/event.sh"])
00:35:52.014  00:05:39.068       ...
00:35:52.014  00:05:39.068     1106		timing_enter $test_name
00:35:52.014  00:05:39.068     1107		echo "************************************"
00:35:52.014  00:05:39.068     1108		echo "START TEST $test_name"
00:35:52.014  00:05:39.068     1109		echo "************************************"
00:35:52.014  00:05:39.068     1110		xtrace_restore
00:35:52.014  00:05:39.068     1111		time "$@"
00:35:52.014  00:05:39.068     1112		xtrace_disable
00:35:52.014  00:05:39.068     1113		echo "************************************"
00:35:52.014  00:05:39.068     1114		echo "END TEST $test_name"
00:35:52.014  00:05:39.068     1115		echo "************************************"
00:35:52.014  00:05:39.068     1116		timing_exit $test_name
00:35:52.014  00:05:39.068       ...
00:35:52.014  00:05:39.069  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/autotest.sh:177 -> main(["/var/jenkins/workspace/BlobFS-phy-autotest/autorun-spdk.conf"])
00:35:52.014  00:05:39.069       ...
00:35:52.014  00:05:39.069     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:52.014  00:05:39.069     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:52.014  00:05:39.069     174 		fi
00:35:52.014  00:05:39.069     175 	
00:35:52.014  00:05:39.069     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:52.014  00:05:39.069  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:52.014  00:05:39.069     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:52.014  00:05:39.069     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:52.014  00:05:39.069     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:52.014  00:05:39.069     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:52.014  00:05:39.069     182 		run_test "version" $rootdir/test/app/version.sh
00:35:52.014  00:05:39.069       ...
00:35:52.014  00:05:39.069  
00:35:52.014  00:05:39.069  ========== Backtrace end ========== ]]
00:35:52.014  + echo '00:05:39.068  ========== Backtrace start: ==========
00:35:52.014  00:05:39.068  
00:35:52.014  00:05:39.068  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/common/autotest_common.sh:960 -> killprocess(["1081163"])
00:35:52.014  00:05:39.068       ...
00:35:52.014  00:05:39.068     955 				kill $1
00:35:52.014  00:05:39.068     956 			fi
00:35:52.014  00:05:39.068     957 	
00:35:52.014  00:05:39.068     958 			# wait for the process regardless if its the dummy sudo one
00:35:52.014  00:05:39.068     959 			# or the actual app - it should terminate anyway
00:35:52.014  00:05:39.068  => 960 			wait $1
00:35:52.014  00:05:39.068     961 		else
00:35:52.014  00:05:39.068     962 			# the process is not there anymore
00:35:52.014  00:05:39.068     963 			echo "Process with pid $1 is not found"
00:35:52.014  00:05:39.068     964 		fi
00:35:52.014  00:05:39.068     965 	}
00:35:52.014  00:05:39.068       ...
00:35:52.014  00:05:39.068  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:52.014  00:05:39.068       ...
00:35:52.014  00:05:39.068     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:52.014  00:05:39.068     35  			sleep 3
00:35:52.014  00:05:39.068     36  		done
00:35:52.014  00:05:39.068     37  	
00:35:52.014  00:05:39.068     38  		waitforlisten $repeat_pid $rpc_server
00:35:52.014  00:05:39.068  => 39  		killprocess $repeat_pid
00:35:52.014  00:05:39.068     40  		trap - SIGINT SIGTERM EXIT
00:35:52.015  00:05:39.068     41  	
00:35:52.015  00:05:39.068     42  		return 0
00:35:52.015  00:05:39.068     43  	}
00:35:52.015  00:05:39.068     44  	
00:35:52.015  00:05:39.068       ...
00:35:52.015  00:05:39.068  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:52.015  00:05:39.068       ...
00:35:52.015  00:05:39.068     1106		timing_enter $test_name
00:35:52.015  00:05:39.068     1107		echo "************************************"
00:35:52.015  00:05:39.068     1108		echo "START TEST $test_name"
00:35:52.015  00:05:39.068     1109		echo "************************************"
00:35:52.015  00:05:39.068     1110		xtrace_restore
00:35:52.015  00:05:39.068     1111		time "$@"
00:35:52.015  00:05:39.068     1112		xtrace_disable
00:35:52.015  00:05:39.068     1113		echo "************************************"
00:35:52.015  00:05:39.068     1114		echo "END TEST $test_name"
00:35:52.015  00:05:39.068     1115		echo "************************************"
00:35:52.015  00:05:39.068     1116		timing_exit $test_name
00:35:52.015  00:05:39.068       ...
00:35:52.015  00:05:39.068  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/event/event.sh:52 -> main([])
00:35:52.015  00:05:39.068       ...
00:35:52.015  00:05:39.068     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:52.015  00:05:39.068     48  	
00:35:52.015  00:05:39.068     49  	if [ $(uname -s) = Linux ]; then
00:35:52.015  00:05:39.068     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:52.015  00:05:39.068     51  		if modprobe -n nbd; then
00:35:52.015  00:05:39.068  => 52  			run_test "app_repeat" app_repeat_test
00:35:52.015  00:05:39.068     53  		fi
00:35:52.015  00:05:39.068     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:52.015  00:05:39.068     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:52.015  00:05:39.068     56  		fi
00:35:52.015  00:05:39.068     57  	fi
00:35:52.015  00:05:39.068       ...
00:35:52.015  00:05:39.068  
00:35:52.015  00:05:39.068  ========== Backtrace end ==========
00:35:52.015  00:05:39.068  ========== Backtrace start: ==========
00:35:52.015  00:05:39.068  
00:35:52.015  00:05:39.068  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/var/jenkins/workspace/BlobFS-phy-autotest/spdk/test/event/event.sh"])
00:35:52.015  00:05:39.068       ...
00:35:52.015  00:05:39.068     1106		timing_enter $test_name
00:35:52.015  00:05:39.068     1107		echo "************************************"
00:35:52.015  00:05:39.068     1108		echo "START TEST $test_name"
00:35:52.015  00:05:39.068     1109		echo "************************************"
00:35:52.015  00:05:39.068     1110		xtrace_restore
00:35:52.015  00:05:39.068     1111		time "$@"
00:35:52.015  00:05:39.068     1112		xtrace_disable
00:35:52.015  00:05:39.068     1113		echo "************************************"
00:35:52.015  00:05:39.068     1114		echo "END TEST $test_name"
00:35:52.015  00:05:39.068     1115		echo "************************************"
00:35:52.015  00:05:39.068     1116		timing_exit $test_name
00:35:52.015  00:05:39.068       ...
00:35:52.015  00:05:39.069  in /var/jenkins/workspace/BlobFS-phy-autotest/spdk/autotest.sh:177 -> main(["/var/jenkins/workspace/BlobFS-phy-autotest/autorun-spdk.conf"])
00:35:52.015  00:05:39.069       ...
00:35:52.015  00:05:39.069     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:52.015  00:05:39.069     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:52.015  00:05:39.069     174 		fi
00:35:52.015  00:05:39.069     175 	
00:35:52.015  00:05:39.069     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:52.015  00:05:39.069  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:52.015  00:05:39.069     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:52.015  00:05:39.069     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:52.015  00:05:39.069     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:52.015  00:05:39.069     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:52.015  00:05:39.069     182 		run_test "version" $rootdir/test/app/version.sh
00:35:52.015  00:05:39.069       ...
00:35:52.015  00:05:39.069  
00:35:52.015  00:05:39.069  ========== Backtrace end =========='
00:35:52.015  + for build in "${build_log_list[@]}"
00:35:52.015  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_73696/build.log
00:35:52.015  + local build_log job_name start_pattern end_pattern backtraces
00:35:52.015  + build_log=/var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_73696/build.log
00:35:52.015  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_73696/build.log
00:35:52.015  ++ basename /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_73696
00:35:52.015  + job_name=zns-vg-autotest_73696
00:35:52.015  + start_pattern='Backtrace start'
00:35:52.015  + end_pattern='Backtrace end'
00:35:52.015  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_73696/build.log ]]
00:35:52.015  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_73696/build.log
00:35:52.015  + backtraces=
00:35:52.015  + [[ -n '' ]]
00:35:52.015  + for build in "${build_log_list[@]}"
00:35:52.015  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_19522/build.log
00:35:52.015  + local build_log job_name start_pattern end_pattern backtraces
00:35:52.015  + build_log=/var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_19522/build.log
00:35:52.015  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_19522/build.log
00:35:52.015  ++ basename /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_19522
00:35:52.015  + job_name=check-so-deps-docker-autotest_19522
00:35:52.015  + start_pattern='Backtrace start'
00:35:52.015  + end_pattern='Backtrace end'
00:35:52.015  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_19522/build.log ]]
00:35:52.015  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_19522/build.log
00:35:52.015  + backtraces=
00:35:52.016  + [[ -n '' ]]
00:35:52.016  + for build in "${build_log_list[@]}"
00:35:52.016  + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_74594/build.log
00:35:52.016  + local build_log job_name start_pattern end_pattern backtraces
00:35:52.016  + build_log=/var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_74594/build.log
00:35:52.016  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_74594/build.log
00:35:52.016  ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_74594
00:35:52.016  + job_name=nvme-vg-autotest_74594
00:35:52.016  + start_pattern='Backtrace start'
00:35:52.016  + end_pattern='Backtrace end'
00:35:52.016  + [[ -f /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_74594/build.log ]]
00:35:52.016  ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_74594/build.log
00:35:52.016  + backtraces='00:13:50.288  ========== Backtrace start: ==========
00:35:52.016  00:13:50.288  
00:35:52.016  00:13:50.288  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:960 -> killprocess(["63037"])
00:35:52.016  00:13:50.288       ...
00:35:52.016  00:13:50.288     955 				kill $1
00:35:52.016  00:13:50.288     956 			fi
00:35:52.016  00:13:50.288     957 	
00:35:52.016  00:13:50.288     958 			# wait for the process regardless if its the dummy sudo one
00:35:52.016  00:13:50.288     959 			# or the actual app - it should terminate anyway
00:35:52.016  00:13:50.288  => 960 			wait $1
00:35:52.016  00:13:50.288     961 		else
00:35:52.016  00:13:50.288     962 			# the process is not there anymore
00:35:52.016  00:13:50.288     963 			echo "Process with pid $1 is not found"
00:35:52.016  00:13:50.288     964 		fi
00:35:52.016  00:13:50.288     965 	}
00:35:52.016  00:13:50.288       ...
00:35:52.016  00:13:50.288  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:52.016  00:13:50.288       ...
00:35:52.016  00:13:50.288     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:52.016  00:13:50.288     35  			sleep 3
00:35:52.016  00:13:50.288     36  		done
00:35:52.016  00:13:50.288     37  	
00:35:52.016  00:13:50.288     38  		waitforlisten $repeat_pid $rpc_server
00:35:52.016  00:13:50.288  => 39  		killprocess $repeat_pid
00:35:52.016  00:13:50.288     40  		trap - SIGINT SIGTERM EXIT
00:35:52.016  00:13:50.288     41  	
00:35:52.016  00:13:50.288     42  		return 0
00:35:52.016  00:13:50.288     43  	}
00:35:52.016  00:13:50.288     44  	
00:35:52.016  00:13:50.288       ...
00:35:52.016  00:13:50.288  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:52.016  00:13:50.288       ...
00:35:52.016  00:13:50.288     1106		timing_enter $test_name
00:35:52.016  00:13:50.288     1107		echo "************************************"
00:35:52.016  00:13:50.288     1108		echo "START TEST $test_name"
00:35:52.016  00:13:50.288     1109		echo "************************************"
00:35:52.016  00:13:50.288     1110		xtrace_restore
00:35:52.016  00:13:50.288     1111		time "$@"
00:35:52.016  00:13:50.288     1112		xtrace_disable
00:35:52.016  00:13:50.288     1113		echo "************************************"
00:35:52.016  00:13:50.288     1114		echo "END TEST $test_name"
00:35:52.016  00:13:50.288     1115		echo "************************************"
00:35:52.016  00:13:50.288     1116		timing_exit $test_name
00:35:52.016  00:13:50.288       ...
00:35:52.016  00:13:50.288  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:52 -> main([])
00:35:52.016  00:13:50.288       ...
00:35:52.016  00:13:50.288     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:52.016  00:13:50.288     48  	
00:35:52.016  00:13:50.288     49  	if [ $(uname -s) = Linux ]; then
00:35:52.016  00:13:50.288     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:52.016  00:13:50.288     51  		if modprobe -n nbd; then
00:35:52.016  00:13:50.288  => 52  			run_test "app_repeat" app_repeat_test
00:35:52.016  00:13:50.288     53  		fi
00:35:52.016  00:13:50.288     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:52.016  00:13:50.288     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:52.016  00:13:50.288     56  		fi
00:35:52.016  00:13:50.288     57  	fi
00:35:52.016  00:13:50.288       ...
00:35:52.016  00:13:50.288  
00:35:52.016  00:13:50.288  ========== Backtrace end ==========
00:35:52.016  00:13:50.288  ========== Backtrace start: ==========
00:35:52.016  00:13:50.288  
00:35:52.016  00:13:50.288  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/home/vagrant/spdk_repo/spdk/test/event/event.sh"])
00:35:52.016  00:13:50.288       ...
00:35:52.016  00:13:50.288     1106		timing_enter $test_name
00:35:52.016  00:13:50.288     1107		echo "************************************"
00:35:52.016  00:13:50.288     1108		echo "START TEST $test_name"
00:35:52.016  00:13:50.288     1109		echo "************************************"
00:35:52.017  00:13:50.288     1110		xtrace_restore
00:35:52.017  00:13:50.288     1111		time "$@"
00:35:52.017  00:13:50.288     1112		xtrace_disable
00:35:52.017  00:13:50.288     1113		echo "************************************"
00:35:52.017  00:13:50.288     1114		echo "END TEST $test_name"
00:35:52.017  00:13:50.288     1115		echo "************************************"
00:35:52.017  00:13:50.288     1116		timing_exit $test_name
00:35:52.017  00:13:50.288       ...
00:35:52.017  00:13:50.288  in /home/vagrant/spdk_repo/spdk/autotest.sh:177 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:52.017  00:13:50.288       ...
00:35:52.017  00:13:50.288     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:52.017  00:13:50.288     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:52.017  00:13:50.288     174 		fi
00:35:52.017  00:13:50.288     175 	
00:35:52.017  00:13:50.288     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:52.017  00:13:50.288  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:52.017  00:13:50.288     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:52.017  00:13:50.288     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:52.017  00:13:50.288     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:52.017  00:13:50.288     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:52.017  00:13:50.288     182 		run_test "version" $rootdir/test/app/version.sh
00:35:52.017  00:13:50.288       ...
00:35:52.017  00:13:50.288  
00:35:52.017  00:13:50.288  ========== Backtrace end =========='
00:35:52.017  + [[ -n 00:13:50.288  ========== Backtrace start: ==========
00:35:52.017  00:13:50.288  
00:35:52.017  00:13:50.288  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:960 -> killprocess(["63037"])
00:35:52.017  00:13:50.288       ...
00:35:52.017  00:13:50.288     955 				kill $1
00:35:52.017  00:13:50.288     956 			fi
00:35:52.017  00:13:50.288     957 	
00:35:52.017  00:13:50.288     958 			# wait for the process regardless if its the dummy sudo one
00:35:52.017  00:13:50.288     959 			# or the actual app - it should terminate anyway
00:35:52.017  00:13:50.288  => 960 			wait $1
00:35:52.017  00:13:50.288     961 		else
00:35:52.017  00:13:50.288     962 			# the process is not there anymore
00:35:52.017  00:13:50.288     963 			echo "Process with pid $1 is not found"
00:35:52.017  00:13:50.288     964 		fi
00:35:52.017  00:13:50.288     965 	}
00:35:52.017  00:13:50.288       ...
00:35:52.017  00:13:50.288  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:52.017  00:13:50.288       ...
00:35:52.017  00:13:50.288     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:52.017  00:13:50.288     35  			sleep 3
00:35:52.017  00:13:50.288     36  		done
00:35:52.017  00:13:50.288     37  	
00:35:52.017  00:13:50.288     38  		waitforlisten $repeat_pid $rpc_server
00:35:52.017  00:13:50.288  => 39  		killprocess $repeat_pid
00:35:52.017  00:13:50.288     40  		trap - SIGINT SIGTERM EXIT
00:35:52.017  00:13:50.288     41  	
00:35:52.017  00:13:50.288     42  		return 0
00:35:52.017  00:13:50.288     43  	}
00:35:52.017  00:13:50.288     44  	
00:35:52.017  00:13:50.288       ...
00:35:52.017  00:13:50.288  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:52.017  00:13:50.288       ...
00:35:52.017  00:13:50.288     1106		timing_enter $test_name
00:35:52.017  00:13:50.288     1107		echo "************************************"
00:35:52.017  00:13:50.288     1108		echo "START TEST $test_name"
00:35:52.017  00:13:50.288     1109		echo "************************************"
00:35:52.017  00:13:50.288     1110		xtrace_restore
00:35:52.017  00:13:50.288     1111		time "$@"
00:35:52.017  00:13:50.288     1112		xtrace_disable
00:35:52.017  00:13:50.288     1113		echo "************************************"
00:35:52.017  00:13:50.288     1114		echo "END TEST $test_name"
00:35:52.017  00:13:50.288     1115		echo "************************************"
00:35:52.017  00:13:50.288     1116		timing_exit $test_name
00:35:52.017  00:13:50.288       ...
00:35:52.017  00:13:50.288  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:52 -> main([])
00:35:52.017  00:13:50.288       ...
00:35:52.017  00:13:50.288     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:52.017  00:13:50.288     48  	
00:35:52.017  00:13:50.288     49  	if [ $(uname -s) = Linux ]; then
00:35:52.017  00:13:50.288     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:52.017  00:13:50.288     51  		if modprobe -n nbd; then
00:35:52.017  00:13:50.288  => 52  			run_test "app_repeat" app_repeat_test
00:35:52.017  00:13:50.288     53  		fi
00:35:52.017  00:13:50.288     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:52.017  00:13:50.288     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:52.017  00:13:50.288     56  		fi
00:35:52.017  00:13:50.288     57  	fi
00:35:52.017  00:13:50.288       ...
00:35:52.017  00:13:50.288  
00:35:52.017  00:13:50.288  ========== Backtrace end ==========
00:35:52.017  00:13:50.288  ========== Backtrace start: ==========
00:35:52.017  00:13:50.288  
00:35:52.017  00:13:50.288  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/home/vagrant/spdk_repo/spdk/test/event/event.sh"])
00:35:52.017  00:13:50.288       ...
00:35:52.017  00:13:50.288     1106		timing_enter $test_name
00:35:52.017  00:13:50.288     1107		echo "************************************"
00:35:52.017  00:13:50.288     1108		echo "START TEST $test_name"
00:35:52.017  00:13:50.288     1109		echo "************************************"
00:35:52.017  00:13:50.288     1110		xtrace_restore
00:35:52.017  00:13:50.288     1111		time "$@"
00:35:52.017  00:13:50.288     1112		xtrace_disable
00:35:52.017  00:13:50.288     1113		echo "************************************"
00:35:52.017  00:13:50.288     1114		echo "END TEST $test_name"
00:35:52.017  00:13:50.288     1115		echo "************************************"
00:35:52.017  00:13:50.288     1116		timing_exit $test_name
00:35:52.017  00:13:50.288       ...
00:35:52.018  00:13:50.288  in /home/vagrant/spdk_repo/spdk/autotest.sh:177 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:52.018  00:13:50.288       ...
00:35:52.018  00:13:50.288     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:52.018  00:13:50.288     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:52.018  00:13:50.288     174 		fi
00:35:52.018  00:13:50.288     175 	
00:35:52.018  00:13:50.288     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:52.018  00:13:50.288  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:52.018  00:13:50.288     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:52.018  00:13:50.288     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:52.018  00:13:50.288     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:52.018  00:13:50.288     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:52.018  00:13:50.288     182 		run_test "version" $rootdir/test/app/version.sh
00:35:52.018  00:13:50.288       ...
00:35:52.018  00:13:50.288  
00:35:52.018  00:13:50.288  ========== Backtrace end ========== ]]
00:35:52.018  + echo '00:13:50.288  ========== Backtrace start: ==========
00:35:52.018  00:13:50.288  
00:35:52.018  00:13:50.288  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:960 -> killprocess(["63037"])
00:35:52.018  00:13:50.288       ...
00:35:52.018  00:13:50.288     955 				kill $1
00:35:52.018  00:13:50.288     956 			fi
00:35:52.018  00:13:50.288     957 	
00:35:52.018  00:13:50.288     958 			# wait for the process regardless if its the dummy sudo one
00:35:52.018  00:13:50.288     959 			# or the actual app - it should terminate anyway
00:35:52.018  00:13:50.288  => 960 			wait $1
00:35:52.018  00:13:50.288     961 		else
00:35:52.018  00:13:50.288     962 			# the process is not there anymore
00:35:52.018  00:13:50.288     963 			echo "Process with pid $1 is not found"
00:35:52.018  00:13:50.288     964 		fi
00:35:52.018  00:13:50.288     965 	}
00:35:52.018  00:13:50.288       ...
00:35:52.018  00:13:50.288  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:39 -> app_repeat_test([])
00:35:52.018  00:13:50.288       ...
00:35:52.018  00:13:50.288     34  			$rootdir/scripts/rpc.py -s $rpc_server spdk_kill_instance SIGTERM
00:35:52.018  00:13:50.288     35  			sleep 3
00:35:52.018  00:13:50.288     36  		done
00:35:52.018  00:13:50.288     37  	
00:35:52.018  00:13:50.288     38  		waitforlisten $repeat_pid $rpc_server
00:35:52.018  00:13:50.288  => 39  		killprocess $repeat_pid
00:35:52.018  00:13:50.288     40  		trap - SIGINT SIGTERM EXIT
00:35:52.018  00:13:50.288     41  	
00:35:52.018  00:13:50.288     42  		return 0
00:35:52.018  00:13:50.288     43  	}
00:35:52.018  00:13:50.288     44  	
00:35:52.018  00:13:50.288       ...
00:35:52.018  00:13:50.288  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["app_repeat"],["app_repeat_test"])
00:35:52.018  00:13:50.288       ...
00:35:52.018  00:13:50.288     1106		timing_enter $test_name
00:35:52.018  00:13:50.288     1107		echo "************************************"
00:35:52.018  00:13:50.288     1108		echo "START TEST $test_name"
00:35:52.018  00:13:50.288     1109		echo "************************************"
00:35:52.018  00:13:50.288     1110		xtrace_restore
00:35:52.018  00:13:50.288     1111		time "$@"
00:35:52.018  00:13:50.288     1112		xtrace_disable
00:35:52.018  00:13:50.288     1113		echo "************************************"
00:35:52.018  00:13:50.288     1114		echo "END TEST $test_name"
00:35:52.018  00:13:50.288     1115		echo "************************************"
00:35:52.018  00:13:50.288     1116		timing_exit $test_name
00:35:52.018  00:13:50.288       ...
00:35:52.018  00:13:50.288  in /home/vagrant/spdk_repo/spdk/test/event/event.sh:52 -> main([])
00:35:52.018  00:13:50.288       ...
00:35:52.018  00:13:50.288     47  	run_test "event_reactor_perf" $testdir/reactor_perf/reactor_perf -t 1
00:35:52.018  00:13:50.288     48  	
00:35:52.018  00:13:50.288     49  	if [ $(uname -s) = Linux ]; then
00:35:52.018  00:13:50.288     50  		run_test "event_scheduler" $testdir/scheduler/scheduler.sh
00:35:52.018  00:13:50.288     51  		if modprobe -n nbd; then
00:35:52.018  00:13:50.288  => 52  			run_test "app_repeat" app_repeat_test
00:35:52.018  00:13:50.288     53  		fi
00:35:52.018  00:13:50.288     54  		if ((SPDK_TEST_CRYPTO == 0)); then
00:35:52.018  00:13:50.288     55  			run_test "cpu_locks" "$testdir/cpu_locks.sh"
00:35:52.018  00:13:50.288     56  		fi
00:35:52.018  00:13:50.288     57  	fi
00:35:52.018  00:13:50.288       ...
00:35:52.018  00:13:50.288  
00:35:52.018  00:13:50.288  ========== Backtrace end ==========
00:35:52.018  00:13:50.288  ========== Backtrace start: ==========
00:35:52.018  00:13:50.288  
00:35:52.018  00:13:50.288  in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1111 -> run_test(["event"],["/home/vagrant/spdk_repo/spdk/test/event/event.sh"])
00:35:52.018  00:13:50.288       ...
00:35:52.018  00:13:50.288     1106		timing_enter $test_name
00:35:52.018  00:13:50.288     1107		echo "************************************"
00:35:52.018  00:13:50.288     1108		echo "START TEST $test_name"
00:35:52.018  00:13:50.288     1109		echo "************************************"
00:35:52.018  00:13:50.288     1110		xtrace_restore
00:35:52.018  00:13:50.288     1111		time "$@"
00:35:52.018  00:13:50.288     1112		xtrace_disable
00:35:52.018  00:13:50.288     1113		echo "************************************"
00:35:52.018  00:13:50.288     1114		echo "END TEST $test_name"
00:35:52.018  00:13:50.288     1115		echo "************************************"
00:35:52.018  00:13:50.288     1116		timing_exit $test_name
00:35:52.018  00:13:50.288       ...
00:35:52.018  00:13:50.288  in /home/vagrant/spdk_repo/spdk/autotest.sh:177 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"])
00:35:52.018  00:13:50.288       ...
00:35:52.018  00:13:50.288     172 		if [[ $SPDK_JSONRPC_GO_CLIENT -eq 0 ]]; then
00:35:52.018  00:13:50.288     173 			run_test "spdkcli_tcp" $rootdir/test/spdkcli/tcp.sh
00:35:52.018  00:13:50.288     174 		fi
00:35:52.018  00:13:50.288     175 	
00:35:52.019  00:13:50.288     176 		run_test "dpdk_mem_utility" $rootdir/test/dpdk_memory_utility/test_dpdk_mem_info.sh
00:35:52.019  00:13:50.288  => 177 		run_test "event" $rootdir/test/event/event.sh
00:35:52.019  00:13:50.288     178 		run_test "thread" $rootdir/test/thread/thread.sh
00:35:52.019  00:13:50.288     179 		run_test "accel" $rootdir/test/accel/accel.sh
00:35:52.019  00:13:50.288     180 		run_test "accel_rpc" $rootdir/test/accel/accel_rpc.sh
00:35:52.019  00:13:50.288     181 		run_test "app_cmdline" $rootdir/test/app/cmdline.sh
00:35:52.019  00:13:50.288     182 		run_test "version" $rootdir/test/app/version.sh
00:35:52.019  00:13:50.288       ...
00:35:52.019  00:13:50.288  
00:35:52.019  00:13:50.288  ========== Backtrace end =========='
00:35:52.019  + run_flamegraph
00:35:52.019  + create_timing_file
00:35:52.019  + local array file
00:35:52.019  + array=($(find $WORKSPACE -type f -name 'timing.txt'))
00:35:52.019  ++ find /var/jenkins/workspace/autotest-per-patch_2 -type f -name timing.txt
00:35:52.019  + timing_dir=/var/jenkins/workspace/autotest-per-patch_2/post_process
00:35:52.019  + mkdir -p /var/jenkins/workspace/autotest-per-patch_2/post_process
00:35:52.019  + timing_file=/var/jenkins/workspace/autotest-per-patch_2/post_process/total_timing.txt
00:35:52.019  + for file in "${array[@]}"
00:35:52.019  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_74496/timing.txt
00:35:52.019  + local job_file job_name
00:35:52.019  + job_file=/var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_74496/timing.txt
00:35:52.019  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_74496/timing.txt
00:35:52.019  ++ basename /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_74496
00:35:52.019  + job_name=iscsi-uring-vg-autotest_74496
00:35:52.019  + sed 's/^/iscsi-uring-vg-autotest_74496;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_74496/timing.txt
00:35:52.019  + for file in "${array[@]}"
00:35:52.019  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_17621/timing.txt
00:35:52.019  + local job_file job_name
00:35:52.019  + job_file=/var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_17621/timing.txt
00:35:52.019  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_17621/timing.txt
00:35:52.019  ++ basename /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_17621
00:35:52.019  + job_name=unittest-main-docker-autotest_17621
00:35:52.019  + sed 's/^/unittest-main-docker-autotest_17621;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_17621/timing.txt
00:35:52.019  + for file in "${array[@]}"
00:35:52.019  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_19665/timing.txt
00:35:52.019  + local job_file job_name
00:35:52.019  + job_file=/var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_19665/timing.txt
00:35:52.019  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_19665/timing.txt
00:35:52.019  ++ basename /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_19665
00:35:52.019  + job_name=doc-docker-autotest_19665
00:35:52.019  + sed 's/^/doc-docker-autotest_19665;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_19665/timing.txt
00:35:52.019  + for file in "${array[@]}"
00:35:52.019  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_69577/timing.txt
00:35:52.019  + local job_file job_name
00:35:52.019  + job_file=/var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_69577/timing.txt
00:35:52.019  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_69577/timing.txt
00:35:52.019  ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_69577
00:35:52.019  + job_name=nvme-phy-autotest_69577
00:35:52.019  + sed 's/^/nvme-phy-autotest_69577;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_69577/timing.txt
00:35:52.019  + for file in "${array[@]}"
00:35:52.019  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_17622/timing.txt
00:35:52.019  + local job_file job_name
00:35:52.019  + job_file=/var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_17622/timing.txt
00:35:52.019  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_17622/timing.txt
00:35:52.019  ++ basename /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_17622
00:35:52.019  + job_name=packaging-docker-autotest_17622
00:35:52.019  + sed 's/^/packaging-docker-autotest_17622;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_17622/timing.txt
00:35:52.019  + for file in "${array[@]}"
00:35:52.019  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_71172/timing.txt
00:35:52.019  + local job_file job_name
00:35:52.019  + job_file=/var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_71172/timing.txt
00:35:52.019  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_71172/timing.txt
00:35:52.019  ++ basename /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_71172
00:35:52.019  + job_name=ubuntu20-vg-autotest_71172
00:35:52.019  + sed 's/^/ubuntu20-vg-autotest_71172;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_71172/timing.txt
00:35:52.019  + for file in "${array[@]}"
00:35:52.019  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_23405/timing.txt
00:35:52.019  + local job_file job_name
00:35:52.019  + job_file=/var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_23405/timing.txt
00:35:52.019  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_23405/timing.txt
00:35:52.019  ++ basename /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_23405
00:35:52.019  + job_name=short-fuzz-phy-autotest_23405
00:35:52.019  + sed 's/^/short-fuzz-phy-autotest_23405;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_23405/timing.txt
00:35:52.019  + for file in "${array[@]}"
00:35:52.019  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_17624/timing.txt
00:35:52.019  + local job_file job_name
00:35:52.019  + job_file=/var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_17624/timing.txt
00:35:52.019  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_17624/timing.txt
00:35:52.019  ++ basename /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_17624
00:35:52.019  + job_name=unittest-valgrind-docker-autotest_17624
00:35:52.019  + sed 's/^/unittest-valgrind-docker-autotest_17624;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_17624/timing.txt
00:35:52.019  + for file in "${array[@]}"
00:35:52.019  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_23417/timing.txt
00:35:52.019  + local job_file job_name
00:35:52.019  + job_file=/var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_23417/timing.txt
00:35:52.020  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_23417/timing.txt
00:35:52.020  ++ basename /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_23417
00:35:52.020  + job_name=vfio-user-phy-autotest_23417
00:35:52.020  + sed 's/^/vfio-user-phy-autotest_23417;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_23417/timing.txt
00:35:52.020  + for file in "${array[@]}"
00:35:52.020  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_19494/timing.txt
00:35:52.020  + local job_file job_name
00:35:52.020  + job_file=/var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_19494/timing.txt
00:35:52.020  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_19494/timing.txt
00:35:52.020  ++ basename /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_19494
00:35:52.020  + job_name=build-files-docker-autotest_19494
00:35:52.020  + sed 's/^/build-files-docker-autotest_19494;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_19494/timing.txt
00:35:52.020  + for file in "${array[@]}"
00:35:52.020  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_23432/timing.txt
00:35:52.020  + local job_file job_name
00:35:52.020  + job_file=/var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_23432/timing.txt
00:35:52.020  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_23432/timing.txt
00:35:52.020  ++ basename /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_23432
00:35:52.020  + job_name=vhost-phy-autotest_23432
00:35:52.020  + sed 's/^/vhost-phy-autotest_23432;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_23432/timing.txt
00:35:52.020  + for file in "${array[@]}"
00:35:52.020  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_69651/timing.txt
00:35:52.020  + local job_file job_name
00:35:52.020  + job_file=/var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_69651/timing.txt
00:35:52.020  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_69651/timing.txt
00:35:52.020  ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_69651
00:35:52.020  + job_name=nvmf-phy-autotest_69651
00:35:52.020  + sed 's/^/nvmf-phy-autotest_69651;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_69651/timing.txt
00:35:52.020  + for file in "${array[@]}"
00:35:52.020  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_17621/timing.txt
00:35:52.020  + local job_file job_name
00:35:52.020  + job_file=/var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_17621/timing.txt
00:35:52.020  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_17621/timing.txt
00:35:52.020  ++ basename /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_17621
00:35:52.020  + job_name=scan-build-docker-autotest_17621
00:35:52.020  + sed 's/^/scan-build-docker-autotest_17621;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_17621/timing.txt
00:35:52.020  + for file in "${array[@]}"
00:35:52.020  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_74582/timing.txt
00:35:52.020  + local job_file job_name
00:35:52.020  + job_file=/var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_74582/timing.txt
00:35:52.020  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_74582/timing.txt
00:35:52.020  ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_74582
00:35:52.020  + job_name=nvmf-tcp-vg-autotest_74582
00:35:52.020  + sed 's/^/nvmf-tcp-vg-autotest_74582;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_74582/timing.txt
00:35:52.020  + for file in "${array[@]}"
00:35:52.020  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_13462/timing.txt
00:35:52.020  + local job_file job_name
00:35:52.020  + job_file=/var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_13462/timing.txt
00:35:52.020  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_13462/timing.txt
00:35:52.020  ++ basename /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_13462
00:35:52.020  + job_name=rocky9-vg-autotest_13462
00:35:52.020  + sed 's/^/rocky9-vg-autotest_13462;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_13462/timing.txt
00:35:52.020  + for file in "${array[@]}"
00:35:52.020  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_74615/timing.txt
00:35:52.020  + local job_file job_name
00:35:52.020  + job_file=/var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_74615/timing.txt
00:35:52.020  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_74615/timing.txt
00:35:52.020  ++ basename /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_74615
00:35:52.020  + job_name=vhost-initiator-vg-autotest_74615
00:35:52.020  + sed 's/^/vhost-initiator-vg-autotest_74615;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_74615/timing.txt
00:35:52.020  + for file in "${array[@]}"
00:35:52.020  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_23409/timing.txt
00:35:52.020  + local job_file job_name
00:35:52.020  + job_file=/var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_23409/timing.txt
00:35:52.020  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_23409/timing.txt
00:35:52.020  ++ basename /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_23409
00:35:52.020  + job_name=crypto-phy-autotest_23409
00:35:52.020  + sed 's/^/crypto-phy-autotest_23409;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_23409/timing.txt
00:35:52.020  + for file in "${array[@]}"
00:35:52.020  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_74631/timing.txt
00:35:52.020  + local job_file job_name
00:35:52.020  + job_file=/var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_74631/timing.txt
00:35:52.020  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_74631/timing.txt
00:35:52.020  ++ basename /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_74631
00:35:52.020  + job_name=iscsi-vg-autotest_74631
00:35:52.020  + sed 's/^/iscsi-vg-autotest_74631;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_74631/timing.txt
00:35:52.021  + for file in "${array[@]}"
00:35:52.021  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_74542/timing.txt
00:35:52.021  + local job_file job_name
00:35:52.021  + job_file=/var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_74542/timing.txt
00:35:52.021  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_74542/timing.txt
00:35:52.021  ++ basename /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_74542
00:35:52.021  + job_name=lvol-vg-autotest_74542
00:35:52.021  + sed 's/^/lvol-vg-autotest_74542;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_74542/timing.txt
00:35:52.021  + for file in "${array[@]}"
00:35:52.021  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_71090/timing.txt
00:35:52.021  + local job_file job_name
00:35:52.021  + job_file=/var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_71090/timing.txt
00:35:52.021  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_71090/timing.txt
00:35:52.021  ++ basename /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_71090
00:35:52.021  + job_name=centos7-vg-autotest_71090
00:35:52.021  + sed 's/^/centos7-vg-autotest_71090;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_71090/timing.txt
00:35:52.021  + for file in "${array[@]}"
00:35:52.021  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_69540/timing.txt
00:35:52.021  + local job_file job_name
00:35:52.021  + job_file=/var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_69540/timing.txt
00:35:52.021  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_69540/timing.txt
00:35:52.021  ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_69540
00:35:52.021  + job_name=nvmf-tcp-phy-autotest_69540
00:35:52.021  + sed 's/^/nvmf-tcp-phy-autotest_69540;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_69540/timing.txt
00:35:52.021  + for file in "${array[@]}"
00:35:52.021  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_74515/timing.txt
00:35:52.021  + local job_file job_name
00:35:52.021  + job_file=/var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_74515/timing.txt
00:35:52.021  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_74515/timing.txt
00:35:52.021  ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_74515
00:35:52.021  + job_name=nvmf-tcp-uring-vg-autotest_74515
00:35:52.021  + sed 's/^/nvmf-tcp-uring-vg-autotest_74515;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_74515/timing.txt
00:35:52.021  + for file in "${array[@]}"
00:35:52.021  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_35184/timing.txt
00:35:52.021  + local job_file job_name
00:35:52.021  + job_file=/var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_35184/timing.txt
00:35:52.021  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_35184/timing.txt
00:35:52.021  ++ basename /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_35184
00:35:52.021  + job_name=ubuntu22-vg-autotest_35184
00:35:52.021  + sed 's/^/ubuntu22-vg-autotest_35184;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_35184/timing.txt
00:35:52.021  + for file in "${array[@]}"
00:35:52.021  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_17619/timing.txt
00:35:52.021  + local job_file job_name
00:35:52.021  + job_file=/var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_17619/timing.txt
00:35:52.021  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_17619/timing.txt
00:35:52.021  ++ basename /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_17619
00:35:52.021  + job_name=unittest-clang-docker-autotest_17619
00:35:52.021  + sed 's/^/unittest-clang-docker-autotest_17619;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_17619/timing.txt
00:35:52.021  + for file in "${array[@]}"
00:35:52.021  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_75716/timing.txt
00:35:52.021  + local job_file job_name
00:35:52.021  + job_file=/var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_75716/timing.txt
00:35:52.021  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_75716/timing.txt
00:35:52.021  ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_75716
00:35:52.021  + job_name=nvme-cmb-pmr-vg-autotest_75716
00:35:52.021  + sed 's/^/nvme-cmb-pmr-vg-autotest_75716;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_75716/timing.txt
00:35:52.021  + for file in "${array[@]}"
00:35:52.021  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_17454/timing.txt
00:35:52.021  + local job_file job_name
00:35:52.021  + job_file=/var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_17454/timing.txt
00:35:52.021  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_17454/timing.txt
00:35:52.021  ++ basename /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_17454
00:35:52.021  + job_name=autobuild-external-vg-autotest_17454
00:35:52.021  + sed 's/^/autobuild-external-vg-autotest_17454;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_17454/timing.txt
00:35:52.021  + for file in "${array[@]}"
00:35:52.021  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_19502/timing.txt
00:35:52.021  + local job_file job_name
00:35:52.021  + job_file=/var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_19502/timing.txt
00:35:52.021  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_19502/timing.txt
00:35:52.021  ++ basename /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_19502
00:35:52.021  + job_name=check-format-docker-autotest_19502
00:35:52.021  + sed 's/^/check-format-docker-autotest_19502;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_19502/timing.txt
00:35:52.021  + for file in "${array[@]}"
00:35:52.021  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_23411/timing.txt
00:35:52.021  + local job_file job_name
00:35:52.021  + job_file=/var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_23411/timing.txt
00:35:52.021  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_23411/timing.txt
00:35:52.021  ++ basename /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_23411
00:35:52.022  + job_name=BlobFS-phy-autotest_23411
00:35:52.022  + sed 's/^/BlobFS-phy-autotest_23411;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_23411/timing.txt
00:35:52.022  + for file in "${array[@]}"
00:35:52.022  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_73696/timing.txt
00:35:52.022  + local job_file job_name
00:35:52.022  + job_file=/var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_73696/timing.txt
00:35:52.022  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_73696/timing.txt
00:35:52.022  ++ basename /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_73696
00:35:52.022  + job_name=zns-vg-autotest_73696
00:35:52.022  + sed 's/^/zns-vg-autotest_73696;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_73696/timing.txt
00:35:52.022  + for file in "${array[@]}"
00:35:52.022  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_19522/timing.txt
00:35:52.022  + local job_file job_name
00:35:52.022  + job_file=/var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_19522/timing.txt
00:35:52.022  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_19522/timing.txt
00:35:52.022  ++ basename /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_19522
00:35:52.022  + job_name=check-so-deps-docker-autotest_19522
00:35:52.022  + sed 's/^/check-so-deps-docker-autotest_19522;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_19522/timing.txt
00:35:52.022  + for file in "${array[@]}"
00:35:52.022  + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_74594/timing.txt
00:35:52.022  + local job_file job_name
00:35:52.022  + job_file=/var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_74594/timing.txt
00:35:52.022  +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_74594/timing.txt
00:35:52.022  ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_74594
00:35:52.022  + job_name=nvme-vg-autotest_74594
00:35:52.022  + sed 's/^/nvme-vg-autotest_74594;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_74594/timing.txt
00:35:52.022  + /usr/local/FlameGraph/flamegraph.pl --title 'Build Timing' --nametype Step: --countname seconds /var/jenkins/workspace/autotest-per-patch_2/post_process/total_timing.txt
00:35:52.022  + run_autorun_post
00:35:52.022  + s_opt=
00:35:52.022  + [[ '' == true ]]
00:35:52.022  + spdk/autorun_post.py -d ./ -r ./spdk
00:35:57.292  lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/module/bdev/nvme/bdev_mdns_client.c:582
00:35:57.292  lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/module/bdev/nvme/bdev_mdns_client.c:613
00:35:57.292  lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/module/bdev/nvme/bdev_mdns_client.c:561
00:35:57.292  lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/module/bdev/nvme/bdev_mdns_client.c:456
00:36:05.399  lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/util/crc64.c:165
00:36:05.399  lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/util/crc32c.c:21
00:36:05.399  lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/util/xor.c:120
00:36:05.656  lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/util/crc16.c:628
00:36:05.656  lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/util/crc16.c:634
00:36:27.574  genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/nvme_spec.h!
00:36:27.574  genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/nvmf_transport.h!
00:36:27.574  genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/nvme.h!
00:36:27.574  genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/thread.h!
00:36:27.574  genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/base64.h!
00:36:27.574  genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/trace.h!
00:36:27.574  genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/mmio.h!
00:36:27.574  genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/histogram_data.h!
00:36:27.574  genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/bdev_module.h!
00:36:27.574  genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/endian.h!
00:36:27.574  genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/util.h!
00:36:27.574  genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/histogram_data.h/histogram_ut.c!
00:36:27.574  genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/env_ocf/include/ocf/ocf_core.h!
00:36:27.574  genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/env_ocf/include/ocf/ocf_io.h!
00:36:27.574  genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/env_ocf/include/ocf/ocf_mngt.h!
00:36:37.539  -----Begin Post Process Script------
00:36:37.539  /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_69577/cov_total.info
00:36:37.539  /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_69651/cov_total.info
00:36:37.539  /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_74582/cov_total.info
00:36:37.539  /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_74615/cov_total.info
00:36:37.539  /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_23409/cov_total.info
00:36:37.539  /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_71090/cov_total.info
00:36:37.539  /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_69540/cov_total.info
00:36:37.539  /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_74515/cov_total.info
00:36:37.539  /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_75716/cov_total.info
00:36:37.539  /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_73696/cov_total.info
00:36:37.539  
00:36:37.539  
00:36:37.539  -----Tests Executed in Build------
00:36:37.539  accel
00:36:37.539  accel_assign_opcode
00:36:37.539  accel_cdev_comp
00:36:37.539  accel_cdev_decmop_full
00:36:37.539  accel_cdev_decomp
00:36:37.539  accel_cdev_decomp_full_mcore
00:36:37.539  accel_cdev_decomp_mcore
00:36:37.539  accel_cdev_decomp_mthread
00:36:37.539  accel_cdev_deomp_full_mthread
00:36:37.539  accel_comp
00:36:37.539  accel_compare
00:36:37.539  accel_compress_verify
00:36:37.539  accel_copy
00:36:37.539  accel_copy_crc32c
00:36:37.539  accel_copy_crc32c_C2
00:36:37.539  accel_crc32c
00:36:37.539  accel_crc32c_C2
00:36:37.539  accel_decmop_full
00:36:37.539  accel_decomp
00:36:37.539  accel_decomp_full_mcore
00:36:37.539  accel_decomp_mcore
00:36:37.539  accel_decomp_mthread
00:36:37.539  accel_deomp_full_mthread
00:36:37.539  accel_dif_functional_tests
00:36:37.539  accel_dif_generate
00:36:37.539  accel_dif_generate_copy
00:36:37.539  accel_dif_verify
00:36:37.539  accel_dualcast
00:36:37.539  accel_fill
00:36:37.539  accel_help
00:36:37.539  accel_missing_filename
00:36:37.539  accel_negative_buffers
00:36:37.539  accel_rpc
00:36:37.539  accel_wrong_workload
00:36:37.539  accel_xor
00:36:37.539  acl
00:36:37.539  alias_rpc
00:36:37.539  allowed
00:36:37.539  app_cmdline
00:36:37.539  app_repeat
00:36:37.539  asan
00:36:37.539  autobuild
00:36:37.539  autobuild_build_doc
00:36:37.539  autobuild_check_dpdk_pci_api
00:36:37.539  autobuild_check_format
00:36:37.539  autobuild_check_so_deps
00:36:37.539  autobuild_external_code
00:36:37.539  autobuild_generated_files_check
00:36:37.539  autobuild_generated_files_check_post_clean
00:36:37.539  autobuild_header_dependency_check
00:36:37.539  autobuild_llvm_precompile
00:36:37.539  autobuild_make_install
00:36:37.539  autobuild_make_uninstall
00:36:37.539  autobuild_ocf_precompile
00:36:37.539  bdev_bounds
00:36:37.539  bdev_crypto_enomem
00:36:37.540  bdev_error
00:36:37.540  bdev_fio
00:36:37.540  bdev_fio_rw_verify
00:36:37.540  bdev_fio_trim
00:36:37.540  bdev_gpt_uuid
00:36:37.540  bdev_hello_world
00:36:37.540  bdev_json_nonarray
00:36:37.540  bdev_json_nonenclosed
00:36:37.540  bdev_nbd
00:36:37.540  bdev_nvme_reset_stuck_adm_cmd
00:36:37.540  bdev_qd_sampling
00:36:37.540  bdev_qos
00:36:37.540  bdev_qos_bw
00:36:37.540  bdev_qos_iops
00:36:37.540  bdev_qos_ro_bw
00:36:37.540  bdev_raid
00:36:37.540  bdev_stat
00:36:37.540  bdev_verify
00:36:37.540  bdev_verify_big_io
00:36:37.540  bdev_write_zeroes
00:36:37.540  bdevperf_config
00:36:37.540  blockdev_crypto_aesni
00:36:37.540  blockdev_crypto_qat
00:36:37.540  blockdev_crypto_sw
00:36:37.540  blockdev_general
00:36:37.540  blockdev_nvme
00:36:37.540  blockdev_nvme_gpt
00:36:37.540  build_rpm_from_gen_spec
00:36:37.540  build_shared_rpm
00:36:37.540  chaining
00:36:37.540  check_header_filenames
00:36:37.540  cmb_copy
00:36:37.540  compress_compdev
00:36:37.540  compress_isal
00:36:37.540  confirm_abi_deps
00:36:37.540  confirm_makefile_deps
00:36:37.540  cpu_locks
00:36:37.540  custom_alloc
00:36:37.540  dd_bs_lt_native_bs
00:36:37.540  dd_copy_to_out_bdev
00:36:37.540  dd_double_input
00:36:37.540  dd_double_output
00:36:37.540  dd_flag_append
00:36:37.540  dd_flag_append_forced_aio
00:36:37.540  dd_flag_directory
00:36:37.540  dd_flag_directory_forced_aio
00:36:37.540  dd_flag_noatime
00:36:37.540  dd_flag_noatime_forced_aio
00:36:37.540  dd_flag_nofollow
00:36:37.540  dd_flag_nofollow_forced_aio
00:36:37.540  dd_flags_misc
00:36:37.540  dd_flags_misc_forced_aio
00:36:37.540  dd_inflate_file
00:36:37.540  dd_invalid_arguments
00:36:37.540  dd_invalid_count
00:36:37.540  dd_invalid_iflag
00:36:37.540  dd_invalid_json
00:36:37.540  dd_invalid_oflag
00:36:37.540  dd_malloc_copy
00:36:37.540  dd_no_input
00:36:37.540  dd_no_output
00:36:37.540  dd_offset_magic
00:36:37.540  dd_rw
00:36:37.540  dd_rw_offset
00:36:37.540  dd_smaller_blocksize
00:36:37.540  dd_sparse_bdev_to_file
00:36:37.540  dd_sparse_file_to_bdev
00:36:37.540  dd_sparse_file_to_file
00:36:37.540  dd_unknown_flag
00:36:37.540  dd_uring_copy
00:36:37.540  dd_wrong_blocksize
00:36:37.540  default_locks
00:36:37.540  default_locks_via_rpc
00:36:37.540  default_setup
00:36:37.540  denied
00:36:37.540  devices
00:36:37.540  dm_mount
00:36:37.540  dma
00:36:37.540  dpdk_governor
00:36:37.540  dpdk_mem_utility
00:36:37.540  driver
00:36:37.540  env
00:36:37.540  env_dpdk_post_init
00:36:37.540  env_mem_callbacks
00:36:37.540  env_memory
00:36:37.540  env_pci
00:36:37.540  env_vtophys
00:36:37.540  even_2G_alloc
00:36:37.540  event
00:36:37.540  event_perf
00:36:37.540  event_reactor
00:36:37.540  event_reactor_perf
00:36:37.540  event_scheduler
00:36:37.540  exit_on_failed_rpc_init
00:36:37.540  external_make_accel_driver_shared
00:36:37.540  external_make_accel_driver_static
00:36:37.540  external_make_accel_module_shared
00:36:37.540  external_make_accel_module_static
00:36:37.540  external_make_hello_bdev_shared_combo
00:36:37.540  external_make_hello_bdev_shared_iso
00:36:37.540  external_make_hello_bdev_static
00:36:37.540  external_make_hello_no_bdev_shared_combo
00:36:37.540  external_make_hello_no_bdev_shared_iso
00:36:37.540  external_make_hello_no_bdev_static
00:36:37.540  external_make_nvme_shared
00:36:37.540  external_make_nvme_static
00:36:37.540  external_run_accel_driver_shared
00:36:37.540  external_run_accel_driver_static
00:36:37.540  external_run_accel_module_shared
00:36:37.540  external_run_accel_module_static
00:36:37.540  external_run_hello_bdev_shared_combo
00:36:37.540  external_run_hello_bdev_shared_iso
00:36:37.540  external_run_hello_bdev_static
00:36:37.540  external_run_hello_no_bdev_shared_combo
00:36:37.540  external_run_hello_no_bdev_shared_iso
00:36:37.540  external_run_hello_no_bdev_static
00:36:37.540  external_run_nvme_shared
00:36:37.540  external_run_nvme_static
00:36:37.540  filesystem_btrfs
00:36:37.540  filesystem_ext4
00:36:37.540  filesystem_in_capsule_btrfs
00:36:37.540  filesystem_in_capsule_ext4
00:36:37.540  filesystem_in_capsule_xfs
00:36:37.540  filesystem_xfs
00:36:37.540  fio_dif_1_default
00:36:37.540  fio_dif_1_multi_subsystems
00:36:37.540  fio_dif_digest
00:36:37.540  fio_dif_rand_params
00:36:37.540  go_rpc
00:36:37.540  guess_driver
00:36:37.540  hugepages
00:36:37.540  idle
00:36:37.540  interrupt_mode
00:36:37.540  ioat
00:36:37.540  ioat_perf
00:36:37.540  ioat_verify
00:36:37.540  is_zoned
00:36:37.540  iscsi_tgt_calsoft
00:36:37.540  iscsi_tgt_filesystem
00:36:37.540  iscsi_tgt_filesystem_btrfs
00:36:37.540  iscsi_tgt_filesystem_ext4
00:36:37.540  iscsi_tgt_filesystem_xfs
00:36:37.540  iscsi_tgt_fio
00:36:37.540  iscsi_tgt_ip_migration
00:36:37.540  iscsi_tgt_iscsi_lvol
00:36:37.540  iscsi_tgt_qos
00:36:37.540  iscsi_tgt_reset
00:36:37.540  iscsi_tgt_rpc_config
00:36:37.540  iscsi_tgt_sock
00:36:37.540  json_config
00:36:37.540  json_config_extra_key
00:36:37.540  kernel_target_abort
00:36:37.540  keyring_file
00:36:37.540  locking_app_on_locked_coremask
00:36:37.540  locking_app_on_unlocked_coremask
00:36:37.540  locking_overlapped_coremask
00:36:37.540  locking_overlapped_coremask_via_rpc
00:36:37.540  lvs_grow_clean
00:36:37.540  lvs_grow_dirty
00:36:37.540  make
00:36:37.540  no_shrink_alloc
00:36:37.540  non_locking_app_on_locked_coremask
00:36:37.540  nvme
00:36:37.540  nvme_arbitration
00:36:37.540  nvme_cli_cuse
00:36:37.540  nvme_cli_plugin
00:36:37.540  nvme_cmb
00:36:37.540  nvme_cuse
00:36:37.540  nvme_cuse_app
00:36:37.540  nvme_cuse_rpc
00:36:37.540  nvme_doorbell_aers
00:36:37.540  nvme_e2edp
00:36:37.540  nvme_err_injection
00:36:37.540  nvme_fio
00:36:37.540  nvme_hello_world
00:36:37.540  nvme_identify
00:36:37.540  nvme_mount
00:36:37.540  nvme_multi_aen
00:36:37.540  nvme_multi_secondary
00:36:37.540  nvme_ns_manage_cuse
00:36:37.540  nvme_overhead
00:36:37.540  nvme_perf
00:36:37.540  nvme_pmr
00:36:37.540  nvme_pmr_persistence
00:36:37.540  nvme_reserve
00:36:37.540  nvme_reset
00:36:37.540  nvme_rpc
00:36:37.540  nvme_rpc_timeouts
00:36:37.540  nvme_scc
00:36:37.540  nvme_sgl
00:36:37.540  nvme_simple_copy
00:36:37.540  nvme_single_aen
00:36:37.540  nvme_smartctl_cuse
00:36:37.540  nvme_startup
00:36:37.540  nvme_zns
00:36:37.540  nvmf_abort
00:36:37.540  nvmf_abort_qd_sizes
00:36:37.540  nvmf_aer
00:36:37.540  nvmf_async_init
00:36:37.540  nvmf_auth
00:36:37.540  nvmf_bdev_io_wait
00:36:37.540  nvmf_bdevio
00:36:37.540  nvmf_bdevio_no_huge
00:36:37.540  nvmf_bdevperf
00:36:37.540  nvmf_connect_disconnect
00:36:37.540  nvmf_connect_stress
00:36:37.540  nvmf_delete_subsystem
00:36:37.540  nvmf_device_removal
00:36:37.540  nvmf_device_removal_pci_remove
00:36:37.540  nvmf_device_removal_pci_remove_no_srq
00:36:37.540  nvmf_dif
00:36:37.540  nvmf_digest
00:36:37.540  nvmf_digest_clean
00:36:37.540  nvmf_digest_error
00:36:37.540  nvmf_discovery
00:36:37.540  nvmf_discovery_remove_ifc
00:36:37.540  nvmf_example
00:36:37.540  nvmf_failover
00:36:37.540  nvmf_filesystem
00:36:37.540  nvmf_filesystem_in_capsule
00:36:37.540  nvmf_filesystem_no_in_capsule
00:36:37.540  nvmf_fio_host
00:36:37.540  nvmf_fio_target
00:36:37.540  nvmf_fips
00:36:37.540  nvmf_fused_ordering
00:36:37.540  nvmf_host_management
00:36:37.540  nvmf_identify
00:36:37.540  nvmf_identify_kernel_target
00:36:37.540  nvmf_identify_passthru
00:36:37.540  nvmf_invalid
00:36:37.540  nvmf_lvol
00:36:37.540  nvmf_lvs_grow
00:36:37.540  nvmf_mdns_discovery
00:36:37.540  nvmf_multicontroller
00:36:37.540  nvmf_multipath
00:36:37.540  nvmf_multitarget
00:36:37.540  nvmf_nmic
00:36:37.540  nvmf_ns_hotplug_stress
00:36:37.540  nvmf_ns_masking
00:36:37.540  nvmf_nvme_cli
00:36:37.540  nvmf_perf
00:36:37.540  nvmf_perf_adq
00:36:37.540  nvmf_queue_depth
00:36:37.540  nvmf_rdma
00:36:37.540  nvmf_referrals
00:36:37.540  nvmf_rpc
00:36:37.540  nvmf_shutdown
00:36:37.540  nvmf_shutdown_tc1
00:36:37.540  nvmf_shutdown_tc2
00:36:37.540  nvmf_shutdown_tc3
00:36:37.540  nvmf_srq_overwhelm
00:36:37.540  nvmf_target_disconnect
00:36:37.540  nvmf_target_disconnect_tc1
00:36:37.540  nvmf_target_disconnect_tc2
00:36:37.540  nvmf_target_disconnect_tc3
00:36:37.540  nvmf_tcp
00:36:37.540  nvmf_timeout
00:36:37.540  nvmf_tls
00:36:37.540  nvmf_vfio_user
00:36:37.540  nvmf_vfio_user_fuzz
00:36:37.540  nvmf_vfio_user_nvme_compliance
00:36:37.540  nvmf_zcopy
00:36:37.540  ocf
00:36:37.540  ocf_bdevperf_iotypes
00:36:37.540  ocf_configuration_change
00:36:37.540  ocf_create_destruct
00:36:37.540  ocf_fio_modes
00:36:37.540  ocf_flush
00:36:37.540  ocf_multicore
00:36:37.540  ocf_remove
00:36:37.540  ocf_stats
00:36:37.540  odd_alloc
00:36:37.540  packaging
00:36:37.540  per_node_1G_alloc
00:36:37.540  raid0_resize_test
00:36:37.540  raid_function_test_concat
00:36:37.540  raid_function_test_raid0
00:36:37.541  raid_rebuild_test
00:36:37.541  raid_rebuild_test_io
00:36:37.541  raid_rebuild_test_sb
00:36:37.541  raid_rebuild_test_sb_io
00:36:37.541  raid_state_function_test
00:36:37.541  raid_state_function_test_sb
00:36:37.541  raid_superblock_test
00:36:37.541  reactor_set_interrupt
00:36:37.541  reap_unregistered_poller
00:36:37.541  rpc
00:36:37.541  rpc_client
00:36:37.541  rpc_daemon_integrity
00:36:37.541  rpc_integrity
00:36:37.541  rpc_plugins
00:36:37.541  rpc_trace_cmd_test
00:36:37.541  rpm_packaging
00:36:37.541  scheduler
00:36:37.541  scheduler_create_thread
00:36:37.541  setup.sh
00:36:37.541  skip_rpc
00:36:37.541  skip_rpc_with_delay
00:36:37.541  skip_rpc_with_json
00:36:37.541  spdk_dd
00:36:37.541  spdk_dd_basic_rw
00:36:37.541  spdk_dd_bdev_to_bdev
00:36:37.541  spdk_dd_malloc
00:36:37.541  spdk_dd_negative
00:36:37.541  spdk_dd_posix
00:36:37.541  spdk_dd_sparse
00:36:37.541  spdk_dd_uring
00:36:37.541  spdk_target_abort
00:36:37.541  spdkcli_nvmf_rdma
00:36:37.541  spdkcli_nvmf_tcp
00:36:37.541  spdkcli_tcp
00:36:37.541  spdkcli_virtio
00:36:37.541  sw_hotplug
00:36:37.541  thread
00:36:37.541  thread_poller_perf
00:36:37.541  thread_spdk_lock
00:36:37.541  ubsan
00:36:37.541  unittest
00:36:37.541  unittest_accel
00:36:37.541  unittest_bdev
00:36:37.541  unittest_bdev_compress
00:36:37.541  unittest_bdev_crypto
00:36:37.541  unittest_bdev_raid5f
00:36:37.541  unittest_blob_blobfs
00:36:37.541  unittest_build
00:36:37.541  unittest_dma
00:36:37.541  unittest_dpdk_compressdev
00:36:37.541  unittest_event
00:36:37.541  unittest_ftl
00:36:37.541  unittest_idxd_user
00:36:37.541  unittest_include
00:36:37.541  unittest_init
00:36:37.541  unittest_ioat
00:36:37.541  unittest_iobuf
00:36:37.541  unittest_iscsi
00:36:37.541  unittest_json
00:36:37.541  unittest_keyring
00:36:37.541  unittest_lib_reduce
00:36:37.541  unittest_log
00:36:37.541  unittest_lvol
00:36:37.541  unittest_notify
00:36:37.541  unittest_nvme
00:36:37.541  unittest_nvme_cuse
00:36:37.541  unittest_nvme_rdma
00:36:37.541  unittest_nvmf
00:36:37.541  unittest_nvmf_rdma
00:36:37.541  unittest_nvmf_transport
00:36:37.541  unittest_nvmf_vfio_user
00:36:37.541  unittest_pci_event
00:36:37.541  unittest_rdma
00:36:37.541  unittest_rpc
00:36:37.541  unittest_scsi
00:36:37.541  unittest_sock
00:36:37.541  unittest_thread
00:36:37.541  unittest_util
00:36:37.541  unittest_vhost
00:36:37.541  valgrind
00:36:37.541  version
00:36:37.541  vhost_blockdev
00:36:37.541  vhost_fuzz
00:36:37.541  vhost_shared
00:36:37.541  zoned_fio
00:36:37.541  
00:36:37.541  
00:36:37.541  -----Tests Missing From Build------
00:36:37.541  accel_scan_dsa_modules
00:36:37.541  accel_scan_iaa_modules
00:36:37.541  balanced
00:36:37.541  blob_io_wait
00:36:37.541  blobfs
00:36:37.541  blobfs_mkfs
00:36:37.541  blobstore
00:36:37.541  blobstore_grow
00:36:37.541  blockdev_raid5f
00:36:37.541  blockdev_rbd
00:36:37.541  blockdev_xnvme
00:36:37.541  build_native_dpdk
00:36:37.541  build_shared_native_dpdk_rpm
00:36:37.541  build_shared_rpm_with_rpmed_dpdk
00:36:37.541  busy
00:36:37.541  core_load
00:36:37.541  ftl
00:36:37.541  ftl_bdevperf
00:36:37.541  ftl_dirty_shutdown
00:36:37.541  ftl_fio_basic
00:36:37.541  ftl_fio_extended
00:36:37.541  ftl_fio_nightly
00:36:37.541  ftl_restore
00:36:37.541  ftl_restore_fast
00:36:37.541  ftl_trim
00:36:37.541  ftl_upgrade_shutdown
00:36:37.541  ftl_write_after_write
00:36:37.541  hello_blob
00:36:37.541  iscsi_tgt
00:36:37.541  iscsi_tgt_bdev_io_wait
00:36:37.541  iscsi_tgt_data_digest
00:36:37.541  iscsi_tgt_digest
00:36:37.541  iscsi_tgt_digests
00:36:37.541  iscsi_tgt_ext4test
00:36:37.541  iscsi_tgt_fio_remote_nvme
00:36:37.541  iscsi_tgt_fuzz
00:36:37.541  iscsi_tgt_initiator
00:36:37.541  iscsi_tgt_login_redirection
00:36:37.541  iscsi_tgt_multiconnection
00:36:37.541  iscsi_tgt_rbd
00:36:37.541  iscsi_tgt_resize
00:36:37.541  iscsi_tgt_trace_record
00:36:37.541  keyring_linux
00:36:37.541  llvm_fuzz
00:36:37.541  lvol
00:36:37.541  lvol_basic
00:36:37.541  lvol_esnap
00:36:37.541  lvol_external_snapshot
00:36:37.541  lvol_hotremove
00:36:37.541  lvol_provisioning
00:36:37.541  lvol_rename
00:36:37.541  lvol_resize
00:36:37.541  lvol_snapshot_clone
00:36:37.541  lvol_tasting
00:36:37.541  nvme_boot_partition
00:36:37.541  nvme_bp
00:36:37.541  nvme_fdp
00:36:37.541  nvme_flexible_data_placement
00:36:37.541  nvme_opal
00:36:37.541  nvme_opal_bdevio
00:36:37.541  nvme_opal_bdevperf
00:36:37.541  nvme_opal_spdk_tgt
00:36:37.541  nvme_xnvme
00:36:37.541  nvmf_digest_dsa_initiator
00:36:37.541  nvmf_digest_dsa_target
00:36:37.541  nvmf_fc
00:36:37.541  nvmf_fuzz
00:36:37.541  nvmf_initiator_timeout
00:36:37.541  nvmf_multiconnection
00:36:37.541  nvmf_vhost
00:36:37.541  raid5f_rebuild_test
00:36:37.541  raid5f_rebuild_test_sb
00:36:37.541  raid5f_state_function_test
00:36:37.541  raid5f_state_function_test_sb
00:36:37.541  raid5f_superblock_test
00:36:37.541  rocksdb
00:36:37.541  rocksdb_insert
00:36:37.541  rocksdb_overwrite
00:36:37.541  rocksdb_randread
00:36:37.541  rocksdb_readwrite
00:36:37.541  rocksdb_writesync
00:36:37.541  scanbuild_make
00:36:37.541  sma
00:36:37.541  sma_crypto
00:36:37.541  sma_discovery
00:36:37.541  sma_nvmf_tcp
00:36:37.541  sma_plugins
00:36:37.541  sma_qos
00:36:37.541  sma_vfiouser_qemu
00:36:37.541  sma_vhost
00:36:37.541  spdkcli_iscsi
00:36:37.541  spdkcli_nvmf_fc
00:36:37.541  spdkcli_raid
00:36:37.541  spdkcli_rbd
00:36:37.541  spdkcli_vhost
00:36:37.541  test_bdev_lvol_delete_lvstore_with_clones
00:36:37.541  test_bdev_lvol_delete_ordering
00:36:37.541  test_clone_decouple_parent
00:36:37.541  test_clone_inflate
00:36:37.541  test_clone_snapshot_relations
00:36:37.541  test_construct_lvol
00:36:37.541  test_construct_lvol_alias_conflict
00:36:37.541  test_construct_lvol_fio_clear_method_none
00:36:37.541  test_construct_lvol_fio_clear_method_unmap
00:36:37.541  test_construct_lvol_full_lvs
00:36:37.541  test_construct_lvol_inexistent_lvs
00:36:37.541  test_construct_lvols_conflict_alias
00:36:37.541  test_construct_lvs
00:36:37.541  test_construct_lvs_clear_methods
00:36:37.541  test_construct_lvs_conflict_alias
00:36:37.541  test_construct_lvs_different_cluster_size
00:36:37.541  test_construct_lvs_nonexistent_bdev
00:36:37.541  test_construct_multi_lvols
00:36:37.541  test_construct_nested_lvol
00:36:37.541  test_construct_two_lvs_on_the_same_bdev
00:36:37.541  test_create_multi_ublk
00:36:37.541  test_create_snapshot_of_snapshot
00:36:37.541  test_create_snapshot_with_io
00:36:37.541  test_create_ublk
00:36:37.541  test_delete_lvol_store_persistent_positive
00:36:37.541  test_delete_snapshot_with_clone
00:36:37.541  test_delete_snapshot_with_snapshot
00:36:37.541  test_destroy_after_bdev_lvol_resize_positive
00:36:37.541  test_esnap_clones
00:36:37.541  test_esnap_late_arrival
00:36:37.541  test_esnap_reload
00:36:37.541  test_esnap_remove_degraded
00:36:37.541  test_hotremove_lvol_store
00:36:37.541  test_hotremove_lvol_store_base
00:36:37.541  test_hotremove_lvol_store_base_with_lvols
00:36:37.541  test_hotremove_lvol_store_multiple_lvols
00:36:37.541  test_lvol_bdev_readonly
00:36:37.541  test_lvol_list
00:36:37.541  test_lvol_rename_negative
00:36:37.541  test_rename_lvs_negative
00:36:37.541  test_rename_positive
00:36:37.541  test_resize_lvol
00:36:37.541  test_resize_lvol_negative
00:36:37.541  test_resize_lvol_with_io_traffic
00:36:37.541  test_save_ublk_config
00:36:37.541  test_sigterm
00:36:37.541  test_snapshot_compare_with_lvol_bdev
00:36:37.541  test_tasting
00:36:37.541  test_thin_lvol_check_integrity
00:36:37.541  test_thin_lvol_check_space
00:36:37.541  test_thin_lvol_check_zeroes
00:36:37.541  test_thin_lvol_resize
00:36:37.541  test_thin_overprovisioning
00:36:37.541  test_unregister_lvol_bdev
00:36:37.541  ublk
00:36:37.541  ublk_recovery
00:36:37.541  unittest_nvmf_fc
00:36:37.541  unittest_nvmf_fc_ls
00:36:37.541  vfio_fuzz
00:36:37.541  vfio_user_nvme_fio
00:36:37.541  vfio_user_nvme_restart_vm
00:36:37.541  vfio_user_qemu
00:36:37.541  vfio_user_virtio_bdevperf
00:36:37.541  vfio_user_virtio_blk_restart_vm
00:36:37.541  vfio_user_virtio_scsi_restart_vm
00:36:37.541  vhost
00:36:37.541  vhost_blk_cores_2ctrl
00:36:37.541  vhost_blk_fs_integrity
00:36:37.541  vhost_blk_hot_remove
00:36:37.541  vhost_blk_integrity
00:36:37.541  vhost_blk_lvol_integrity
00:36:37.541  vhost_blk_packed_ring_integrity
00:36:37.541  vhost_boot
00:36:37.541  vhost_hotplug
00:36:37.541  vhost_migration
00:36:37.541  vhost_migration_tc1
00:36:37.541  vhost_migration_tc2
00:36:37.541  vhost_negative
00:36:37.541  vhost_readonly
00:36:37.541  vhost_scsi_cores_2ctrl
00:36:37.541  vhost_scsi_fs_integrity
00:36:37.541  vhost_scsi_hot_remove
00:36:37.541  vhost_scsi_integrity
00:36:37.541  vhost_scsi_lvol_integrity
00:36:37.541  vmd
00:36:37.541  vmd_bdev_svc
00:36:37.541  vmd_fio
00:36:37.541  vmd_hello_world
00:36:37.541  vmd_identify
00:36:37.541  vmd_perf
00:36:37.541  xnvme_bdevperf
00:36:37.541  xnvme_to_malloc_dd_copy
00:36:37.541  
00:36:37.541  
00:36:37.541  -----Tests Missing ASAN------
00:36:37.541  accel_cdev_comp
00:36:37.541  accel_cdev_decmop_full
00:36:37.541  accel_cdev_decomp
00:36:37.541  accel_cdev_decomp_full_mcore
00:36:37.541  accel_cdev_decomp_mcore
00:36:37.542  accel_cdev_decomp_mthread
00:36:37.542  accel_cdev_deomp_full_mthread
00:36:37.542  accel_comp
00:36:37.542  accel_decmop_full
00:36:37.542  accel_decomp
00:36:37.542  accel_decomp_full_mcore
00:36:37.542  accel_decomp_mcore
00:36:37.542  accel_decomp_mthread
00:36:37.542  accel_deomp_full_mthread
00:36:37.542  accel_scan_dsa_modules
00:36:37.542  accel_scan_iaa_modules
00:36:37.542  app_repeat
00:36:37.542  autobuild
00:36:37.542  autobuild_build_doc
00:36:37.542  autobuild_check_dpdk_pci_api
00:36:37.542  autobuild_check_format
00:36:37.542  autobuild_check_so_deps
00:36:37.542  autobuild_external_code
00:36:37.542  autobuild_generated_files_check
00:36:37.542  autobuild_generated_files_check_post_clean
00:36:37.542  autobuild_header_dependency_check
00:36:37.542  autobuild_llvm_precompile
00:36:37.542  autobuild_make_install
00:36:37.542  autobuild_make_uninstall
00:36:37.542  autobuild_ocf_precompile
00:36:37.542  balanced
00:36:37.542  bdev_crypto_enomem
00:36:37.542  bdev_gpt_uuid
00:36:37.542  bdev_nvme_reset_stuck_adm_cmd
00:36:37.542  blob_io_wait
00:36:37.542  blobfs
00:36:37.542  blobfs_mkfs
00:36:37.542  blobstore
00:36:37.542  blobstore_grow
00:36:37.542  blockdev_crypto_aesni
00:36:37.542  blockdev_crypto_qat
00:36:37.542  blockdev_crypto_sw
00:36:37.542  blockdev_nvme
00:36:37.542  blockdev_nvme_gpt
00:36:37.542  blockdev_raid5f
00:36:37.542  blockdev_rbd
00:36:37.542  blockdev_xnvme
00:36:37.542  build_native_dpdk
00:36:37.542  build_rpm_from_gen_spec
00:36:37.542  build_shared_native_dpdk_rpm
00:36:37.542  build_shared_rpm
00:36:37.542  build_shared_rpm_with_rpmed_dpdk
00:36:37.542  busy
00:36:37.542  chaining
00:36:37.542  check_header_filenames
00:36:37.542  cmb_copy
00:36:37.542  compress_compdev
00:36:37.542  compress_isal
00:36:37.542  confirm_abi_deps
00:36:37.542  confirm_makefile_deps
00:36:37.542  core_load
00:36:37.542  dd_uring_copy
00:36:37.542  dma
00:36:37.542  dpdk_governor
00:36:37.542  external_make_accel_driver_shared
00:36:37.542  external_make_accel_driver_static
00:36:37.542  external_make_accel_module_shared
00:36:37.542  external_make_accel_module_static
00:36:37.542  external_make_hello_bdev_shared_combo
00:36:37.542  external_make_hello_bdev_shared_iso
00:36:37.542  external_make_hello_bdev_static
00:36:37.542  external_make_hello_no_bdev_shared_combo
00:36:37.542  external_make_hello_no_bdev_shared_iso
00:36:37.542  external_make_hello_no_bdev_static
00:36:37.542  external_make_nvme_shared
00:36:37.542  external_make_nvme_static
00:36:37.542  external_run_accel_driver_shared
00:36:37.542  external_run_accel_driver_static
00:36:37.542  external_run_accel_module_shared
00:36:37.542  external_run_accel_module_static
00:36:37.542  external_run_hello_bdev_shared_combo
00:36:37.542  external_run_hello_bdev_shared_iso
00:36:37.542  external_run_hello_bdev_static
00:36:37.542  external_run_hello_no_bdev_shared_combo
00:36:37.542  external_run_hello_no_bdev_shared_iso
00:36:37.542  external_run_hello_no_bdev_static
00:36:37.542  external_run_nvme_shared
00:36:37.542  external_run_nvme_static
00:36:37.542  filesystem_btrfs
00:36:37.542  filesystem_ext4
00:36:37.542  filesystem_in_capsule_btrfs
00:36:37.542  filesystem_in_capsule_ext4
00:36:37.542  filesystem_in_capsule_xfs
00:36:37.542  filesystem_xfs
00:36:37.542  fio_dif_1_default
00:36:37.542  fio_dif_1_multi_subsystems
00:36:37.542  fio_dif_digest
00:36:37.542  fio_dif_rand_params
00:36:37.542  ftl
00:36:37.542  ftl_bdevperf
00:36:37.542  ftl_dirty_shutdown
00:36:37.542  ftl_fio_basic
00:36:37.542  ftl_fio_extended
00:36:37.542  ftl_fio_nightly
00:36:37.542  ftl_restore
00:36:37.542  ftl_restore_fast
00:36:37.542  ftl_trim
00:36:37.542  ftl_upgrade_shutdown
00:36:37.542  ftl_write_after_write
00:36:37.542  go_rpc
00:36:37.542  hello_blob
00:36:37.542  idle
00:36:37.542  interrupt_mode
00:36:37.542  ioat
00:36:37.542  ioat_perf
00:36:37.542  ioat_verify
00:36:37.542  is_zoned
00:36:37.542  iscsi_tgt
00:36:37.542  iscsi_tgt_bdev_io_wait
00:36:37.542  iscsi_tgt_calsoft
00:36:37.542  iscsi_tgt_data_digest
00:36:37.542  iscsi_tgt_digest
00:36:37.542  iscsi_tgt_digests
00:36:37.542  iscsi_tgt_ext4test
00:36:37.542  iscsi_tgt_filesystem
00:36:37.542  iscsi_tgt_filesystem_btrfs
00:36:37.542  iscsi_tgt_filesystem_ext4
00:36:37.542  iscsi_tgt_filesystem_xfs
00:36:37.542  iscsi_tgt_fio
00:36:37.542  iscsi_tgt_fio_remote_nvme
00:36:37.542  iscsi_tgt_fuzz
00:36:37.542  iscsi_tgt_initiator
00:36:37.542  iscsi_tgt_ip_migration
00:36:37.542  iscsi_tgt_iscsi_lvol
00:36:37.542  iscsi_tgt_login_redirection
00:36:37.542  iscsi_tgt_multiconnection
00:36:37.542  iscsi_tgt_qos
00:36:37.542  iscsi_tgt_rbd
00:36:37.542  iscsi_tgt_reset
00:36:37.542  iscsi_tgt_resize
00:36:37.542  iscsi_tgt_rpc_config
00:36:37.542  iscsi_tgt_sock
00:36:37.542  iscsi_tgt_trace_record
00:36:37.542  kernel_target_abort
00:36:37.542  keyring_file
00:36:37.542  keyring_linux
00:36:37.542  llvm_fuzz
00:36:37.542  lvol
00:36:37.542  lvol_basic
00:36:37.542  lvol_esnap
00:36:37.542  lvol_external_snapshot
00:36:37.542  lvol_hotremove
00:36:37.542  lvol_provisioning
00:36:37.542  lvol_rename
00:36:37.542  lvol_resize
00:36:37.542  lvol_snapshot_clone
00:36:37.542  lvol_tasting
00:36:37.542  lvs_grow_clean
00:36:37.542  lvs_grow_dirty
00:36:37.542  nvme
00:36:37.542  nvme_arbitration
00:36:37.542  nvme_boot_partition
00:36:37.542  nvme_bp
00:36:37.542  nvme_cli_cuse
00:36:37.542  nvme_cli_plugin
00:36:37.542  nvme_cmb
00:36:37.542  nvme_cuse
00:36:37.542  nvme_cuse_app
00:36:37.542  nvme_cuse_rpc
00:36:37.542  nvme_doorbell_aers
00:36:37.542  nvme_e2edp
00:36:37.542  nvme_err_injection
00:36:37.542  nvme_fdp
00:36:37.542  nvme_fio
00:36:37.542  nvme_flexible_data_placement
00:36:37.542  nvme_hello_world
00:36:37.542  nvme_identify
00:36:37.542  nvme_multi_aen
00:36:37.542  nvme_multi_secondary
00:36:37.542  nvme_ns_manage_cuse
00:36:37.542  nvme_opal
00:36:37.542  nvme_opal_bdevio
00:36:37.542  nvme_opal_bdevperf
00:36:37.542  nvme_opal_spdk_tgt
00:36:37.542  nvme_overhead
00:36:37.542  nvme_perf
00:36:37.542  nvme_pmr
00:36:37.542  nvme_pmr_persistence
00:36:37.542  nvme_reserve
00:36:37.542  nvme_reset
00:36:37.542  nvme_rpc
00:36:37.542  nvme_rpc_timeouts
00:36:37.542  nvme_scc
00:36:37.542  nvme_sgl
00:36:37.542  nvme_simple_copy
00:36:37.542  nvme_single_aen
00:36:37.542  nvme_smartctl_cuse
00:36:37.542  nvme_startup
00:36:37.542  nvme_xnvme
00:36:37.542  nvme_zns
00:36:37.542  nvmf_abort
00:36:37.542  nvmf_abort_qd_sizes
00:36:37.542  nvmf_aer
00:36:37.542  nvmf_async_init
00:36:37.542  nvmf_auth
00:36:37.542  nvmf_bdev_io_wait
00:36:37.542  nvmf_bdevio
00:36:37.542  nvmf_bdevio_no_huge
00:36:37.542  nvmf_bdevperf
00:36:37.542  nvmf_connect_disconnect
00:36:37.542  nvmf_connect_stress
00:36:37.542  nvmf_delete_subsystem
00:36:37.542  nvmf_device_removal
00:36:37.542  nvmf_device_removal_pci_remove
00:36:37.542  nvmf_device_removal_pci_remove_no_srq
00:36:37.542  nvmf_dif
00:36:37.542  nvmf_digest
00:36:37.542  nvmf_digest_clean
00:36:37.542  nvmf_digest_dsa_initiator
00:36:37.542  nvmf_digest_dsa_target
00:36:37.542  nvmf_digest_error
00:36:37.542  nvmf_discovery
00:36:37.542  nvmf_discovery_remove_ifc
00:36:37.542  nvmf_example
00:36:37.542  nvmf_failover
00:36:37.542  nvmf_fc
00:36:37.542  nvmf_filesystem
00:36:37.542  nvmf_filesystem_in_capsule
00:36:37.542  nvmf_filesystem_no_in_capsule
00:36:37.542  nvmf_fio_host
00:36:37.542  nvmf_fio_target
00:36:37.542  nvmf_fips
00:36:37.542  nvmf_fused_ordering
00:36:37.542  nvmf_fuzz
00:36:37.542  nvmf_host_management
00:36:37.542  nvmf_identify
00:36:37.542  nvmf_identify_kernel_target
00:36:37.542  nvmf_identify_passthru
00:36:37.542  nvmf_initiator_timeout
00:36:37.542  nvmf_invalid
00:36:37.542  nvmf_lvol
00:36:37.542  nvmf_lvs_grow
00:36:37.542  nvmf_mdns_discovery
00:36:37.542  nvmf_multiconnection
00:36:37.542  nvmf_multicontroller
00:36:37.542  nvmf_multipath
00:36:37.542  nvmf_multitarget
00:36:37.542  nvmf_nmic
00:36:37.542  nvmf_ns_hotplug_stress
00:36:37.542  nvmf_ns_masking
00:36:37.542  nvmf_nvme_cli
00:36:37.542  nvmf_perf
00:36:37.542  nvmf_perf_adq
00:36:37.542  nvmf_queue_depth
00:36:37.542  nvmf_rdma
00:36:37.542  nvmf_referrals
00:36:37.542  nvmf_rpc
00:36:37.542  nvmf_shutdown
00:36:37.542  nvmf_shutdown_tc1
00:36:37.542  nvmf_shutdown_tc2
00:36:37.542  nvmf_shutdown_tc3
00:36:37.542  nvmf_srq_overwhelm
00:36:37.542  nvmf_target_disconnect
00:36:37.542  nvmf_target_disconnect_tc1
00:36:37.542  nvmf_target_disconnect_tc2
00:36:37.542  nvmf_target_disconnect_tc3
00:36:37.542  nvmf_tcp
00:36:37.542  nvmf_timeout
00:36:37.542  nvmf_tls
00:36:37.542  nvmf_vfio_user
00:36:37.542  nvmf_vfio_user_fuzz
00:36:37.542  nvmf_vfio_user_nvme_compliance
00:36:37.542  nvmf_vhost
00:36:37.542  nvmf_zcopy
00:36:37.542  ocf
00:36:37.542  ocf_bdevperf_iotypes
00:36:37.542  ocf_configuration_change
00:36:37.542  ocf_create_destruct
00:36:37.542  ocf_fio_modes
00:36:37.542  ocf_flush
00:36:37.542  ocf_multicore
00:36:37.542  ocf_remove
00:36:37.542  ocf_stats
00:36:37.542  packaging
00:36:37.542  raid5f_rebuild_test
00:36:37.542  raid5f_rebuild_test_sb
00:36:37.543  raid5f_state_function_test
00:36:37.543  raid5f_state_function_test_sb
00:36:37.543  raid5f_superblock_test
00:36:37.543  raid_function_test_concat
00:36:37.543  raid_function_test_raid0
00:36:37.543  raid_rebuild_test
00:36:37.543  raid_rebuild_test_io
00:36:37.543  raid_rebuild_test_sb
00:36:37.543  raid_rebuild_test_sb_io
00:36:37.543  rocksdb
00:36:37.543  rocksdb_insert
00:36:37.543  rocksdb_overwrite
00:36:37.543  rocksdb_randread
00:36:37.543  rocksdb_readwrite
00:36:37.543  rocksdb_writesync
00:36:37.543  rpm_packaging
00:36:37.543  scanbuild_make
00:36:37.543  scheduler
00:36:37.543  sma
00:36:37.543  sma_crypto
00:36:37.543  sma_discovery
00:36:37.543  sma_nvmf_tcp
00:36:37.543  sma_plugins
00:36:37.543  sma_qos
00:36:37.543  sma_vfiouser_qemu
00:36:37.543  sma_vhost
00:36:37.543  spdk_dd_uring
00:36:37.543  spdk_target_abort
00:36:37.543  spdkcli_iscsi
00:36:37.543  spdkcli_nvmf_fc
00:36:37.543  spdkcli_nvmf_rdma
00:36:37.543  spdkcli_nvmf_tcp
00:36:37.543  spdkcli_raid
00:36:37.543  spdkcli_rbd
00:36:37.543  spdkcli_vhost
00:36:37.543  spdkcli_virtio
00:36:37.543  sw_hotplug
00:36:37.543  test_bdev_lvol_delete_lvstore_with_clones
00:36:37.543  test_bdev_lvol_delete_ordering
00:36:37.543  test_clone_decouple_parent
00:36:37.543  test_clone_inflate
00:36:37.543  test_clone_snapshot_relations
00:36:37.543  test_construct_lvol
00:36:37.543  test_construct_lvol_alias_conflict
00:36:37.543  test_construct_lvol_fio_clear_method_none
00:36:37.543  test_construct_lvol_fio_clear_method_unmap
00:36:37.543  test_construct_lvol_full_lvs
00:36:37.543  test_construct_lvol_inexistent_lvs
00:36:37.543  test_construct_lvols_conflict_alias
00:36:37.543  test_construct_lvs
00:36:37.543  test_construct_lvs_clear_methods
00:36:37.543  test_construct_lvs_conflict_alias
00:36:37.543  test_construct_lvs_different_cluster_size
00:36:37.543  test_construct_lvs_nonexistent_bdev
00:36:37.543  test_construct_multi_lvols
00:36:37.543  test_construct_nested_lvol
00:36:37.543  test_construct_two_lvs_on_the_same_bdev
00:36:37.543  test_create_multi_ublk
00:36:37.543  test_create_snapshot_of_snapshot
00:36:37.543  test_create_snapshot_with_io
00:36:37.543  test_create_ublk
00:36:37.543  test_delete_lvol_store_persistent_positive
00:36:37.543  test_delete_snapshot_with_clone
00:36:37.543  test_delete_snapshot_with_snapshot
00:36:37.543  test_destroy_after_bdev_lvol_resize_positive
00:36:37.543  test_esnap_clones
00:36:37.543  test_esnap_late_arrival
00:36:37.543  test_esnap_reload
00:36:37.543  test_esnap_remove_degraded
00:36:37.543  test_hotremove_lvol_store
00:36:37.543  test_hotremove_lvol_store_base
00:36:37.543  test_hotremove_lvol_store_base_with_lvols
00:36:37.543  test_hotremove_lvol_store_multiple_lvols
00:36:37.543  test_lvol_bdev_readonly
00:36:37.543  test_lvol_list
00:36:37.543  test_lvol_rename_negative
00:36:37.543  test_rename_lvs_negative
00:36:37.543  test_rename_positive
00:36:37.543  test_resize_lvol
00:36:37.543  test_resize_lvol_negative
00:36:37.543  test_resize_lvol_with_io_traffic
00:36:37.543  test_save_ublk_config
00:36:37.543  test_sigterm
00:36:37.543  test_snapshot_compare_with_lvol_bdev
00:36:37.543  test_tasting
00:36:37.543  test_thin_lvol_check_integrity
00:36:37.543  test_thin_lvol_check_space
00:36:37.543  test_thin_lvol_check_zeroes
00:36:37.543  test_thin_lvol_resize
00:36:37.543  test_thin_overprovisioning
00:36:37.543  test_unregister_lvol_bdev
00:36:37.543  ublk
00:36:37.543  ublk_recovery
00:36:37.543  unittest_bdev_compress
00:36:37.543  unittest_bdev_crypto
00:36:37.543  unittest_dpdk_compressdev
00:36:37.543  unittest_lib_reduce
00:36:37.543  unittest_nvmf_fc
00:36:37.543  unittest_nvmf_fc_ls
00:36:37.543  unittest_nvmf_vfio_user
00:36:37.543  valgrind
00:36:37.543  vfio_fuzz
00:36:37.543  vfio_user_nvme_fio
00:36:37.543  vfio_user_nvme_restart_vm
00:36:37.543  vfio_user_qemu
00:36:37.543  vfio_user_virtio_bdevperf
00:36:37.543  vfio_user_virtio_blk_restart_vm
00:36:37.543  vfio_user_virtio_scsi_restart_vm
00:36:37.543  vhost
00:36:37.543  vhost_blk_cores_2ctrl
00:36:37.543  vhost_blk_fs_integrity
00:36:37.543  vhost_blk_hot_remove
00:36:37.543  vhost_blk_integrity
00:36:37.543  vhost_blk_lvol_integrity
00:36:37.543  vhost_blk_packed_ring_integrity
00:36:37.543  vhost_blockdev
00:36:37.543  vhost_boot
00:36:37.543  vhost_fuzz
00:36:37.543  vhost_hotplug
00:36:37.543  vhost_migration
00:36:37.543  vhost_migration_tc1
00:36:37.543  vhost_migration_tc2
00:36:37.543  vhost_negative
00:36:37.543  vhost_readonly
00:36:37.543  vhost_scsi_cores_2ctrl
00:36:37.543  vhost_scsi_fs_integrity
00:36:37.543  vhost_scsi_hot_remove
00:36:37.543  vhost_scsi_integrity
00:36:37.543  vhost_scsi_lvol_integrity
00:36:37.543  vhost_shared
00:36:37.543  vmd
00:36:37.543  vmd_bdev_svc
00:36:37.543  vmd_fio
00:36:37.543  vmd_hello_world
00:36:37.543  vmd_identify
00:36:37.543  vmd_perf
00:36:37.543  xnvme_bdevperf
00:36:37.543  xnvme_to_malloc_dd_copy
00:36:37.543  zoned_fio
00:36:37.543  
00:36:37.543  
00:36:37.543  -----Tests Missing UBSAN------
00:36:37.543  accel_scan_dsa_modules
00:36:37.543  accel_scan_iaa_modules
00:36:37.543  autobuild
00:36:37.543  autobuild_build_doc
00:36:37.543  autobuild_check_dpdk_pci_api
00:36:37.543  autobuild_check_format
00:36:37.543  autobuild_check_so_deps
00:36:37.543  autobuild_external_code
00:36:37.543  autobuild_generated_files_check
00:36:37.543  autobuild_generated_files_check_post_clean
00:36:37.543  autobuild_header_dependency_check
00:36:37.543  autobuild_make_install
00:36:37.543  autobuild_make_uninstall
00:36:37.543  balanced
00:36:37.543  blob_io_wait
00:36:37.543  blobfs
00:36:37.543  blobfs_mkfs
00:36:37.543  blobstore
00:36:37.543  blobstore_grow
00:36:37.543  blockdev_raid5f
00:36:37.543  blockdev_rbd
00:36:37.543  blockdev_xnvme
00:36:37.543  build_native_dpdk
00:36:37.543  build_rpm_from_gen_spec
00:36:37.543  build_shared_native_dpdk_rpm
00:36:37.543  build_shared_rpm
00:36:37.543  build_shared_rpm_with_rpmed_dpdk
00:36:37.543  busy
00:36:37.543  check_header_filenames
00:36:37.543  cmb_copy
00:36:37.543  confirm_abi_deps
00:36:37.543  confirm_makefile_deps
00:36:37.543  core_load
00:36:37.543  external_make_accel_driver_shared
00:36:37.543  external_make_accel_driver_static
00:36:37.543  external_make_accel_module_shared
00:36:37.543  external_make_accel_module_static
00:36:37.543  external_make_hello_bdev_shared_combo
00:36:37.543  external_make_hello_bdev_shared_iso
00:36:37.543  external_make_hello_bdev_static
00:36:37.543  external_make_hello_no_bdev_shared_combo
00:36:37.543  external_make_hello_no_bdev_shared_iso
00:36:37.543  external_make_hello_no_bdev_static
00:36:37.543  external_make_nvme_shared
00:36:37.543  external_make_nvme_static
00:36:37.543  external_run_accel_driver_shared
00:36:37.543  external_run_accel_driver_static
00:36:37.543  external_run_accel_module_shared
00:36:37.543  external_run_accel_module_static
00:36:37.543  external_run_hello_bdev_shared_combo
00:36:37.543  external_run_hello_bdev_shared_iso
00:36:37.543  external_run_hello_bdev_static
00:36:37.543  external_run_hello_no_bdev_shared_combo
00:36:37.543  external_run_hello_no_bdev_shared_iso
00:36:37.543  external_run_hello_no_bdev_static
00:36:37.543  external_run_nvme_shared
00:36:37.543  external_run_nvme_static
00:36:37.543  ftl
00:36:37.543  ftl_bdevperf
00:36:37.543  ftl_dirty_shutdown
00:36:37.543  ftl_fio_basic
00:36:37.543  ftl_fio_extended
00:36:37.543  ftl_fio_nightly
00:36:37.543  ftl_restore
00:36:37.543  ftl_restore_fast
00:36:37.543  ftl_trim
00:36:37.543  ftl_upgrade_shutdown
00:36:37.543  ftl_write_after_write
00:36:37.543  hello_blob
00:36:37.543  is_zoned
00:36:37.543  iscsi_tgt
00:36:37.543  iscsi_tgt_bdev_io_wait
00:36:37.543  iscsi_tgt_data_digest
00:36:37.543  iscsi_tgt_digest
00:36:37.543  iscsi_tgt_digests
00:36:37.543  iscsi_tgt_ext4test
00:36:37.543  iscsi_tgt_fio_remote_nvme
00:36:37.543  iscsi_tgt_fuzz
00:36:37.543  iscsi_tgt_initiator
00:36:37.543  iscsi_tgt_login_redirection
00:36:37.543  iscsi_tgt_multiconnection
00:36:37.543  iscsi_tgt_rbd
00:36:37.543  iscsi_tgt_resize
00:36:37.543  iscsi_tgt_trace_record
00:36:37.543  keyring_linux
00:36:37.543  llvm_fuzz
00:36:37.543  lvol
00:36:37.543  lvol_basic
00:36:37.543  lvol_esnap
00:36:37.543  lvol_external_snapshot
00:36:37.543  lvol_hotremove
00:36:37.543  lvol_provisioning
00:36:37.543  lvol_rename
00:36:37.543  lvol_resize
00:36:37.543  lvol_snapshot_clone
00:36:37.543  lvol_tasting
00:36:37.543  nvme_boot_partition
00:36:37.543  nvme_bp
00:36:37.543  nvme_cmb
00:36:37.543  nvme_fdp
00:36:37.543  nvme_flexible_data_placement
00:36:37.543  nvme_opal
00:36:37.543  nvme_opal_bdevio
00:36:37.543  nvme_opal_bdevperf
00:36:37.543  nvme_opal_spdk_tgt
00:36:37.543  nvme_pmr
00:36:37.543  nvme_pmr_persistence
00:36:37.543  nvme_simple_copy
00:36:37.543  nvme_xnvme
00:36:37.543  nvme_zns
00:36:37.543  nvmf_digest_dsa_initiator
00:36:37.543  nvmf_digest_dsa_target
00:36:37.543  nvmf_fc
00:36:37.543  nvmf_fuzz
00:36:37.543  nvmf_initiator_timeout
00:36:37.543  nvmf_multiconnection
00:36:37.543  nvmf_vhost
00:36:37.543  packaging
00:36:37.543  raid5f_rebuild_test
00:36:37.543  raid5f_rebuild_test_sb
00:36:37.543  raid5f_state_function_test
00:36:37.543  raid5f_state_function_test_sb
00:36:37.543  raid5f_superblock_test
00:36:37.543  rocksdb
00:36:37.543  rocksdb_insert
00:36:37.543  rocksdb_overwrite
00:36:37.543  rocksdb_randread
00:36:37.543  rocksdb_readwrite
00:36:37.543  rocksdb_writesync
00:36:37.543  rpm_packaging
00:36:37.544  scanbuild_make
00:36:37.544  sma
00:36:37.544  sma_crypto
00:36:37.544  sma_discovery
00:36:37.544  sma_nvmf_tcp
00:36:37.544  sma_plugins
00:36:37.544  sma_qos
00:36:37.544  sma_vfiouser_qemu
00:36:37.544  sma_vhost
00:36:37.544  spdkcli_iscsi
00:36:37.544  spdkcli_nvmf_fc
00:36:37.544  spdkcli_raid
00:36:37.544  spdkcli_rbd
00:36:37.544  spdkcli_vhost
00:36:37.544  test_bdev_lvol_delete_lvstore_with_clones
00:36:37.544  test_bdev_lvol_delete_ordering
00:36:37.544  test_clone_decouple_parent
00:36:37.544  test_clone_inflate
00:36:37.544  test_clone_snapshot_relations
00:36:37.544  test_construct_lvol
00:36:37.544  test_construct_lvol_alias_conflict
00:36:37.544  test_construct_lvol_fio_clear_method_none
00:36:37.544  test_construct_lvol_fio_clear_method_unmap
00:36:37.544  test_construct_lvol_full_lvs
00:36:37.544  test_construct_lvol_inexistent_lvs
00:36:37.544  test_construct_lvols_conflict_alias
00:36:37.544  test_construct_lvs
00:36:37.544  test_construct_lvs_clear_methods
00:36:37.544  test_construct_lvs_conflict_alias
00:36:37.544  test_construct_lvs_different_cluster_size
00:36:37.544  test_construct_lvs_nonexistent_bdev
00:36:37.544  test_construct_multi_lvols
00:36:37.544  test_construct_nested_lvol
00:36:37.544  test_construct_two_lvs_on_the_same_bdev
00:36:37.544  test_create_multi_ublk
00:36:37.544  test_create_snapshot_of_snapshot
00:36:37.544  test_create_snapshot_with_io
00:36:37.544  test_create_ublk
00:36:37.544  test_delete_lvol_store_persistent_positive
00:36:37.544  test_delete_snapshot_with_clone
00:36:37.544  test_delete_snapshot_with_snapshot
00:36:37.544  test_destroy_after_bdev_lvol_resize_positive
00:36:37.544  test_esnap_clones
00:36:37.544  test_esnap_late_arrival
00:36:37.544  test_esnap_reload
00:36:37.544  test_esnap_remove_degraded
00:36:37.544  test_hotremove_lvol_store
00:36:37.544  test_hotremove_lvol_store_base
00:36:37.544  test_hotremove_lvol_store_base_with_lvols
00:36:37.544  test_hotremove_lvol_store_multiple_lvols
00:36:37.544  test_lvol_bdev_readonly
00:36:37.544  test_lvol_list
00:36:37.544  test_lvol_rename_negative
00:36:37.544  test_rename_lvs_negative
00:36:37.544  test_rename_positive
00:36:37.544  test_resize_lvol
00:36:37.544  test_resize_lvol_negative
00:36:37.544  test_resize_lvol_with_io_traffic
00:36:37.544  test_save_ublk_config
00:36:37.544  test_sigterm
00:36:37.544  test_snapshot_compare_with_lvol_bdev
00:36:37.544  test_tasting
00:36:37.544  test_thin_lvol_check_integrity
00:36:37.544  test_thin_lvol_check_space
00:36:37.544  test_thin_lvol_check_zeroes
00:36:37.544  test_thin_lvol_resize
00:36:37.544  test_thin_overprovisioning
00:36:37.544  test_unregister_lvol_bdev
00:36:37.544  thread_spdk_lock
00:36:37.544  ublk
00:36:37.544  ublk_recovery
00:36:37.544  unittest_bdev_compress
00:36:37.544  unittest_bdev_crypto
00:36:37.544  unittest_dpdk_compressdev
00:36:37.544  unittest_lib_reduce
00:36:37.544  unittest_nvmf_fc
00:36:37.544  unittest_nvmf_fc_ls
00:36:37.544  unittest_nvmf_vfio_user
00:36:37.544  valgrind
00:36:37.544  vfio_fuzz
00:36:37.544  vfio_user_nvme_fio
00:36:37.544  vfio_user_nvme_restart_vm
00:36:37.544  vfio_user_qemu
00:36:37.544  vfio_user_virtio_bdevperf
00:36:37.544  vfio_user_virtio_blk_restart_vm
00:36:37.544  vfio_user_virtio_scsi_restart_vm
00:36:37.544  vhost
00:36:37.544  vhost_blk_cores_2ctrl
00:36:37.544  vhost_blk_fs_integrity
00:36:37.544  vhost_blk_hot_remove
00:36:37.544  vhost_blk_integrity
00:36:37.544  vhost_blk_lvol_integrity
00:36:37.544  vhost_blk_packed_ring_integrity
00:36:37.544  vhost_boot
00:36:37.544  vhost_hotplug
00:36:37.544  vhost_migration
00:36:37.544  vhost_migration_tc1
00:36:37.544  vhost_migration_tc2
00:36:37.544  vhost_negative
00:36:37.544  vhost_readonly
00:36:37.544  vhost_scsi_cores_2ctrl
00:36:37.544  vhost_scsi_fs_integrity
00:36:37.544  vhost_scsi_hot_remove
00:36:37.544  vhost_scsi_integrity
00:36:37.544  vhost_scsi_lvol_integrity
00:36:37.544  vmd
00:36:37.544  vmd_bdev_svc
00:36:37.544  vmd_fio
00:36:37.544  vmd_hello_world
00:36:37.544  vmd_identify
00:36:37.544  vmd_perf
00:36:37.544  xnvme_bdevperf
00:36:37.544  xnvme_to_malloc_dd_copy
00:36:37.544  zoned_fio
00:36:37.544  Not all tests were run. Failing the build.
00:36:37.544  ['blob_io_wait', 'blobfs', 'blobfs_mkfs', 'blobstore', 'blobstore_grow', 'blockdev_raid5f', 'blockdev_rbd', 'blockdev_xnvme', 'ftl', 'ftl_bdevperf', 'ftl_dirty_shutdown', 'ftl_fio_basic', 'ftl_restore', 'ftl_trim', 'ftl_upgrade_shutdown', 'hello_blob', 'iscsi_tgt', 'iscsi_tgt_bdev_io_wait', 'iscsi_tgt_initiator', 'iscsi_tgt_login_redirection', 'iscsi_tgt_rbd', 'iscsi_tgt_resize', 'iscsi_tgt_trace_record', 'llvm_fuzz', 'lvol', 'lvol_basic', 'lvol_esnap', 'lvol_external_snapshot', 'lvol_hotremove', 'lvol_provisioning', 'lvol_rename', 'lvol_resize', 'lvol_snapshot_clone', 'lvol_tasting', 'nvme_fdp', 'nvme_flexible_data_placement', 'nvme_xnvme', 'nvmf_vhost', 'raid5f_rebuild_test', 'raid5f_rebuild_test_sb', 'raid5f_state_function_test', 'raid5f_state_function_test_sb', 'raid5f_superblock_test', 'rocksdb', 'rocksdb_insert', 'rocksdb_overwrite', 'rocksdb_randread', 'rocksdb_readwrite', 'rocksdb_writesync', 'scanbuild_make', 'sma', 'sma_crypto', 'sma_discovery', 'sma_nvmf_tcp', 'sma_plugins', 'sma_qos', 'sma_vfiouser_qemu', 'sma_vhost', 'spdkcli_iscsi', 'spdkcli_raid', 'spdkcli_rbd', 'spdkcli_vhost', 'test_bdev_lvol_delete_lvstore_with_clones', 'test_bdev_lvol_delete_ordering', 'test_clone_decouple_parent', 'test_clone_inflate', 'test_clone_snapshot_relations', 'test_construct_lvol', 'test_construct_lvol_alias_conflict', 'test_construct_lvol_fio_clear_method_none', 'test_construct_lvol_fio_clear_method_unmap', 'test_construct_lvol_full_lvs', 'test_construct_lvol_inexistent_lvs', 'test_construct_lvols_conflict_alias', 'test_construct_lvs', 'test_construct_lvs_clear_methods', 'test_construct_lvs_conflict_alias', 'test_construct_lvs_different_cluster_size', 'test_construct_lvs_nonexistent_bdev', 'test_construct_multi_lvols', 'test_construct_nested_lvol', 'test_construct_two_lvs_on_the_same_bdev', 'test_create_multi_ublk', 'test_create_snapshot_of_snapshot', 'test_create_snapshot_with_io', 'test_create_ublk', 'test_delete_lvol_store_persistent_positive', 'test_delete_snapshot_with_clone', 'test_delete_snapshot_with_snapshot', 'test_destroy_after_bdev_lvol_resize_positive', 'test_esnap_clones', 'test_esnap_late_arrival', 'test_esnap_reload', 'test_esnap_remove_degraded', 'test_hotremove_lvol_store', 'test_hotremove_lvol_store_base', 'test_hotremove_lvol_store_base_with_lvols', 'test_hotremove_lvol_store_multiple_lvols', 'test_lvol_bdev_readonly', 'test_lvol_list', 'test_lvol_rename_negative', 'test_rename_lvs_negative', 'test_rename_positive', 'test_resize_lvol', 'test_resize_lvol_negative', 'test_resize_lvol_with_io_traffic', 'test_save_ublk_config', 'test_sigterm', 'test_snapshot_compare_with_lvol_bdev', 'test_tasting', 'test_thin_lvol_check_integrity', 'test_thin_lvol_check_space', 'test_thin_lvol_check_zeroes', 'test_thin_lvol_resize', 'test_thin_overprovisioning', 'test_unregister_lvol_bdev', 'ublk', 'ublk_recovery', 'vfio_fuzz', 'vfio_user_nvme_fio', 'vfio_user_nvme_restart_vm', 'vfio_user_qemu', 'vfio_user_virtio_bdevperf', 'vfio_user_virtio_blk_restart_vm', 'vfio_user_virtio_scsi_restart_vm', 'vhost', 'vhost_blk_lvol_integrity', 'vhost_blk_packed_ring_integrity', 'vhost_boot', 'vhost_negative', 'vhost_scsi_lvol_integrity', 'xnvme_bdevperf', 'xnvme_to_malloc_dd_copy']
00:36:37.544  + compress_coverage_and_docs
00:36:37.544  + echo 'Start compress coverage and docs'
00:36:37.544  Start compress coverage and docs
00:36:37.544  + tar -C coverage -czf coverage_autotest-per-patch_120656.tar.gz ./ --remove-files
00:36:38.477  + tar -C ut_coverage -czf ut_coverage_autotest-per-patch_120656.tar.gz ./ --remove-files
00:36:39.043  + tar -C doc -czf doc_autotest-per-patch_120656.tar.gz ./ --remove-files
00:36:39.609  + [[ -d llvm ]]
00:36:39.609  + true
00:36:39.609  + echo 'End compress coverage and docs'
00:36:39.609  End compress coverage and docs
00:36:39.609  + remove_partial_coverage_files
00:36:39.609  + find . -name 'cov_*.info' -delete
00:36:39.609  + echo 'End stage post_gen_coverage.sh'
00:36:39.609  End stage post_gen_coverage.sh
00:36:39.624  [Pipeline] }
00:36:39.627  ERROR: script returned exit code 1
00:36:39.651  [Pipeline] // catchError
00:36:39.660  [Pipeline] catchError
00:36:39.661  [Pipeline] {
00:36:39.681  [Pipeline] dir
00:36:39.681  Running in /var/jenkins/workspace/autotest-per-patch_2/post_process
00:36:39.683  [Pipeline] {