[00:59:47] *** Quits: ziyeyang_ (ziyeyang@nat/intel/x-uxhhuwvpsvvamrtg) (Quit: Leaving) [01:43:40] *** Joins: sethhowe_ (~sethhowe@192.55.54.38) [01:43:47] *** Quits: sethhowe (sethhowe@nat/intel/x-cpdfswcvvqmxwtjo) (Remote host closed the connection) [02:39:19] *** Quits: Shuhei (caf6fc61@gateway/web/freenode/ip.202.246.252.97) (Ping timeout: 260 seconds) [07:54:53] Do we have updated QEMU on TP? [08:46:36] pwodkowx: looks like the current vhost qemu on fedora-08 is one commit behind the latest on our GitHub spdk/qemu fork [08:47:24] same on fedora-05 and fedora-07 [09:15:44] *** Joins: lhodev (~Adium@inet-hqmc03-o.oracle.com) [09:49:25] i believe we can't update QEMU on TP just yet [09:49:45] newest QEMU got Changpeng's SET_CONFIG/GET_CONFIG patches [09:49:57] and modified version of virtio-blk that relies on those [09:50:39] so until we have changpe1 patches in SPDK, we have to use the outdated QEMU [09:51:08] otherwise virtio-blk tests will simply fail [09:52:32] (and these SPDK patches won't get it until after 18.01) [10:01:02] *** Quits: tomzawadzki (~tomzawadz@192.55.54.44) (Ping timeout: 256 seconds) [10:39:10] the patch that enables the vhost live migration tests fails on the test pool: https://ci.spdk.io/spdk/builds/review/d51b566e9a28256a6f7c213c64b03607d34ea7df.1517332739/ [10:39:23] yes I see [10:39:24] is that because of the QEMU updates? [10:39:37] (i.e. because we have not updated the qemu version?) [10:40:22] the second qemu did not started afer migration [10:40:27] I will check that. [10:40:30] ok, thanks [10:41:13] Can you install new qemu in different path? we can use this while testing [10:41:55] which qemu do we need? [10:42:04] is it just the one extra commit from the spdk branch? [10:42:22] https://github.com/spdk/qemu/commits/spdk [10:43:11] checking [10:44:26] I tested live migration on commit that changpeng pointed: commit 406d2aa2 “contrib/vhost-user-blk: introduce a vhost-user-blk sample application” [10:46:44] there are many important fixes for live miggration in QEMU master so we need this for sure. About the changes needed in SPDK for new QEMU I will check we need something other than just test scripts adjustments [10:47:30] okay, so that is a totally different branch, not the old spdk one we are currently using? [10:48:22] from master [10:48:45] oh, I see, that commit is on the qemu master branch with no extra SPDK patches applied? [10:49:00] should we just move our spdk branch to point to that then? [10:49:06] yes, It is working out of box [10:49:12] doesn't seem like that commit is in the master branch [10:49:22] and push the vhost-nvme experimental patches after the 18.01 release [10:49:29] i see it [10:49:37] for live migation it is OK but for virtio-blk it might break some tests [10:49:41] never mind, my master branch was out of date [10:49:43] it is there [10:50:19] I think we did check in the SPDK side of changpe1's vhost-user-blk changes, although I'm not sure [10:51:18] yeah - they are there, just an older version of the patches (not what was committed to master) [10:51:57] commit 53c24913 ("vhost_lib: enable virtio configuration space feature") is in SPDK master - is that all we need for the SPDK side? [10:52:08] yes [10:52:49] sorry, is this sommit before 406d2aa2 “contrib/vhost-user-blk: introduce a vhost-user-blk sample application” ? [10:53:36] I'm looking at patches to SPDK, not QEMU (I thought we would be compatible to QEMU master now?) [10:59:48] so maybe I am a bit lost - can we switch over our vhost/qemu build to upstream qemu master or not with current SPDK? [10:59:52] nope, the https://review.gerrithub.io/#/c/386545/ is not enought. It only adds sceleton for new messages but SPDK vhost-blk is not supporting this yet [11:00:47] pls install it in different path so I can produce needed fixes [11:01:01] and test it [11:03:07] *** Joins: gila_ (~gila@5ED4D9C8.cm-7-5d.dynamic.ziggo.nl) [11:03:08] *** Quits: gila (~gila@5ED4D9C8.cm-7-5d.dynamic.ziggo.nl) (Read error: Connection reset by peer) [11:04:13] there is a patch from changpeng https://review.gerrithub.io/#/c/386546/ [11:05:55] this patch should be enought (if it will work of cource:) ) for switching to new QEMU [11:18:31] will this break compatibility with our current spdk/qemu spdk branch? [11:18:42] yes [11:21:17] drv pwodkowx: how about we enable live migration tests on a machine that doesn't run vhost-blk tests? [11:21:54] that seems like a good idea; do we have one of those currently? [11:21:55] we announce live migration as experimental anyway, do we? [11:22:14] I think all of the vhost test machines run both vhost-user-scsi and vhost-user-blk right now [11:22:22] I have no idea unfortunately [11:23:14] there is just a single SPDK_TEST_VHOST flag, so I don't think we can control them individually without changing the test code [11:24:06] jimharris: what are your thoughts on how to resolve this? should we try to update to qemu master before the release and get changpe1's vhost-blk patch in? [11:24:21] or defer the vhost-blk changes until after the release? [11:24:22] I can play with checking qemu version and try to maintain backword comaptibility [11:24:30] (this still leaves us with no way to test live migration right now, I believe) [11:25:27] as the last resort, we can test it manually [11:26:52] drv: we should probably add SPDK_TEST_VHOST_MIGRATION flag anyway [11:27:16] so we can if (migration_flag) test_migration; else test_vhost_blk [11:27:17] yeah, that is probably a good idea [11:27:23] + a proper comment [11:27:51] I am okay with checking in live migration support without the test patch, as long as it has been tested locally, and then do the qemu update after the release [11:28:31] it would have been nice to get the vhost-blk upstream compatibility changes in, but I don't know that we want to go changing that at the last minute [11:47:05] so to be sure, we test it manualy? [12:16:49] I uploaded my version: https://review.gerrithub.io/c/397246/ [12:34:36] *** Quits: sethhowe_ (~sethhowe@192.55.54.38) (Remote host closed the connection) [13:42:03] drv: live migration patches are +2'd [13:42:23] ok, let's go ahead and push those [13:42:55] i pushed everything up through vhost: add live migration support [13:45:18] we need to add a changelog entry for live migration [13:45:23] and also thin provisioning [14:04:50] vhost initiator tests are taking a minute longer than they did yesterday [14:05:52] the patch "Added latency to channel statistics" increased the time by a minute. I don't know if that patch is responsible or if the hardware in the pool changed [14:14:19] the compile time for SPDK inside the VM seems variable [14:14:23] not sure why it is taking that much longer now [14:14:33] i feel like it must be hardware in the pool [14:15:24] test patches were all pretty consistent around 7.5 minutes and then suddenly jumped to 8.0-8.5 minutes (no release builds in there) [14:15:38] yeah, that seems unrelated to the patches [14:15:43] so it must be some external change [14:15:45] that setup_vm timing band needs to be broken up for sure [14:16:00] agreed [14:18:07] the FreeBSD machine is taking pretty long too [14:23:34] posted some changelog updates: https://review.gerrithub.io/#/c/397272/ [14:23:45] I didn't really know what details to provide for either feature - comments welcome [14:28:17] I think it looks good - CHANGELOG should just hit the highlights - more details belong elsewhere [14:32:07] peluse: fixed a couple of typos for you in https://review.gerrithub.io/#/c/384118/ - bwalker or drv: needs another +2 from one of you [14:32:55] jimharris, LOL I can't believe you're the first one to catch Atrocity instead of Atomicity :) I ran all that through a spellchecker, that's what I get! [14:33:30] yeah - that's one of the funniest typos i've ever seen [14:40:19] there is now a giant "Make a Donation" button on GerritHub [14:40:22] all the sudden [14:40:30] it's not clickable [14:40:32] we need one of those for SPDK! [14:43:49] *** Joins: sethhowe (~sethhowe@134.134.139.83) [14:44:30] it's not a donation you're thinking about [14:44:40] apparently the button leads to https://www.gofundme.com/shawn-pearce-memorial-fund [14:44:52] hmm, it's still not clickable for me [14:45:16] it's clickable during loading screen [15:04:13] we might want to rework some of the API comments now that thin provisioning is in [15:04:24] e.g. spdk_blob_get_num_clusters() is described as the number of clusters allocated to a blob [15:04:48] but I think it's actually the size of the blob, not all of which will necessarily be allocated [15:55:18] *** Quits: sethhowe (~sethhowe@134.134.139.83) (Ping timeout: 240 seconds) [16:02:37] *** Joins: Shuhei (caf6fc61@gateway/web/freenode/ip.202.246.252.97) [16:05:07] *** Joins: sethhowe (sethhowe@nat/intel/x-ekcxacdskkbplcfm) [16:17:50] drv: https://review.gerrithub.io/#/c/394116/ [16:18:29] lgtm [16:19:10] are we putting in bwalker's nvmf rpc patches? [16:19:13] or waiting? [16:20:09] for the release? I don't know [16:20:19] I was just clearing out my queue [16:20:20] "nvmf: Add an rpc to add allowed hosts" failed with an assert: https://review.gerrithub.io/#/c/396063/ [16:20:31] https://ci.spdk.io/builds/review/37040b42be5c84d9967798cb3c02ab455bba78ee.1517350559/fedora-06/ [16:21:26] I think we can defer those until after the release [16:21:28] yeah I missed one [16:24:01] ok - i cleared the 18.01 tag from those three patches [16:26:22] this one is marked 18.01 https://review.gerrithub.io/#/c/394682/7/doc/fio.md - do we want to get this one in? if so, i'll do another run at it [16:28:43] in my opinion, it's not critical for 18.01 [16:33:58] not critical [16:34:06] ok [16:34:16] i'll clear 18.01 from it but will still do some edits for Liang [17:30:07] bdev question: I could be interpreting the code wrong but it appears that vbdevs have to be stacked vertically on top of a bdev; ie vbdevA claims vbdevB claims bdev. So in the event that a new vbdev finds a bdev that it wants it can't claim the bdev it has to find the topmost vbdev and claim it correct? If not just say no and I'll keep digging :) [17:33:01] I think you'd generally just want to fail to create the vbdev in that case (if somebody else claimed it already) [17:34:02] e.g. if the split vbdev claimed Nvme0n1, and then your vbdev also wants to claim Nvme0n1, it should just print an error saying it's already claimed rather than trying to find the split vbdev [17:34:05] (in my opinion) [17:36:50] *** Joins: nvme_noob (d05b0202@gateway/web/freenode/ip.208.91.2.2) [17:42:56] @drv ipv6 works fine :) [17:43:04] cool, good to hear it [17:43:12] what RDMA NIC are you using, if you don't mind? [17:46:35] I am using Mellanox cx4 [17:59:05] drv, OK thanks. I didn't hunt down who is using the linked list of vbdevs in struct spdk_bdev but assumed that would be a use as I was thinking of how to handle that case in my examine fn... obviously can skip it for now but seems like a pretty big limitation to have a bdev only claimable by one vbdev [18:02:17] drv, and did I hear you say that a hot remove callback provided at open() is required for the standard teardown path? Might have been hallucinating, hard to say... [18:15:18] peluse: yes, the remove callback is required now [18:15:43] also, on the previous topic, you wouldn't generally be able to find the vbdev that has a bdev open, since the vbdev -> bdev relationship is not necessarily one-to-one [18:16:01] drv, OK, cool thanks. I think I'm gonna go watch some comedy now (State of the Union)... [18:16:02] the partition-style vbdevs can create multiple vbdevs that have a single base bdev open at once [18:16:21] heh, good luck with that [18:16:59] It will enhance the watching of SNL this weekend for sure! [18:22:25] *** Quits: nvme_noob (d05b0202@gateway/web/freenode/ip.208.91.2.2) (Quit: Page closed) [20:05:59] *** Joins: ziyeyang_ (~ziyeyang@192.55.54.42) [20:39:02] sethhowe, when you get a chance can you look at the output at https://ci.spdk.io/builds/review/0563e9e47f8a3db4e24462a280e8b824a01cfb3d.1517364524/fedora-04/build.log [20:40:12] and the patch that generated it, specifically the "ls" cmds that I added to try and figure out why the diff statement in blobstore.sh can't find one of the files, the permissions aren't what I expect and just for debug I tried to cat the output of the cli and get a permissions issue there too [22:18:37] *** Quits: lhodev (~Adium@inet-hqmc03-o.oracle.com) (Quit: Leaving.)