[01:36:25] *** Joins: gila (~gila@5ED74129.cm-7-8b.dynamic.ziggo.nl) [05:36:49] As a reminder, the the Euro community call that is happening soon, see https://spdk.io/community/ for your local time zone info HOWEVER do not use the WebEx link on that page. [05:36:49] Use this instead: [05:36:49] Tue Nov 13 Euro Call: [05:36:49] Access code: 652734# [05:36:50] Online meeting ID: paul_e_luse [05:36:51] Join the online meeting: https://join.freeconferencecall.com/paul_e_luse [07:13:58] comm meeting starts in 45 min..... might want to start connecting early in case of issues with new conf service.... [07:27:57] FYI anyone getting a failure on Jenkins block tests (jscon_config) that looks like this: spdk_io_device_register: *ERROR*: io_device 0x1d2b9d0 already registered FYI I'm looking into it. If you have any thoughts, etc., let me know. [07:28:30] I have a test patch running through CI now to add more debug output but it appears only to happen on Jenkins and its happening pretty frequently [07:53:46] peluse: yep, quite anoing... [07:55:07] for sure! [09:32:36] somebody added io_device->name a while ago [09:32:55] we should at least print it in that 'already registered' message [11:35:39] *** Quits: tomzawadzki (uid327004@gateway/web/irccloud.com/x-vhmraxxnavrpadmy) (Quit: Connection closed for inactivity) [12:38:50] I did [12:39:01] and it finally failed, with the debug prints https://ci.spdk.io/spdk-jenkins/results/autotest-per-patch/builds/14961/archive/blockdev_autotest/build.log [12:39:54] looks like it's the split module but I haven't dug into it yet - previously based on lack of detail and the order in the log we thought it was the crypto module that was not unregistering something (well, still might issues anywhere, just about to look into the split module) [12:53:31] looks like split is trying to do a io_device register on an io_device previously registered by crypto (and still is) [12:57:48] can somone look at this issue https://github.com/spdk/spdk/issues/499 I can't help much there, but seems we had the same issue before. [12:58:35] did you search the dpdk mailing list? [12:59:08] some NICs could have failed on the same thing [13:00:39] there's no chance to screw anything up in SPDK in that case, because the error appears before we even get to do anything with the device [13:01:00] it's all in dpdk [13:11:56] pwodkowx: i'll take a look into it tomorrow [13:17:50] there are only nvm bound to UIO there so don't think NICs are problem there [13:49:04] jimharris, bwalker so could it be the case the vbdevs are supposed to be unregistering their IO device when the vbdev is destrouyed (in this case via RPC)? Because I certainly wasn't and it looks like it was just by chance that sometimes an attempt to register something totally different would end up trying to use the same address of what crypto was holding on to [13:50:09] I added an unregister and tested locally and it works fine. I have a few hours of meetings so will push a fix later today but note that there are inconsistencies in this rea with at least 3 or 4 of our modules. I'll work on crypto first and make sure that's solid and then get the others fixed up as well [13:50:42] if a module does an io_device_register, then it is also responsible for doing the io_device_unregister [13:51:49] usually the io_device ctx pointer is something that was malloc'd, so the randomness is likely due to whether or not a previously registered io_device gets immediately reallocated via malloc and registered again as an io_device [13:53:51] yeah, I believe this is the problem then. Look for a real fix later today... thanks! [13:56:34] here I thought it was a missing unregister in an obscure error path :) [14:48:55] jimharris, bwalker : here is the short patch that looks like the right fix https://review.gerrithub.io/c/spdk/spdk/+/433194 [14:51:05] I assume I might need to unregister the io device in my .destruct function but didn't put that in the above patch - will add next once this is in [15:32:24] *** Joins: JoeGruher (c0373627@gateway/web/freenode/ip.192.55.54.39) [15:33:07] if I start nvmf_tgt without any configuration, and then configure bdevs, raid, lvstores, lvs, and nvmeof subsystems with rpc.py, is there a command to then dump that out to a config file i can feed to nvmf_tgt next time i start it? [16:41:00] peluse: i posted comments on your patch [18:02:57] *** Quits: JoeGruher (c0373627@gateway/web/freenode/ip.192.55.54.39) (Ping timeout: 256 seconds)