18:33:28 #startmeeting 18:33:28 Meeting started Tue Oct 2 18:33:28 2018 UTC. The chair is rperier. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:33:28 Useful Commands: #action #agreed #help #info #idea #link #topic. 18:34:03 so the agenda of the meeting can be found here https://salsa.debian.org/kernel-team/kernel-team/wikis/meetings 18:35:28 #info meeting agenda at https://salsa.debian.org/kernel-team/kernel-team/wikis/meetings 18:35:47 #topic Feedback from DebConf 18 18:36:20 rperier might want to #chair people 18:36:26 Right 18:36:26 rperier: #chair bwh 18:36:41 Or just repeat those commands 18:37:11 #chair bwh 18:37:11 Current chairs: bwh rperier 18:37:44 #info meeting agenda at https://salsa.debian.org/kernel-team/kernel-team/wikis/meetings 18:37:52 #topic Feedback from DebConf 18 18:38:30 Who added this item? 18:38:43 It was me 18:39:30 What kind of feedback are you looking for? 18:39:31 that's just to know if interesting feedbacks or technical points have been shared by other debian teams (regarding the debian kernel team) 18:40:00 I wasn't at debconf, but wasn't there some secure-boot work done there? 18:40:44 * vagrantc waves 18:40:50 I think some work was done there, but not on the kernel side 18:41:27 There's nothing I specifically made a note of 18:42:39 There were several events that might be particularly interesting to kenrel developers; I could send a list of these if people think that's useful 18:43:16 If it' 18:43:24 yes please, it might be interesting 18:43:26 If it's not too hard to find that list, yes 18:43:59 #action bwh to send list of interesting DebConf talks 18:44:20 Next topic? 18:44:27 yep 18:44:40 #topic What is the focus for buster? what is the targeted version and features? 18:45:13 is the target version already fixed? 4.19? 18:45:29 Since the next long-term stable branch will be based on 4.19, I think we should use that for buster 18:45:59 I previously talked about 4.20/5.0 having projected that that would be long-term stable, but since it won't that no longer makes sense 18:46:23 this makes definitively sense to target the 4.19 one 18:47:08 agreed 18:47:38 it would be nice to write this somewhere on the wiki, like in a shared roadmap or something (including the targeted version, the important features or topics, etc...) 18:47:40 imho 18:47:44 not sure what “features” means in context? -rt featureset? 18:48:08 -rt usually depends on the rt guys, as it's hardly possible to rebase that patch set 18:48:21 maybe as well which e.g. hardening measures we want to enable 18:48:58 #agreed We will use Linux 4.19.y in buster 18:49:16 by feature I mean, we have many new features and/or componants for each new kernel release, some of these might interest us (or not) 18:49:22 that's what I meant by "feature" 18:50:03 it would be a good idea to crawl through the whole bunch of new options that pop up during building 18:50:17 From time to time I review the kconfig options that we're not explicitly setting. It's a while since I've done that, so this should definitely be done soon. 18:50:41 I try to keep an eye on interesting hardening options from Kees Cook blog posts 18:51:01 Corsac: and thanks for that, because I kept meaning to do that and didn't 18:51:22 I'll do a “final” review for 4.19 so we have a chance to enable things if needed 18:51:43 Would someone volunteer to review the kconfig options in general? Otherwise I will probably find time eventually. 18:51:44 (also I'll try to test these on other arches, arm at least) 18:52:14 personnally, I try to keep an eye on interesting features via phoronix and lwn for each new release. 18:52:34 #action Corsac will review kconfig options for hardening 18:53:07 it might be worth trying to look at the BTS for wishlist bugs asking to enable some options, unfortunately I'm unsure it's currently possible with the state of bdo/src:linux 18:53:33 I sometimes do that but never get very far 18:54:00 is there a statistic about how much the list of bugs would reduce when old bugs are closed (as suggested later in the agenda)? 18:54:57 #action bwh to review kconfig options in general 18:55:27 We really need to get Secure Boot supported in buster, so I intend to keep that enabled for 4.19 when it goes to unstable 18:55:40 please do 18:56:19 We still don't have a planned date for switching the signing service to the production key. At that point we need to change the internal certificate too. 18:56:30 Next topic? 18:56:35 is there a way we can test it on real hardware? 18:56:56 I have a new laptop supporting secureboot, so if I can help for testing secureboot, feel free to ping me 18:57:02 like, do we have the specific keys we could put in the right place in the UEFI vars? 18:57:08 bwh: i'll try to get some anwers during the next ftp-teem meeting 18:57:30 Corsac: we have shim for that, if i'm not mistaken 18:57:43 err, only for the real ones 18:58:09 I think I tried it but failed (not sure our shim is MS-signed?) 18:58:11 Let's discuss this at the end 18:58:14 ack 18:58:19 Corsac: qemu with ovmf might be an easier solution to test it 18:58:22 yep 18:58:33 #topic What is the state of the next (pending) stretch 9.6? Do we bump firmware-nonfree? 18:59:20 Current plan is to update linux a.s.a.p (now that the security update is done) 18:59:32 I don't know why firmware-nonfree would be updated...? 18:59:34 bwh: ack, either 4.9.128 as it is prepared 18:59:48 or 4.9.130 as it is pending in the merge request, but which needs resolving the ABI change 19:00:07 #action bwh to review MR for 4.9.130 in stretch 19:00:10 https://www.openwall.com/lists/oss-security/2018/10/02/2 should probably first released as well as DSA, then mergend 19:00:17 and then upload what we think is good 19:00:29 #action bwh/carnil to upload linux to stretch 19:01:14 bwh: the last question was from me. CVE and bugfixes will only be acceptable for firmware-nonfree for 9.6 right ? 19:01:22 Right 19:01:26 ok 19:01:32 Oh, I guess there are wifi firmware security fixes? 19:01:50 rperier: we can do DSA for non-free (we did it for intel-microcode) 19:01:53 no support for new hardware? 19:02:26 it's not something we like to do (and it requires manual building on various arches), but in case there are serious vulns it's possible 19:03:16 waldi: Theoretically, sure, but I don't think we made any kernel changes that would require the new firmware 19:03:25 (should IMHO be noted that it still should remain exceptional cases and non very critical issues can be defered as usual to point release) 19:04:00 considering the CVEs are from 2016 and 2017 I think we can push them to -pu and wait for 9.6 19:04:35 #action bwh to apply relevant security fixes to firmware-nonfree/stretch and upload 19:04:56 #link https://security-tracker.debian.org/tracker/source-package/firmware-nonfree issues look bad indeed 19:05:06 Anything more on this topic? 19:05:23 yes 19:05:43 do we postpone the version decision after your review of the 4.9.130 MR? 19:06:48 I think we need to move forward, whether or not it requires an ABI bump 19:07:47 that'd be fine by me 19:08:31 I agree with that. The ABI changes which I see in 4.9.130 should only be related to the two comits in the nfc driver, so hopefully we can avoid bumbing ABI only for that, if it is not avoidable we could revert the two commits, leaving that CVE open, but defer those fixed to the earliest point we need anyway an ABI bump 19:08:36 Greg sent the .131 review mail today so the more we wait the more we'll be in desync 19:10:30 so if we agree on .130 asap, then let's go to the next topic 19:11:34 #topic Management of stretch-security and stretch-pu updates (timing, ABI bumps, changelogs...) 19:13:31 I was the one adding this: our current handling of -security and -pu is basically to add small patches on top of the current stretch version, then from time to time (when there's a serious enough vuln) pushing a DSA 19:13:33 Well, we previously agreed we should update to stable-pu more often, but have kind of failed to do that 19:14:00 Corsac: Sorry, that wasn't a response to you 19:14:17 in parallel, we pile point releases to stretch branch (trying to avoid ABI breaks) and a bit before the debian point release we upload it 19:14:43 I somehow think we might have a lot of unfixed vulnerabilities in both stretch and stretch-pu 19:14:53 yes 19:15:01 not always serious ones, but it's hard to spot them from the -stable CCs 19:15:06 Can we maybe start doing time-based releases to stable-py? 19:15:08 (and it's a lot of time) 19:15:10 stable-pu 19:15:14 I'd like it yes 19:15:23 Say, once a month? 19:15:28 how about the following proposal on top of the summarizing of Corsac: import fast new stable releases in stretch branch, if ABI change are avoided, actually do release such updates to stretch-pu already 19:15:48 (timely) 19:16:27 if there's no ABI break, I think we could upload even more often (unless there are other stuff which wouldn't like it, like the buildd network) 19:16:38 so we would spot build failures more often 19:16:53 Maybe 19:17:11 last few release (since .126 I think) I tried to build on arm64 and ppc64el on top of amd64 so we can spot abi breaks earlier 19:17:17 How about, aim to update stable-pu at least once a month? 19:17:42 sound good 19:17:48 ok 19:18:00 I'd aim for a bit faster but if you're more comfortable like this, yes let's try that at first 19:18:15 (I don't think we should aim for an ABI break a week either :) 19:18:23 Corsac: Even once a month would be quite an improvement 19:18:54 btw per-ARCH abi would be a bad idea I guess? 19:19:12 I'm not sure d-i can cope with it 19:19:32 it could, it did in the past 19:19:43 I think per-arch ABI numbers worked in gencontrol.py at one point but I haven't tried it for a long time 19:19:53 ok, let's not play with that right now :) 19:20:01 waldi: Well yeah, in the past we couldn't even use the same upstream version for all arches! 19:20:27 yeah. i don't want to have that back 19:20:44 bwh: is the current workflow working for you? reviewing MR on salsa is ok? 19:20:45 The ABI name goes into linux-support and linux-latest wouldn't get a proper versioned dependency on that 19:20:58 Corsac: I'm very happy with Gitlab MRs 19:21:30 right now you're basically the only one doing the “final” review; is there something we can do as member to facilitate your life? 19:22:30 I can't think of anything 19:22:41 ack 19:22:56 i'm thinking if we could leverage the gitlab ci a bit 19:23:21 can it run stuff on multiple architectures? 19:23:43 #action Team should rebase and upload to stable-pu at least once a month 19:23:45 in theory it can, but we only have x86-64 runners 19:24:21 We didn't seem to come to a conclusion about security updates 19:25:25 bwh: for security updates maybe we can cherry-pick the commits more often into the stretch-security branch, no matter if stretch branch moves fast forward, when we have to decide if a DSA is needed this makes it easier then to just release from the stretch-security branch as needed or merge stretch into stretch-security and release a rebased one 19:26:03 carnil: I would like to keep the number of cherry-picks down 19:26:06 I was in past a bit reluctant to commit as well isolated commits to stretch-security branch because fear of wasting time/work given several of the last updates were then rebases 19:26:13 ok 19:26:29 (the ok refers to "carnil: I would like to keep the number of cherry-picks down") 19:26:31 Firstly, there are a lot of issues that don't get CVE IDs 19:26:39 but do get fixed in stable anyway 19:26:57 Secondly, it's extra work 19:27:17 So, I was thinking we could base on a stretch-pu upload that is at least a week or two old 19:27:39 so use stretch-pu as staging before uploading to -security? 19:27:42 yes 19:28:43 So we would still need to cherry-pick fixes for urgent issues, but for most security issues we would just get the fixeds through stable 19:29:01 Does that sound like it could work? 19:29:11 so that would possibly include stuff like driver backports? 19:29:36 On the rare occasions that we do driver backports, yes those would end up in the security update too 19:29:58 bwh: and at which frequency would we push a DSA? like right now, when a serious vulnerability needs urgent fix? 19:30:18 bwh: Greg includes driver backports in stable, I guess that's what jcristau is referring to 19:30:25 do we want to rebuild the security uploads or just pull the binary packages and push to security-master with a new changes file? 19:30:48 Corsac: No he doesn't. Added device IDs are about the sum of it. 19:30:50 Corsac: no, i meant hw enablement stuff we sometimes do on pu 19:30:52 Corsac: there was as well one prepared by jmm in 4.9.128-1 19:31:01 (megaraid_sas: Add support for Perc 740P/840 (Closes: #890034) 19:31:04 like that one yes 19:31:10 ok :) 19:31:18 or ones Ben did previously 19:31:56 As for DSA frequency, we could probably stand to do them more often 19:33:09 I don't have a really specific idea for any polcy change there though 19:33:46 Anything more on this topic? 19:33:54 not right now 19:34:37 nop (there's the subtopic on embargoes but I guess that's where we headed next?) 19:34:53 #topic Can we improve handling of embargoed updates? 19:35:24 what do you mean by embargoed updates ? 19:35:49 Security updates where the issue is discussed in private before a coordinated release 19:36:12 ok 19:36:24 Gitlab has support for private repositories, so I wonder if we could use those for preparing linux & kernel-sec updates? 19:37:15 I think that "private" allows access by all members of the team, so we might to create a separate and smaller group for this 19:37:25 carnil? Corsac? 19:37:29 yes 19:37:41 hmhm, not sure about policy wrt. security@k.o and the distros-list 19:38:11 basically that means adding the gitlab admins to the people who /could/ see embargoed stuff, that's it? 19:38:18 (DSA already has access anyway) 19:38:19 and DSA 19:38:19 yes 19:38:39 however all salsa admins have access to security-master 19:38:57 waldi: Alex doesn't? 19:39:00 by chance, because they're the same people? 19:39:19 Corsac: Bastian and Joerg are ftp-master 19:39:31 Corsac: yeah 19:40:03 but if/when that changes, we won't really know 19:40:14 (I'm merely stating it, I don't have a hard feeling against it) 19:40:26 I think other distros involve way more people than us 19:40:27 security@kernel.org doesn't seem to have an explicit policy, and doesn't use PGP encryption so I wouldn't worry about that 19:40:40 Can someone check against distros list policy? 19:41:56 currently reading it 19:42:02 https://oss-security.openwall.org/wiki/mailing-lists/distros#handling-of-embargoed-information 19:42:11 #link https://oss-security.openwall.org/wiki/mailing-lists/distros#handling-of-embargoed-information 19:42:25 basically share on a need-to-know basis within the distro 19:43:14 I think infrastructure admins fall under that umbrella and we should try to reduce the number when possible but we can't really do the impossible either 19:43:45 I /could/ ask on oss-sec (but then maybe we'll don't like the answer :) 19:44:16 You should probably ask 19:44:42 ack, let's #action it 19:45:06 other distros probably also have infrastructure where they prepare stuff before they ship. and qa, and ... 19:45:11 jcristau: Indeed 19:45:42 almost sure this is the case for the enterprisy distros 19:45:55 so i wouldn't worry about it, maybe beyond letting the salsa admins know what you're going to do as a fyi 19:46:06 also the information is not shared with infrastructure admins 19:46:26 OK then 19:46:43 #action bwh to create Salsa group for embargoed updates 19:47:07 Next topic? 19:47:16 or just ask me, as i can override options 19:47:17 bwh: can you #action me too? 19:48:08 Corsac: jcristau persuaded me that it's not really necessary 19:49:03 ok 19:49:43 Next topic? 19:49:52 ack 19:50:01 #topic Do we close old bugs for kernel < a specific version ? (for very old kernels) 19:50:12 the next topic (old bugs) is from me: we have ~2300 bugs opened, with a lot of bugs about very old kernels (3.2 sometimes), I think that some of these are probably outdated or won't probably be resolved... what do we do for bugs reported on very old kernel ? perhaps when the version is < than a specific version, we should just close the bug ? 19:50:47 Anything that hasn't been reported on 3,16 (jessie) or later, please ask the reporter whether it is still reproducible 19:51:08 And any bug where the reporter doesn't respond to a question within a month, can be closed 19:51:37 so we aske the reporter to try to reproduce the issue with a maintained kernel version ? 19:51:39 Anything that is already known fixed by 3.16, should of course be closed immediately 19:51:41 s/aske/ask/ 19:51:42 rperier: Right 19:51:49 (re the sharing within the distro, I just volunteered myself on #debian-admin for writing some common policy/code of conduct/guidelines for people with elevated access so we can make it explicit) 19:51:50 probably worth to automate that? 19:52:25 Mithrandir: thanks (DSA is delegated, isn't it? so it could be part of delegation or something?) 19:52:45 you can definitively add an action for me to help to triage bugs like this 19:53:46 Corsac: it should cover salsa admins and such too, which, iirc, are not delegated. But, let's continue this in #debian-admin if there's more to talk about, I don't want to interrupt your meeting. :-) 19:54:01 ukleinek: It's probably possible to automate part way, but would require manual checks because much information is free-form and not copied to the bug metadata. 19:54:43 #action rperier will triage older bugs 19:54:55 we could perhaps tag bugs that "should be monitored during a month". Then if no activities is viewed on this bug during a month, it could be automatically closed 19:54:56 ? 19:55:05 (with a special tag) 19:55:16 user-tags should be suitable 19:55:26 automate this is a good idea, it might help 19:55:38 OK, who will script this? 19:58:05 ... 19:58:31 didn't rperier volunteer? 19:58:55 Saying something "is a good idea" isn't quite agreeing to do it :-) 19:59:12 1538510054 < rperier> you can definitively add an action for me to help to triage bugs like this 19:59:13 I can add this to my todolist if you want yes (sorry I was not on the laptop) :) 19:59:36 * rperier is cooking in the same time 19:59:46 Ah, sorry we are running quite over time 20:00:39 np :) 20:00:41 I won't add a separate action point; it's up to you to work out whether you can script some of the work 20:00:51 Next topic? 20:00:58 ack 20:01:12 ack 20:01:17 #topic Potential use of dgit & git-debrebase 20:01:57 I added this topic 20:02:44 For those who don't know these tools: dgit is a package and command for maintaining Debian source packages in git, including downloading from and uploading to the archive 20:03:41 git-debrebase is a new addition that handles maintaining Debian changes to the upstream source *as commits* and then turning them back into debian/patches before upload 20:04:14 And, as the name suggests, rebasing them onto new upstream releases (but keeping fast-forwardable history through pseudo-merges) 20:04:42 is that similar to dgit? 20:04:59 It works with dgit and is part of the dgit package 20:05:23 This was presented at DebConf and seemed to me like it might be a tool which would actually work for the kernel team 20:05:40 a bit like gbp pq? 20:05:52 how would you keep out non-free stuff from the vcs? 20:06:03 #link https://debconf18.debconf.org/talks/60-git-debrebase-new-tool-for-managing-debian-packaging-in-git/ 20:06:05 bwh: does it work with merge requests? 20:06:13 Corsac: probably 20:06:23 vagrantc: The non-free bits would still be in the vcs 20:07:15 So I've investigated this, reported some bugs (which have bee fixed) 20:07:38 and I have a branch with scripts and patches updated ready for conversion 20:07:44 #link https://salsa.debian.org/benh/linux/tree/use-dgit 20:08:06 * vagrantc looks forward to a much-improved workflow 20:08:28 not that i do a lot of the work... 20:08:31 maybe we should take a month and play with it? 20:08:48 vagrantc: For the non-free removal, I replaced genorig.py with a genupstream script that commits the removals and creates an upstream tag after that 20:09:17 waldi: Well, there's no easy way to go back 20:09:33 bwh: with the copy of cause 20:10:37 so there's a branch where the whole upstream source is available? 20:10:58 Corsac: Yes, the upstream source would be merged with our debian directory 20:11:28 And all our patches would be regular commits (e.g. git cherry-pick should just work) 20:11:40 rt sadly still has to be patches-in-git 20:11:43 i would like to see how this works with merge requests 20:11:50 waldi: Yes, good point 20:12:11 because dgit, which sounds likt it does something similar, does not work with it 20:12:25 So, why don't I create a dgitified linux repo under kernel-team and then we can try to do all the same things there that we do in the "real" repo 20:12:30 you can use git-debrebase without dgit, fwiw 20:12:40 bwh: sounds like a good idea 20:13:04 and I'll send mail to the list explaining this and the necessary workflow changes 20:13:09 bwh: we couldn't work with a -rt branch for generating the patches? 20:13:36 * waldi .o( Format: 3.0 (custom)… ) 20:13:57 * vagrantc thought it produced 3.0 (quilt) 20:14:07 Corsac: You mean, when you need to resolve conflicts? 20:14:31 because genpatch-rt already works with the stable-rt git branches 20:15:27 I guess it's possible but I think it's outside the scope of git-debrebase 20:15:39 yeah nevermind, I'm not really sure what I'm asking anyway, I'd need to think a bit more about it 20:16:14 #action bwh to create linux-with-dgit repo under kernel-team for team to experiment with 20:16:30 #action bwh to send mail to debian-kernel explaining workfow changes for linux-with-dgit repo 20:16:37 Next topic? 20:16:39 +1 20:17:26 yes 20:17:34 #topic Create a shared roadmap for the current development release that contains all the expected technical goals or actions to complete until the next release (in addition to existing opened bugs) 20:17:58 Did we already cover this with "What is the focus for buster?" 20:18:00 the last topic is from me: basically, we're a team so a simple shared roadmap somewhere (the wiki?) would be good. 20:18:13 yes and no 20:18:47 Given that we do time-based releases I don't think that a "release roadmap" makes sense 20:18:47 the last topic is an action for writing importants topics somewhere, like a shared todolist for the team 20:19:11 todo list would be issues 20:19:18 I do see some value in a list of "things we'd like to do", just not tied to specific releases 20:19:46 (If something really needs to be done before releases it should be a bug, I think) 20:19:46 yeah it can be release unrelated 20:19:47 milestones could help for things that really should be done until a specific point 20:22:04 sorry for.the inyromission but a backlog always help in order to keep the milestones going 20:23:29 we could add some of the relevant #actions as issues in gitlab and see how it works? 20:23:35 (like the hardening stuff for me) 20:23:52 and add milestones to it? 20:24:29 for this one, yes as there's a clear milestone for 4.19 (and buster) 20:25:15 (pff, why do I have debian-med and freedombox milestones ><) 20:25:26 please use group milestones 20:25:52 you have milestones? 20:26:03 me too 20:26:19 I don't know about Gitlab issue tracking at all; can someone volunteer to start on this? 20:26:38 bwh: I can try with the hardening options at least, and report back 20:26:47 bwh: sure 20:27:23 #action Corsac and waldi to create issues/milestones on Salsa for planned work 20:27:49 That's the last topic on the agenda; are we finished? 20:27:58 as an additional datapoint if this is of interest: there are way to many bugs to be able to hold an overview, having tracking of thinks which should be fixed in next release/track them im some common way can help 20:28:04 sorry might be an out of order comment 20:31:48 bwh: I think we are 20:32:13 carnil: All bugs should be fixed, but maybe we need to change severities more often or have a separate user-tag for our priorities 20:32:45 (since severity and priority could be quite different for bugs that affect obscure hardware) 20:32:52 #endmeeting