17:58:05 #startmeeting 17:58:05 Meeting started Tue Apr 25 17:58:05 2023 UTC. The chair is Emperor. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:58:05 Useful Commands: #action #agreed #help #info #idea #link #topic. 17:58:17 Hi folks :) 17:58:40 #topic Roll Call 17:58:43 Matthew Vernon 17:58:43 Christoph Berg 17:58:44 Helmut Grohne 17:58:51 Simon McVittie 17:59:40 I'm guessing spwhitton is unwell enough to not be joining us, but let's give mjg59 a few minutes 18:00:53 Matthew Garrett 18:01:00 hi :) 18:01:13 #topic Review of previous meeting AIs 18:01:32 I did post to d-d-a about nominations, but I don't think we had any takers, alas 18:01:46 I did the thing I was tasked to do. 18:02:05 thanks. 18:02:29 I think helmut's point on the agenda ordering was fair enough, so let's go with that. 18:02:37 aye 18:02:40 #topic the moratorium on moving files into /usr 18:03:11 We issued advice to stop moving individual packages' files into /usr 18:03:30 would someone remind me which problems exactly the moratorium targets? I know about the problem of moving files from / to /usr + moving files between packages. Any other problem? 18:03:43 and said we expected that advice would cease to be necessary at some point during the Debian 13 release cycle 18:03:45 I believe that is the only significant problem it targets 18:04:06 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994388#110 is the text in question 18:04:24 I believe that I found a way to solve that problem in a different way just two hours ago in joint work with pochu 18:05:01 that's great news. dpkg changes, or package changes, or what? 18:05:39 I intended to post a long explanation to d-devel before our meeting, but really the prospective solution only came about just two hours ago in joint work with pochu 18:06:07 Fair enough - is there a TL;DR version for now so we have a rough idea? 18:06:11 we took bluca seriously (in moving all the files post bookworm) and considered how to fix the moratorium issue 18:06:30 and while breaks+replaces doesn't cut it, conflicts does 18:06:47 so in 90% of the cases, moves just work and in the remaining 10% we can resort to conflicts 18:06:58 "in 90% of the cases, moves just work" how so? 18:07:16 except that this doesn't always work for a) complex upgrades where apt is unable to find a solution and b) essential packages 18:07:26 smcv: those where we only move from / to /usr and not between packages 18:07:43 so likely more 95 or 98% 18:08:13 the thing that makes this transition particularly annoying is that the natural time to do it is, as bluca wants, at the beginning of a release cycle 18:08:33 and in th 1% of caes where we have both move categories and cannot use conflicts, we can add a --no-rename diversion in preinst of the new package and remove that diversion in postinst to avert the removal 18:08:53 but in mid 2023, when moving let's say /lib/systemd/system/dbus.service to /usr, we don't know yet whether it will become necessary to split/merge dbus before mid 2025 18:09:38 smcv: yeah, right, but we can deal with that at that time as long as we always have a way forward. and the first step is breaks+replaces -> conflicts, and if that doesn't work, diversions are messy, but they'll work. 18:09:40 helmut: heh, that's a cute idea indeed 18:09:46 that's alright, if that happens later you can then add the conflicts+replaces (or diversion if oldbin is essential) 18:10:23 I think my point from -devel (and -dpkg) still stands, "not enough Breaks/Replaces" is already a RC bug, yet I frequently find myself reporting it 18:11:16 "not enough Conflicts/Replaces or diversions" will equally be RC, but ideally we'd be detecting the need for it *before* it causes files to disappear 18:11:31 that's often counter-intuitive, I wonder if dpkg couldn't do better there and be more forgiving 18:11:46 Myon: please send thanks to pochu. I pondered this for like three days and then when I talked to pochu, it cleared up in like an hour 18:11:52 Do we have any tooling that automatically identifies cases where binaries have moved (a) between / and /usr and (b) between packages? 18:12:30 mjg59: according to bluca, piuparts does that, but my understanding is that his piuparts patch does something different 18:12:30 pochu[m]: thanks! 18:12:33 if the file we're moving is something like /lib/systemd/system/dbus.service (or worse, /lib/*/libc.so.6) then finding out that we have the "it disappears" bug after it has made a testing/unstable user's system unbootable is kinda too late 18:12:41 smcv: +1 18:13:12 having said that, there is value in "let unstable be unstable" 18:13:13 smcv: we'd notice it before it goes to testing, I guess 18:13:24 and unstable is, well, unstable 18:13:50 regarding detection: I think that detecting the / -> /usr + package move scenario is relatively easily possible, not with piuparts, but with a new tool (dedup.d.n could be extended at relative ease) 18:13:59 If there's a technical mechanism that can be implemented in the packages which allows us to guarantee that these transitions occur safely, that seems fine 18:14:02 plus that's only the dbus and libc maintainers that have to be careful, not the whole project 18:14:30 well, dbus was just the first example that popped into my head of a systemd system unit 18:14:31 mjg59: the question kinda is: is that diversion mechanism safe? 18:14:39 (because it's the highest-popcon systemd unit I maintain) 18:14:40 But I'm not familiar enough with dpkg to have an immediate understanding of how this approach interacts with the failure modes 18:14:43 smcv: s/dbus and libc/base system/ 18:15:05 So I think I'd want to see a more detailed description of the interactions to be able to form an opinion 18:15:07 we can hopefully expect these people to be careful 18:15:12 (Or I'm going to need to spend some time with the source) 18:15:14 Myon: oh I see your point, only the packages in the boot -> desktop critical path need to worry about whether they broke that critical path 18:15:20 I shall have a long version posted to d-devel today 18:15:25 helmut: Awesome, thank you 18:15:34 and e.g. with my openarena-server maintainer hat on, I don't have to be that careful 18:15:49 for unstable we only have lintian autorejects, and we can't do it in lintian. But it could be easily done in an external service that files RC bugs as necessary, so testing could be kept safe 18:15:59 the other point regarding detection is to figure out when we cannot do conflicts. I don't have a good idea on that one 18:16:37 helmut: am I right in saying that Conflicts are always safe in terms of not having this bug, but they can cause the upgrade path to become impossible? 18:16:37 So from a making-a-decision point of view, are we at the point where we want to hold off deciding on the moratorium until helmut & pochu's work has been explained and reviewed a bit? Our next meeting is likely before the bookworm release? 18:16:47 the problem I see with using conflicts is with apt finding an upgrade path to trixie 18:17:04 smcv: yes plus the issue that you cannot use conflicts to move files between essential packages 18:17:12 but I'm not too familiar with resolvers to anticipate if that's going to be too problematic 18:17:27 I think the bar is (should be) higher for the amount of care taken by Essential maintainers 18:17:44 perhaps the moratorium could be narrowed to say "don't move between /usr and packages in the base system" with the idea that anything outside will likely be resolvable by apt? 18:18:12 and that would be removed after trixie 18:18:27 Myon: that would disallow doing this automatically as packages get rebuilt, e.g. in debhelper, if I understand correctly 18:18:38 Myon: that seems like the sort of thing that might fall out of a more detailed discussion of the proposal? 18:18:54 smcv: that's only the / -> /usr move, not between packages, so not a problem 18:18:59 at this time, I'm in favour of deferring the question of the moratorium due to all the news 18:19:16 helmut: +1 18:19:19 Emperor: yes, just thinking ahead 18:19:23 sure 18:19:38 if the projected diversion hack actually works, I'm in favour of explaining the solution and dropping the moratorium immediately 18:19:45 I think we are beginning to see a good plan 18:19:52 any opposition to deferring a decision on the moratorium until the May meeting? 18:20:13 Emperor: I'd need time to think about that anyway 18:20:15 what I had expected to come into this meeting and say was: our moratorium is strictly speaking redundant with the release team's, but for clarity we should extend ours into trixie too 18:20:44 Scratch proposal: If rough consensus on d-d after this proposal is described is that it satisfies our previously described concerns, the moratorium can be dropped without explicit t-c action? 18:20:46 if you encounter any issues the projected idea, please reach out to me immediately. it may still have a fatal flaw and become a no-brainer 18:20:48 smcv: I think that was roughly my position too 18:20:55 but if helmut and pochu[m]'s plan is as promising as it seems to be, then ... yes, what Myon said 18:21:04 *what mjg59 said 18:21:12 mjg59: Mmm 18:21:15 thanks, tab completion. invaulable 18:21:24 Mjon :) 18:21:56 I mean this is like 3 hours old now. it seems too good to be true. 18:22:00 #agreed revisit moratorium on moving files into /usr at May meeting in light of discussion around helmut and pochu's proposal 18:22:11 #topic DEP17 18:22:25 My feeling on this is that the TC doesn't need to get involved at this point 18:22:42 the main reason for doing DEP17 is to address the issues of the moratorium 18:22:54 (notwithstanding the impact of the previously-discussed work) 18:23:20 so any discussion on DEP17 must be based on the assumption that the previous idea with conficts + diversions doesn't work for some reason 18:23:33 If we can solve this problem through package policy rather than doing significant engineering work in dpkg, that seems preferable 18:24:08 +1 18:24:11 mjg59: agreed 18:24:13 it also must be based on the assumption that for the reason of conflicts and diversions not working, we continue imposing the moratorium until dpkg is fixed 18:24:35 But I'm kind of reluctant to just say "Let's not talk about this" because if the new proposal doesn't work out, we should have some roadmap for this as soon as possible 18:24:40 so we're in a rather hypothetical land at this point 18:25:13 there are a couple of things that I keep thinking about DEP17 and related dpkg work 18:25:15 Although if we aim to hold May's meeting on the original date (ie, two weeks from now) maybe that's a reasonable compromise? 18:25:18 mjg59: next meeting is in 2 weeks, which isn't that long to punt this for; and by then we may well have a clearer idea of where we stand 18:25:21 mjg59: we still need dpkg engineering to fix `dpkg -S` even if all packages get updated to /usr 18:25:45 Myon: why? 18:26:09 because dpkg -S /bin/bla doesn't find /usr/bin/bla 18:26:15 (workaround: dpkg -S bin/bla) 18:26:45 helmut: if your scheme from today works, am I right to think that your intention is to keep dpkg -S the same as it currently is, only finding files by their "canonical" path as seen in the data.tar? 18:27:19 ...and will dpkg -S $(which foo) produce the right answer? 18:27:20 which hopefully by the time we release trixie would always start with /etc, /usr or /var for software in Debian? 18:27:33 I consider the -S issue non-rc, so yeah, I'd leave it as is and leave it to others to fix when they see fit (or whether) 18:28:01 non-rc, yes, but users will still hate us for it 18:28:12 smcv: I think assumptions like "/bin/sh exists and is resolvable via dpkg" aren't obviously unreasonable 18:28:41 but if we move everything into /usr after the moratorium is lifted, then that's not much of an issue is it? 18:28:42 yeah, in trixie the goal should be that all files ahve been moved to their canonical location and since $PATH usually lists /usr/bin before /bin, dpkg -S $(which foo) should work 18:28:48 on my bookworm system dpkg -S $(which bash) fails, which is pretty poor 18:28:52 are we saying the solution is to remove /bin from the default PATH? 18:29:16 Myon: e.g. Arch did 18:29:36 Myon: ATM I think /usr/bin comes before it anyway 18:29:46 I expect /bin to be removed from $PATH eventually 18:29:52 nod 18:30:00 my $dayjob work on making Steam stuff cross-distro-portable makes me painfully familiar with distro quirks like that 18:30:02 still, `dpkg -S /bin/sh` should continue to work 18:30:22 but I made us digress 18:30:31 if we support chrooting into an older or non-Debian environment, then rushing to remove /bin from PATH would be unfriendly 18:30:35 Mmm, I was wondering if I should try and drag us back to topic :) 18:30:39 so back to DEP17, do you consider this to be a reasonable plan-b? 18:31:00 I have a couple of observations I want to make 18:31:43 one issue that keeps coming up in DEP17 and similar efforts is that dpkg's structure (and dpkg's maintainer's dislike of usrmerge) encourages interfaces like this one to be general 18:31:54 which then makes them API surface area 18:32:22 I think it would be good to be clear about whether an interface is intended to be sysadmin-facing API, or whether it's only intended to be distro-integrator-facing 18:32:40 do you see any way to avoid them being API surface? I was assuming that this would be inherent in the problem at hand. 18:32:59 for example if I offload /usr/share/games onto /srv/more-space-on-this-disk/games, am I expected/encouraged/discouraged to register that alias with dpkg? 18:33:27 (IMO: no, only distro integrators should be using this interface) 18:33:40 vaguely agreed 18:33:44 (and also I should be using a bind-mount of course) 18:33:57 why does it have to be API when we are just talking about a handful of fixed locations? 18:34:04 because of this loop: 18:34:24 ok so I want to add code to handle merged-/usr 18:34:46 I don't like merged-/usr, add a general-purpose feature or nothing 18:34:47 Myon: Becuase people make it more generic to cover /etc (hi, conffiles!) and include revert-from-usrmerge-to-non-usrmerge-in-case-omnious-sponsors-disapper and support-non-usrmerge-Debian-based-distros. 18:34:54 Myon: 1) because dpkg is used by many projects some of which are debian derivatives 2) because dpkg can be used to operate on an (unmerged) installation via --root and thus the merged-ness must be a property of the installation not a property of dpkg 18:35:14 helmut: that's still a boolean flag in a config file 18:35:18 ok, here's a general-purpose interface to handle arbitrary aliasing 18:35:33 now this is "API" that I have to support forever, don't do that 18:35:40 Myon: It's not a configuration option, so it should not be a configuration file. 18:35:51 smcv: yes, I'm saying we should break the loop at 1) 18:35:56 either literally that, or a perception that there would be that 18:36:31 smcv: I think I understand the problem you describe. could you more explicitly note your conclusion from that? 18:37:00 my conclusion is that people perceive (rightly or wrongly) that there is no contribution they can make that would be acceptable 18:37:32 that seemed more of the problem part to me. I suppose you don't have a solution in mind 18:37:36 because a generic version imposes a support cost on the maintainer forever, and a specific version is giving dpkg code to do something its maintainer thinks it shouldn't do 18:37:52 correct, I don't know what the answer to this is 18:38:11 if I did, I'd have suggested it already 18:38:24 still your descriptions increases clarity of the problem. thanks 18:38:26 it is a bit "well, I wouldn't start from here" 18:38:33 but this relates to the other thing I wanted to put into the record of the meeting 18:38:55 helmut: if --add-alias only records the value (and doesn't modify anything else), you don't need that --option, and can just put a list in a config file 18:39:00 which is: dpkg is a native package, which in theory means it does what Debian wants, no more no less 18:39:03 perhaps that would make it less API-ish? 18:39:22 but at the same time, dpkg maintenance in some ways behaves like an upstream 18:39:42 analogous to let's say rpm.org being upstream from all the RPM-based distros 18:39:44 smcv: guillem sees debian uploads as upstream releases, and yeah this is a problem 18:39:49 and we kinda can't have it both ways 18:40:20 if it's an upstream, then Debian needs to be able to apply delta if the cost exceeds the benefit, the same as we do for dbus 18:40:23 Myon: then that config file becomes the api, no? and then you'd have to think where to store it (not /etc) 18:40:38 (for a while I was applying delta to Debian's dbus that I had explicitly rejected with my upstream hat on!) 18:41:15 smcv: in principle, I'd agree with you, but my impression is that having this discussion for real would shrink the dpkg team 18:41:20 helmut: yes, but it's perhaps more an internal detail 18:41:36 helmut: We once had a larger maintainer team for dpkg... 18:42:36 OK, is there more we think we should discuss re DEP17 today? 18:42:40 Myon: to me, that is a different color of the same shed. I'm not opposed to changing the color, but the implementation is the same 18:42:53 I think I'm fine punting DEP17 to next meeting assuming we expect to hold it 2 weeks from now 18:43:00 * Emperor still mindful that we might be able to have a less hypothetical discussion in a couple of weeks 18:43:14 yeah that's the outcome I'm hoping for 18:43:17 Emperor: well, I don't perceive anything as an answer to the question of whether dep17 is something we could move forward as a plan-b. 18:44:01 I'm not sure we'll be able to have a more useful conversation about DEP17 in two weeks 18:44:09 But we may be able to just not have that conversation 18:44:10 mjg59++ 18:44:13 that 18:44:31 So not spending more time on it now seems fine 18:44:38 +1 18:45:05 (though it would be a pity if a workaround (yay!) would mean dpkg remains unfixed) 18:45:10 to the extent of my understanding of dpkg (which is not very much in terms of finer technical detail), DEP17 seems like a viable plan b, modulo some wrangling about exactly how the "/bin aliases /usr/bin" is stored/represented 18:45:28 The broader "Is dpkg maintained in the best way possible for Debian's benefit" question is one that may be outside our remit 18:45:32 Myon: once files are moved from / to /usr, nothing is aliased anymore, so dpkg doesn't need any fix anymore 18:45:39 mjg59: yes 18:45:54 helmut: /bin/sh will be a thing forever 18:45:58 (and I think if we do have to get really into the details of dpkg's operations, I'd find it easier to do a bunch of reading first) 18:46:08 Myon: so be it 18:47:18 #agreed revisit DEP17 at the May meeting iff necessary 18:47:21 #topic AOB 18:47:42 not here 18:47:58 I should note my apologies for the 9th May meeting (I'm away for a couple of weeks, so there are not obvious alternative dates, so I don't suggest moving the date) 18:48:07 ...but I have no further things for today 18:48:33 I hear we have a provisional release date! So if you have (or want to have) last minute bookworm fixes or post-trixie plans, now is the time 18:48:46 *post-start-of-trixie 18:49:25 oooh 18:50:10 * Emperor is going to make themself popular by bumping the "you're still using the obsolete pcre3 library" bugs to RC once bookworm is out 18:50:31 But, that's way off-topic. Thanks for your time folks, and I hope spwhitton is feeling better soon :) 18:50:34 #endmeeting