12:28:37 <h01ger> #startmeeting dpkg-buildinfo files https://pad.riseup.net/p/dpkg-buildinfo-keep
12:28:37 <MeetBot> Meeting started Fri Sep 25 12:28:37 2026 UTC.  The chair is h01ger. Information about MeetBot at https://wiki.debian.org/MeetBot.
12:28:37 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
12:28:59 <h01ger> heya everyone & thanks for joining, especially guillem
12:29:05 <guillem> hmm I'm not sure if I renamed someone else's user in the pad (felt like I had to type my name twice :D)
12:29:30 <h01ger> guillem: no idea :)
12:30:00 * kpcyrd waves :)
12:30:10 <mapreri> #link https://pad.riseup.net/p/dpkg-buildinfo-keep
12:30:26 <h01ger> i believe kpcyrd Snu[m] jochensp mapreri and me are the people interested in this discussion from the r-b meeting in gothenburg
12:30:56 <h01ger> i also believe everyone is familar with the pad?
12:31:20 <h01ger> not sure if we should repeat the contents there or whether guillem wants to jump in or how else to start :)
12:32:05 <h01ger> i guess its fine if guillem starts - as we have started with the pad anyway
12:32:08 <h01ger> guillem: ? :)
12:32:14 <guillem> ok let's start with p1 then :D
12:32:19 <h01ger> hehe, great
12:32:28 <guillem> I'm not sure what part is missing, is it the arch?
12:32:41 <h01ger> the hash
12:33:16 <guillem> there is currently a problem with multiarch (where josch provided a patch that still needs to be split/detangled)
12:33:58 <h01ger> how is that related? i thought dpkg doesnt know the hash (atm), only apt knows them ?
12:34:54 <guillem> sorry I meant in that dpkg-genbuildinfo does not properly print accurate information on multiarch cases (as in AFAIR) as in missing (or wrong?) arch qualifiers and such
12:35:04 <h01ger> ah
12:35:30 <h01ger> our usecase is not multiarch though :)
12:35:42 <Snu[m]> The problem we're referring to is a unique identifier of a build dep
12:35:51 <h01ger> our usecase are really only native (re)builds
12:35:54 <Snu[m]> name + version + arch is not such identifier
12:35:57 <guillem> ok so then I assume p1 and p2 are the same (or different faces of the coin)?
12:36:04 <Snu[m]> Yess
12:36:09 <Snu[m]> *Yes
12:37:01 <guillem> I think we have discussed this in the past, and it has always looked to me like this information is not critical/required, but it would be a qualify of life improvement
12:37:16 <guillem> I don't object nor have any problem with trying to get that implemented though!
12:37:42 <guillem> I think the concerns expressed in the pad have been the main "blockers", mostly potential performance degradation
12:38:11 <h01ger> AIUI: having the version number in there requires snapshot.d.o to exists, while if there were hashes one could use files from disc. (correct me if i'm wrong)
12:38:50 <Snu[m]> I think we could start with it being configurable to address the performance issue (listed lower in the pad)
12:38:53 <guillem> I see two main ways to get this (AFAIR): one would be to extend the debian-binary member to add a new line with the entire (or selected parts of the) .deb digest, the problem is that this then breaks with stuff that might extend the .deb container in supported ways
12:38:57 <guillem> such as signed .debs
12:39:00 <jochensp> with the hash we can be sure that we get the same package
12:39:59 <h01ger> guillem: whats a debian-binary member?
12:40:01 <guillem> the pre-computed hash also means that we need to trust that the data (supposedly generated at construction time by a newer dpkg-deb) is correct
12:40:17 <guillem> h01ger: man deb(5)
12:40:18 <mapreri> it would be much better to us to have a hash of the whole container, as that is what is used everywhere to identify it (in the apt archive, etc)
12:40:35 <kpcyrd> the hash could be calculated while the file is being read (which dpkg needs to do anyway), so the performance overhead in that case should be minimal? :)
12:41:10 <guillem> kpcyrd: that would be the second option yes, the problem there is how dpkg drives dpkg-deb to get data out of .debs
12:41:18 <kpcyrd> (I agree that an extra `sha256sum file.deb` wouldn't be ideal)
12:41:38 <kpcyrd> guillem: is it seeking?
12:42:01 <guillem> kpcyrd: currently dpkg runs dpkg-deb twice, once for the control.tar extraction for the db handling, and then another time for the fsys data during unpack
12:42:26 <guillem> and then in both cases the tar data gets output on stdout
12:43:12 <guillem> while we could add some out-of-band mechanism to transfer additional metadata like the digests via say some dpkg-deb --metadata-fd or something, it's a bit meh (but certainly a possibility=
12:43:23 <guillem> s/=/)/
12:43:33 <h01ger> what would other options be?
12:43:44 <h01ger> not having hashes in buildinfos files is also pretty meh ;)
12:45:44 <guillem> going back to my earlier comment on why I see this being a quality-of-life improvement (which I think I might have covered in the past), is because I've always understood the .buildinfo files as untrustworthy anyway (because we are tying to repro package that are potentially handled by an adversary)
12:46:57 <kpcyrd> I consider the buildinfo file a source input :) source code still needs to be reviewed/reasoned about ofc
12:47:12 <jochensp> we plan to add the buildinfo file to the apt archive and link it from the Packages file, that should make it trustworthy
12:47:23 <guillem> if that's the worst case scenario then relying on its trustworthinness or accuracy does not seem like it should be a fundamental property, and thus it feels like any information there should be treated as a guide
12:47:52 <guillem> the only true artifact that we can rely on is the actual artifacts we want to repro, which in most cases are just the .debs we have in front of us, and can examine etc
12:48:34 <guillem> jochensp: for the archive, I guess, still that assumes the buildds have not been compromised for example
12:48:42 <jochensp> sure
12:50:42 <h01ger> with hashes it would be a way more precice guide ;) & also working if snapshot were down
12:50:46 <guillem> h01ger: I guess the other option that I could see is to trust the frontend driving dpkg to feed to it extra metadata about, that then gets stored in the db somehow
12:51:02 <Snu[m]> The buildinfo serves as part of the build claim though
12:51:20 <guillem> h01ger: sure, it might make things easier, but as I mentioned that still seems like a quality-of-life thing :)
12:51:24 <Snu[m]> Build-Depends: <pkg> (= <ver>) is not a claim precise enough for say research purposes
12:51:25 <h01ger> guillem: that frontend would be apt, or?
12:51:32 <guillem> h01ger: yeah something like apt
12:51:43 <h01ger> (or something similar, but for us it would be enough if it were apt)
12:53:02 <Snu[m]> The efficiency brought by hashed build dependencies is tmk removing a chunky index in s.d.o
12:53:21 <Snu[m]> So that you don't need to go to the snapshot for resolution of package/version/arch to hash
12:53:31 <h01ger> tmk=to my knowledge
12:56:37 <guillem> maybe my view is colored by not fully understanding how painful this might currently be though?
12:57:16 <jochensp> currently we need metasnap.debian.net to resolve the package
12:57:33 <guillem> my perception has always been that this would be nice to have but, has not been a real problem (in terms of limiting/blocking repro efforts in general)
12:57:41 <h01ger> guillem: thats probably true, at least several people in this room said yes :) so lets try to explain better
12:58:19 <jochensp> (and it is an incomplete resolution)
12:58:23 <h01ger> atm we need metasnap.d.n and snapshot.d.o and buildinfos.d.n to rebuild a package
12:58:29 <h01ger> thats very brittle
12:58:37 <guillem> jochensp: incomplete in what sense?
12:58:52 <kpcyrd> the only kind of "trust" in the buildinfo file I could get up with, is "assuming the source code is perfectly secure, and a recent compiler is perfectly accurate, but 10 years ago there was a defective compiler that silently drops bounds checks" - a malicious buildinfo file could ask our rebuilder to use this compiler, and we would confirm it as reproducible
12:59:01 <jochensp> see p#4 in the pad
12:59:38 <kpcyrd> that's why I consider the buildinfo "source input" that needs to be reviewed/reasoned about :)
12:59:42 <jochensp> ..and even within Debian the package/version/architecture is not unique
13:00:23 <h01ger> kpcyrd: if there were hashes in the buildinfo file (and no more version numbers maybe) and snapshot were down, the buildinfo file could become the source of trust, which i agree with guillem would be stupi^wwrong
13:01:38 <h01ger> OTOH if we are going to put the .buildinfo files in the archive, they are signed by the packages file like the debs
13:02:15 <guillem> jochensp: IMO the archive should always guarantee that uniquness as I think the package manager stack kind of assume that hold true (dpkg, apt, etc), AFAIR there's a report against dak for that, and I'm not implying that would be it, as mentioned this does not cover out of archive artifacts
13:02:40 <h01ger> dak has a bug..
13:02:54 <Snu[m]> But dpkg is not just used in Debian
13:02:56 <h01ger> #1072205
13:03:15 <guillem> Snu[m]: sure, just mentioning the in archive side of this
13:04:19 <guillem> (ah the word I was looking for before was s/guide/advisory/ :)
13:05:27 <guillem> also not trying to shift the burden to someone else, but would it be possible to integrate metasnap.d.n into snapshot.d.o somehow?
13:06:05 <Snu[m]> That does not resolve the fundamental problem of installed package claims
13:06:05 <guillem> (as in I think regardless it would still be nice to have some kind of digest in the dpkg db, now sure how though)
13:06:16 <Snu[m]> dpkg makes claims about software versions installed
13:06:33 <Snu[m]> Those claims are not precise, because they don't uniquely identify the input dpkg got
13:07:18 <Snu[m]> This problem manifests itself as a reason for existence of the resolution, but it exists on its own
13:07:32 <Snu[m]> s/resolution/resolution services/
13:08:21 <kpcyrd> regarding trust - I think the builder does two things :) 1) resolve the build environment 2) compile the binary - we humans can reason about the output of the first one, but we can't reason about the output of the second one (that's what we want to use reproducible builds for) :)
13:08:25 <guillem> can you expand on how you see those claims playing out? For example I've never looked at dpkg doing those claims, probably biased by what dpkg expected from its inputs
13:08:50 <guillem> Snu[m]: is this related somehow to a SBOMs worldview and similar?
13:09:25 <kpcyrd> even if we pull the build environment resolution out of the builder, to make the builder fully trustless(tm), it would still need to be done somewhere, so I'm not sure we actually gain anything
13:09:26 <Snu[m]> I don't know:) I don't work with those directly
13:09:41 <guillem> from dpkg PoV the inputs are something it acts on, and it's happy to reinstall the same pkg/ver/arch tuple even with different contents, and that should always work
13:10:10 <guillem> AFAIR apt for example is not as happy with that invariant being broken (so they probably have a different worldview :)
13:11:04 <Snu[m]> As a user, I would want stricter requirements (or visibility) into that
13:11:20 <Snu[m]> Reporting a problem with software that does not have inqueness guarantee is problematic
13:12:00 <Snu[m]> s/inque/unique/
13:12:36 <guillem> I don't think dpkg can break that, it would make many packagers unhappy when they are iterating over a specific version before release
13:12:50 <guillem> (I think not even behind a --force option)
13:12:57 <Snu[m]> How does visibility break that?
13:13:32 <guillem> Snu[m]: ah, you mean at most a warning, or just tracking?
13:13:37 <guillem> I guess that would be fine yes
13:13:38 <Snu[m]> We are here to only provide visibility into that, not to enforce uniqueness
13:14:19 <Snu[m]> So that when I list packages on my system I have an option to get the unique identifier alongside user-friendly ones (like name/version/arch)
13:14:33 <guillem> sorry that was from a general dpkg PoV, and I thought your "as a user" comment was in general not specific to repro stuff
13:15:01 <Snu[m]> It is also in general. As a User I would like to be able to do that
13:15:27 <Snu[m]> I don't want to force everybody to care about that, but I want to be able to care:)
13:15:39 <guillem> related to that I've been meaning to include automatically the vendor that generated an artifact, but that is tricky because that's something that would need to be done by the buildds
13:16:34 <h01ger> #save
13:16:58 <guillem> (I had a branch doing the auto-injection, but that would mean anyone building on a Debian system would get "Origin: Debian" which I guess is true depending on what one assumes the field to mean, either built on a Debian system, or built by the Debian project for example)
13:17:02 <Snu[m]> That would be nice too. If I have the hash, I have means to look up that information
13:19:00 <h01ger> guillem: this could also default to "unknown", and the (Debian) buildds could then set it to "Debian", or?
13:19:53 <Snu[m]> Do we want to get back into the space of potential solutions paths?
13:20:12 <h01ger> Snu[m]: seems like a good idea to me!
13:22:20 <Snu[m]> guillem: do you agree that allowing this info into the dpkg db is useful enough for us too look into solutions?
13:23:18 <guillem> Snu[m]: see above, I think it would make sense, to me it just depends on how, and what potential tradeoffs or interface this would require
13:24:26 <guillem> I think there's also a difference between what dpkg claims itself (as you said) vs what an external entity claims (for example a frontend like apt)
13:25:11 <guillem> so as long as that would be clear from the source of the information in dpkg, or a namespaced field in the status file or something
13:25:25 <Snu[m]> I have a plain stupid solution patch that puts it into pkgbin, and calculates it using sha256sum in the process_archive function
13:25:34 <guillem> the problem with relying on a frontend to fill that info, is that then manual dpkg invocations stop tracking tht information altogether
13:25:58 <h01ger> guillem: that would be fine for us ;) (that manual dpkg invocations dont provide that feature..)
13:26:14 <h01ger> and i think it would also be fine for everyone else, really
13:26:37 <Snu[m]> The patch is very old from gerow@ (a DD I believe), and may not be properly up-to-date, but it does work
13:26:38 <h01ger> (i can also see how this is suboptimal from a dpkg maintainer PoV ;)
13:26:54 <guillem> h01ger: heh was about to say that :)
13:26:59 <Snu[m]> In an actual solution, I would obviously not use sha256sum binary, and would hide the calculation behind an option
13:27:00 <h01ger> :)
13:27:25 <kpcyrd> (if there was a way to have the hash even with `dpkg -i ...`, I would prefer that) (:
13:27:30 <Snu[m]> This is not performant enough obviously, but it is a point we could improve on if it is reasonable enough
13:27:43 <h01ger> kpcyrd: maybe you should become a dpkg co-maintainer? ;)
13:27:51 <kpcyrd> (:
13:28:07 <h01ger> (i would also prefer that, it does seem cleaner)
13:28:41 <guillem> although if there a generic way for a frontend to feed extra metadata, then whether it gets used for this, would be kind of irrelevant for dpkg's PoV as in it might be nice to integrate properly, but it would be under a specific tool responsibility
13:29:03 <guillem> there's been similar requests for example to be able to annotate why a package was installed (due to Recommends, etc)
13:29:25 <kpcyrd> the problem is that the .deb isn't processed in one pass, but processed by dpkg-deb twice (each time partially), right?
13:29:38 <guillem> kpcyrd: yes that's one problem
13:29:55 <guillem> it also skips anything after the data.tar member
13:30:22 <guillem> which should be fine (in theory) for archive .debs, but it's not something that can be generally assumed to be fine
13:30:48 <kpcyrd> there should never be any data after data.tar anyway, right?
13:31:00 <guillem> and it would need to process the entire .deb from beginngin to end
13:31:48 <guillem> the deb(5) format explicitly mentions that any new member can be appended
13:34:20 <guillem> OTOH, I'm kind of going into the direction that dpkg (via dpkg-deb) should validate .debs more strictly so that we do not get broken unpacks when we could have avoided them because dpkg-deb could have detected .deb truncation or similar earlier on
13:34:52 <guillem> but that usually only requires scanning the ar(5) structure, not much more
13:35:37 <Snu[m]> The patch I was talking about above is available here: https://salsa.debian.org/-/snippets/877
13:37:42 <guillem> hmm I think we might already have a potential way to transfer the repo digests into the status file, but that would require to fill them into available entries, but I don't recall whether these get copied over
13:40:51 <guillem> right that would probably need a couple of lines during unpack, then we could record the name of the package installed, and its associated metadata
13:41:03 <guillem> it would also need to get support for other digests than md5 though :)
13:41:13 <guillem> that would probably be the most trivial way to implement it
13:41:41 <guillem> but it would rely entirely on the frontend to provide truthful information matching the actual .deb
13:42:16 <guillem> for individual .debs this would then come from --record-avail
13:42:21 <guillem> (potentially)
13:42:41 <guillem> ideally in the future the information could be verified, or computed by dpkg itself if needed or something
13:42:59 <h01ger> the frontend also provides truthful packages, atm, no? so providing truthful info should be trivial, no?
13:43:49 <guillem> apt in theory yes, and verifies digests and signatures etc, but not all frontends are made equal :D
13:44:27 <h01ger> well, there could be trustworthy frontends (apt) and others, no? (agreed that this adds complexitiy which might not what we want)
13:44:39 <guillem> it also feels a bit icky to rely on metadata from an external party for something that we have at hand (in contrast to the location on the repo, which we do not)
13:44:40 <h01ger> otoh, in our usecase the frontend always is apt.
13:45:14 <guillem> but it seems like the easiest way to go about this TBH, w/o introducing perf degradation into the picture
13:46:00 <guillem> I can try to cook a patch during the weekend or so, to see how this could look like
13:46:03 <Snu[m]> In my humble opinion passing this info from frontends should not be the solution, yeah
13:47:50 <Snu[m]> But I would be happy if the information is available
13:49:08 <jochensp> guillem: a patch sounds great :)
13:49:12 <guillem> Snu[m]: I think this would be suboptimal, but if it's going to be helpful for repro (and the archive information is something that should also be helpful to trace where a pacakge came from) then it feels like a good incremental improvement
13:49:27 <guillem> I'd still mark it as FIXME or so in the code to revisit in the future :D
13:49:56 <guillem> ok so, I guess let's leave leave these for now, what about p3?
13:50:44 <guillem> I'm not sure I understand how that relates to dpkg
13:51:33 <jochensp> p3 is from kpcyrd how is currently not in the room but I think it is not a main point with the current agreement
13:51:47 <guillem> (a quick note, once dpkg tracks somehow the .deb digests, then adding them into .buildinfo should not be an issue, famous last words :)
13:52:20 <Snu[m]> That would require a change of the spec, right?:)
13:53:02 <guillem> Snu[m]: yeah, but I think it could be extended in a non-breaking way
13:53:06 <guillem> say by adding a new field f.ex
13:53:34 <Snu[m]> sounds good, yes
13:54:40 <guillem> h01ger: I think you mentioned some other issue to discuss?
13:55:29 <h01ger> a sec
13:55:52 <guillem> ah about getting all installed files into the .buildinfo
13:56:06 <guillem> what would be the rationale for that?
13:56:12 <h01ger> all installed debs maybe rather?
13:56:29 <guillem> is this to track packages that can affect the build but that are not part of build depeds or build-essential
13:56:47 <guillem> h01ger: right sorry!
13:57:15 <guillem> my main concern with including eveyrthing has always been about privacy leaks
13:57:31 <h01ger> though this is actually an annoyance from the past, we've identified and fixed all probs here
13:58:02 <guillem> if we have a list of packages that we know can affect the global state, I'd be happy to include such list as part of the dpkg vendor support, or perhaps we can be a new virtual or metapackage or something that then gets used as a ref
13:58:16 <h01ger> i also think we all might need a break now, after 90min discussing this. i'm *really really* happy about the progress we made here today!
13:58:51 <h01ger> guillem: oh oh oh! nice! do you already have such a list? ca-certificates, apt and openssl should be on it
13:59:01 <jochensp> (please don't)
13:59:01 <guillem> ok, fine for me to stop, happy to continue some other day if you'd like that, or leave it here or whatever :)
13:59:33 <jochensp> I think we leave it at the current state, I have written in https://release.debian.org/reproducible-builds-host-requirements.html that there could be more package on the system
13:59:44 <jochensp> we fixed all packages in Debian
13:59:47 <h01ger> guillem: also *many thanks* to you for joining this discussion!
14:00:11 <jochensp> and I would really like to have packages define all there build dependencies (and this helped finding missing ones)
14:00:12 <Snu[m]> Thank you so much!
14:00:37 <jochensp> but yes, let's stop here. Thanks a lot guillem!
14:00:39 * h01ger plans to link to this meetbot log and the pad from https://reproducible-builds.org/events/gothenburg2026/
14:00:54 <h01ger> we can always have another meeting like this ;)
14:01:01 <guillem> no problem, glad to!
14:01:06 <h01ger> \
14:01:07 <h01ger> \o/
14:02:13 <h01ger> so what are the next steps now?
14:02:15 <guillem> jochensp: ah thanks for that link, that's helpful, I agree that ideally we'd not hardcode such things, if we needed some marking I think it would be prepferable to add probably a new field that can then be queried
14:02:53 <guillem> (say first-thing-that-comes-to-mind "Affects-Build-Globally: yes")
14:03:32 <jochensp> #
14:03:51 <guillem> h01ger: I'll try to prep a patch based on the available file, then we can see from there?
14:04:10 <jochensp> #agreed guillem will try to prep a patch based on the available file
14:05:06 <h01ger> guillem: awesome!
14:05:45 <h01ger> #agreed s#available file#available patch#  in https://salsa.debian.org/-/snippets/877
14:05:59 * h01ger cheers
14:06:20 <guillem> h01ger: no jochensp comment was correct, based on the dpkg available db file :)
14:06:28 <h01ger> oh
14:06:38 <h01ger> #agreed guillem> | h01ger: no jochensp comment was correct, based on the dpkg available db file :)
14:06:42 <h01ger> #agreed :-D
14:06:49 <h01ger> #save
14:07:10 <h01ger> so its now visible here too: https://meetbot.debian.net/debian-reproducible/2026/debian-reproducible.2026-09-25-12.28.html
14:07:58 <h01ger> so shall we conclude this meeting now, thus i can go out and by some nice chocolate or some such to bring back to hamburg? :)
14:08:14 * h01ger is very happy how this meeting went, btw, but i think i said so already
14:09:05 <guillem> ah the archive does not track SHA-512 of artifacts only SHA-256
14:09:16 * h01ger giggles and face palms
14:09:40 <h01ger> i believe 256 is good for everyone^wnow
14:09:41 <guillem> (well neither does dpkg produce them anyway :D)
14:09:52 <h01ger> 512 was just introduced by *buntu
14:10:22 <guillem> I think because there was concern about size while still providing all the other ones, which I'm not sure we can drop w/o breaking users
14:10:40 <h01ger> i believe 256 is good for now
14:11:04 <guillem> yeah, sure, was trying to shave that yak in my head, and it looked too much to block on :D
14:11:11 <h01ger> *g*
14:11:43 <h01ger> alright, lets end this meeting. we can still talk and discuss here, just no more logging and also people can move on to other stuff
14:12:09 <h01ger> have a good day everyone and thank you for flying reproducible builds airlines!
14:12:38 <h01ger> (waiting 2 more minutes in case someone else wants to comment too)
14:15:14 <h01ger> #endmeeting