15:00:01 #startmeeting 15:00:01 Meeting started Tue Jun 29 15:00:01 2021 UTC. The chair is h01ger. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:01 Useful Commands: #action #agreed #help #info #idea #link #topic. 15:00:22 https://pad.riseup.net/p/rb-irc-meetings-keep has the agenda 15:00:45 #topic welcome to this monthly meeting, please briefly introduce yourself 15:01:03 * lamby is Chris Lamb, glad to see folks here 15:01:07 * h01ger Holger Levsen is happy we're finally resuming some regular meetings 15:01:16 Hi everyone, Rasmus Dahlberg here! My affiliations are Karlstad University (PhD student) and Mullvad VPN (software engineer). A large part of my work involves research and applications of transparent logs in practise. I am, e.g., involved in the System Transparency Project that was mentioned in January on rb-general. 15:01:27 Today my goal is to simply say hello and get the hang of the meeting structure, etc. 15:01:32 nice to meet you all! 15:01:35 hello! sangy here, professor at purdue doing research on software supply chain security. Happy to see this resurface! 15:01:41 * bmwiedemann is Bernhard M. Wiedemann from openSUSE (also working on some SUSE Enterprise reproducibility as well as upstream) 15:01:45 rgdd: oh, hi! how nice you came by! 15:01:51 Hi, I'm Roland Clobus, I've been working on Pioneers for a long time. A few year ago I started on the documentation for the live images, and since then have been working on the live images too. 15:01:57 hey rgdd 15:02:02 happy to be here =) 15:02:25 o/ 15:02:44 * fepitre is Frédéric Pierret (Phd) working primarily for Qubes OS at the IT level :) 15:02:44 * h01ger will give some more minutes for introductions & saying hi 15:03:09 * obfusk is Felix C. Stegerman 15:03:15 Hi, Tobias Wiese here. Just a Student, but Interested in the Concept of making builds reproducible and trying to be helpful. 15:03:17 :) 15:03:31 rgdd: nice to meet you. Would love to have some pointers on your work. 15:03:38 * sangy is happy to see so many students! 15:03:50 * ericonr is Érico Nogueira from Void Linux, mostly spectating and figuring how to move things forward in our distro 15:04:14 * dongcarl is Carl Dong from Bitcoin Core and sometimes Guix 15:04:23 * marmarek is Marek Marczykowski-Górecki - Qubes OS project lead 15:04:42 hello, Graham Christensen from NixOS -- mostly observing today but around 15:04:51 bmwiedemann: definitely, i was thinking I would propose an actual agenda item about tlogs and R-B applications for a future meeting 15:04:51 work meeting cancelled so I get to join this meeting (much cooler meeting) :-) 15:05:04 * obfusk is bad at introductions but has a website with a brief into at https://obfusk.ch 15:05:07 (happy to provide pointers sooner though, lets leave that for after intros then!) 15:05:14 Nice to put some 'IRC faces' to names. 15:05:16 * obfusk likes diagnosing & fixing bugs 15:05:19 * h01ger is very happy to see so many nice short intros 15:05:20 Yooo, Morten Linderud. Arch Linux reprobuilds team, general supply chain security stuff :) Wrote my master thesis about applying transparency logs to debian, reprobuilds and rebuilders! 15:05:33 Hi, Eli Schwartz here (eschwartz/elibrokeit most of the time). I'm a packager for Arch Linux and contribute when I can with package-specific reproducibility patches. I'm also one of the developers of our package manager and have helped get our tooling in shape to record buildinfo etc. and written one of our two rebuilder tools 15:05:46 * obfusk has worked on RB for f-droid and python-for-android 15:06:16 * realtime-neil is a random Debian user who likes reproducible builds and packaging them as part of his day job. 15:06:21 you can also all still add stuff to the agenda at https://pad.riseup.net/p/rb-irc-meetings-keep :) right now it has some general bits, some nixos and 3 debian topics ;) 15:06:43 * obfusk is a package maintainer for Debian and NixOS 15:06:51 * obfusk is a student :) 15:07:23 * h01ger is responsible for all the bad parts of tests.r-b.o ;) for some of the good parts too :) 15:07:51 * fepitre thinks h01ger is doing an awesome work all the time 15:07:57 +1 15:08:00 +! 15:08:28 :) 15:08:46 thanks 15:08:54 hello, i work on debian and to a small degree guix and also maitain some of the tests.reproducible-builds.org infrastructure 15:10:49 ok, hello everyone & thanks for all the intros! i guess its safe to move on to the next topic... 15:11:17 #topic what shall this monthly meeting be about? 15:11:30 or as i wrote in the agenda: 15:11:32 General: is "monthly IRC meeting" a good term? should this be more of a (structured) lounge or a meeting? or, ..., OR? (h01ger) 15:11:34 less about the term but more about *what* we'd like this to be 15:12:04 i wrote this because i thought a 1h meeting was a bit short and rushed in the past 15:12:13 so i came up with 15:12:24 "the meetings are supposed to last between 1-2h, maybe rather an hour, but we have lots of time (just after 23-42m we move on anyway)" (also from the agenda) 15:12:52 (after 23-42m on *one* topic we should move on to the next.. as a rule of thumb obviously) 15:13:03 I think it would be good to summarise the activities of the previous month and then a short overview of planned activities 15:13:28 +1 15:14:08 that sounds like a good addition, so s#welcome to this monthly meeting, please briefly introduce yourself#welcome to this monthly meeting, please briefly introduce yourself or update us on your last months activities and/or next months plans# ?! i like it 15:14:16 it's a bit tricky to not re-invent our monthly reports in real-time, though. striking a balance between what makes sense for an irc meeting vs. a mailing list post vs. montlhy report 15:14:56 much easier to read+write status asynchronously 15:15:33 bmwiedemann: was that an argument pro having those here or rather in the monthly report? 15:15:34 we can still discuss interesting questions about that here, e.g. if someone wants to join in an activity 15:15:34 I think it would be nice to know what people are working on, esp anything that could use more input/help. 15:16:03 ML seems better for some of that 15:16:20 i.e. rb-general 15:17:06 * h01ger likes the ability to give a quick overview, a mail feels more heavy to write 15:17:17 ^ 15:17:21 and for longer stuff, mails are certainly better 15:17:26 agree 15:17:27 sure, just trying to point out that there is a balance to be struck :) 15:17:28 ML seems more for "formal request for help" 15:17:29 Not everyone is 24/7 on IRC (is the chat archived in any way?), but I noticed that the IRC channel is a nice supplement to status-mails on the mailing list. I see the monthly report more like something for the outsider, to get a glimpse of the activities of the reproducible team, without too many technical details. 15:17:50 rclobus: this chat meeting is archived on https://metboot.debian.net 15:18:10 #save 15:18:13 h01ger: This meeting indeed, but outside the meeting? 15:18:27 not outside 15:19:07 so, do you think its a good idea to give 1-2h room for the meeting (as opposed to the 1h we had before)? 15:19:17 * obfusk (and others using a bouncer) should have logs 15:19:50 I mean that a lot of the discussions on IRC are invisible to many who are not logged on permanently. 15:20:01 h01ger: holding N people's attention simultaneously for more than an hour seems like a lot of time to block off 15:20:12 * obfusk would like to help w/ more stuff. but it can be hard to know where to start (unless someone explicitly asks for help). irc seems more suited for that than the ML. 15:20:47 vagrantc: i'm happy for shorter meetings but i also think the meetings we had last year were too rushed to make them end after one hour 15:20:51 Remember that the point of IRC is, mostly, to get real time communications. Bouncer logs for those not here may be nice, but aren't the most important point of IRC. 15:21:30 * h01ger agrees that hardly anyone will read those logs except maybe AIs ;) 15:21:57 elibrokeit: Indeed, that's why I summarised in the info gathered on IRC to the mailing list. 15:22:18 if there's something really interesting going on on IRC, maybe have someone copy paste it somewhere for wider sharing? (if all participants agree) 15:22:41 (outside of meetings I mean) 15:22:50 e.g. this is a *meeting*, meetings are all about realtime rather than posting requests and status updates to the communal message board of the mailing list 15:22:57 discussing irc outside of the meeting is a bit off-topic now 15:23:08 start an etherpad with a collaborative summary and mail that to rb-general when done 15:23:32 similar to meeting minutes 15:23:42 so i guess the other implied question, should this be called lounge or meeting or whatever has also been answered: meeting is a good term, the other time might be described as a lounge maybe, or simply irc chat 15:24:11 i guess what i would like to see is a semi-structured short time slot for checkins from various projects 15:24:27 maybe an easy way to let people do short reports is to just have a bullet list in the agenda 15:24:29 by short, i mean maybe ten minutes 15:24:34 where people can add in advance short update + pointers 15:24:52 doesn't have to take too much time of the actual meeting unless there are discuss actions from that, in which case it should probably be an actual agenda item 15:25:13 I would assume these meetings to fall in between the "formal" ML and the "really informal" regular IRC. so more structure than IRC, but less detail and more off-the-cuff than ML. 15:25:16 * h01ger likes 15:25:20 rgdd: i like your "add in advance" angle :) 15:25:33 And one of the uses of this could be to help collect knowledge about what we're all up to, that could then be consolidated into status reports on the ML? 15:25:50 +1 15:25:56 elibrokeit: that's what our monthly newsletters does? 15:26:08 yes, I like having an 'updates section' of the agenda, with things we can write (and read) in preparation of the meeting, and that may or may not lead to thing to be discussed 15:26:28 if they're part of the meeting minutes and likely also in the monthly status reports that seems sufficient to me 15:26:37 Foxboron: partially. but that is for a wider audience, so written differently 15:26:40 really, i see this as an opportunity to share a bit about what we're doing and get everyone excited that we're part of some larger community working on similar things with similar goals 15:27:00 rather than a few people acting more-or-less in isolation 15:27:08 * obfusk thinks it's a lot easier to ask a few small questions on IRC vs the ML 15:27:16 the semi-realtimeness of IRC can help for that 15:27:16 bmwiedemann: I think keeping a balance if going to be hard :p 15:27:18 * h01ger has edited the next meetings agenda based on suggestions here (same url as current meetings agenda, just scroll down) 15:27:38 vagrantc: +1 15:27:42 vagrantc: i see this happening right now 15:27:57 h01ger: agreed :) 15:28:32 team-building happens in real-time :-) 15:28:34 #info h01ger has edited the next meetings agenda based on suggestions here (same url as current meetings agenda, just scroll down) 15:29:04 h01ger: I was curious about the "proper rebuilder 15:29:10 shall we move on to the next topic or do you have some more good ideas? 15:29:23 sangy: thats a topic in 4 topics :) 15:29:33 ah, wait, we are discussing on moving the lower list stuff into the upper list? my bad 15:30:34 np 15:31:06 ok, I'm next up I guess :) 15:31:21 raboof: i guess so too 15:31:23 so, the part that should have been an update: in NixOS we recently hit the (somewhat arbitrary) milestone of having our minimal ISO image reproducible \o/ 15:31:33 #topic NixOS: reproducible minimal ISO (raboof) 15:31:44 raboof: yay! 15:31:46 \o/ 15:31:50 woohoo! 15:31:59 <3 raboof thank you so much for your hard work on getting that done 15:32:00 * kushal forgot once again 15:32:01 raboof: do you have some write up for that? :) 15:32:07 great thing to achieve 15:32:14 awesome! 15:32:20 this was somewhat unexpectedly picked up by HackerNews (https://news.ycombinator.com/item?id=27573393), which was a bit unfortunate because we didn't have a nice writeup about it yet ;) 15:32:31 it's definitely a v. good milestone! 15:32:52 raboof: it's never late for a writeup though! I would assume it'd also help to reason about future directions in terms of reproducibility inside of nixos? 15:33:00 so there was some confusion on why this matters and we didn't really credit all the work done by the wider r-b community properly 15:33:10 sangy: yeah definitely want to do one for the monthly report 15:33:28 raboof: do you have an idea of a next target? I know we've done a run for the GNOME iso, but not since February 15:34:04 raboof: nice! and don't worry, I think overall "what is reproducibility and how do you get it" is a... constantly misunderstood notion :P 15:34:04 raboof: do you have some tests in place to ensure this will always stay like this now? and what will happen if it doesnt? 15:34:15 dirty little secret: even though people have successfully reproduced the ISO among each other, there's still some weirdness in our CI system that the 'official' ISO is actually different, there is like 3 file timestamps that are still wrong ;) - working on it. 15:34:27 raboof: :o 15:35:10 raboof: i'd love to see this documented/explained/linked on https://reproducible-builds.org/projects/#NixOS too ;) 15:35:16 reproducible until proven guilty 15:35:20 haha 15:35:22 h01ger: we have a 'dashboard' (https://r13y.com/, https://status.nixos.org/grafana/d/cUz63QLWz/reproducibility-of-nixos-unstable-iso_minimal-x86_64-linux), so we will keep an eye on that 15:35:23 .oO( good work always creates more work ) 15:35:44 #save 15:35:56 h01ger: beyond that there's some thinking about including it in our CI/review process, but no very specific proposals yet I think 15:36:09 #info in NixOS we recently hit the (somewhat arbitrary) milestone of having our minimal ISO image reproducible 15:36:20 #info this was somewhat unexpectedly picked up by HackerNews (https://news.ycombinator.com/item?id=27573393) 15:36:21 raboof: I wonder if you'd like to use reprotest in your CI instead 15:36:41 #info https://r13y.com/ 15:36:50 #info https://status.nixos.org/grafana/d/cUz63QLWz/reproducibility-of-nixos-unstable-iso_minimal-x86_64-linux 15:36:56 not sure if it's overkill, but I've seen some projects put it in there to make tests fail if this particular build doesn't reproduce 15:37:18 btw, anyone can use #info (at the beginning of the line) and those lines will automagically show up in the automated summary meetbot writes 15:37:20 as for 'next goals', I think reproducing the 'bigger' ISO's is cool, but at this point it is equally interesting to give people a nicer way to consume the reproducibility information. https://github.com/tweag/trustix is an interesting project for us there, but it's early days still, not really working yet right now 15:37:33 raboof: much agreed 15:38:10 I think that's what I wanted to share, any things I missed? 15:38:33 raboof: this is something I've been hoping to get to with rebuilderd + in-toto. Have you had a chance to take a look at the apt-transport? 15:38:43 raboof: getting another round of applause! :) 15:38:47 * h01ger claps 15:38:57 I think the QubesOS ppl have also done amazing stuff to get rebuilder attestations shared between rebuilders 15:39:03 * sangy applause 15:39:16 yes indeed, we do have start generating intoto metadata for Debian 15:39:17 of course it's the work of many people, so I'm joining in the general applause ;) 15:39:38 :)) 15:40:15 so then next topic i guess.. 15:40:41 #topic Debian: a brief intro to https://debian.notset.fr/snapshot/ (fepitre/h01ger) 15:41:02 * sangy clicks 15:41:07 sadly fepitre cannot be here right now to present his work, so i'll have to do :) 15:41:14 I'm here? 15:41:18 oh 15:41:20 hi 15:41:24 i thought.. anyhow 15:41:37 #info https://debian.notset.fr/snapshot/ is a snapshot.d.o amd64 mirror 15:41:42 ok so the original issue is snapshot.d.o having throttling limits for scale rebuild of debian 15:41:42 :) 15:41:44 #info there 15:41:50 #info there's API documentation: https://github.com/fepitre/debian-snapshot#api 15:42:08 for example when you try to rebuild more than one package at the time, you are screwed up 15:42:19 and its still work in progress at the moment, but we want more users for this service :) 15:42:21 wait this sounds super useful 15:42:27 so I've started mirroring snapshot.d.o for data at first 15:42:39 I'm already planning to use this service 15:42:42 but then I finally wrote the corresponding API to the data like snapshot.d.o would do 15:42:43 we're also in the process of adding another machine in front of it, at OSUOSL.org 15:42:59 (currently all of this is home served) 15:43:19 on this snapshot instance, you have access to Debian data since 2017 to now 15:43:32 fepitre: hmm, what's the scale of traffic we're expecting here? I think I can probably donate some compute/network... 15:43:33 which hopefully shouldnt take that long (but then there's a heatwave in portland atm and all our machines there are actually turned off at the moment..) 15:43:35 for unstable, buster and bullseye and arch source, all and amd64 15:43:58 sangy: we'll sort this out at osuosl.. things have already been started there. but thanks! 15:44:00 sangy, scale means several rebuilder in parallel hitting snapshot.d.o throttling issues 15:44:05 Feature request: will there also be an API for 'the latest, completed mirror timestap'? 15:44:35 got it, may be better if I host a rebuilder then :) 15:44:41 fepitre: what are the disk ressources used atm? last you told me 4TB space 15:44:57 I can check it's roughly still the same 15:45:01 * h01ger nods 15:45:31 no need to say that has been a real pain to mirror all this amount of data 15:45:32 pretty deep into debian's freeze cycle right now, not likely to change a lot day to day 15:45:32 I have a snapshot mirror for openSUSE in IPFS at opensuse.zq1.de - if there is was some 6 TB sponsored space, It could be mirrored there with 2 years worth of history. 15:45:47 * h01ger has shared what he wanted to share. happy to have some more discussion here and now, but more details could also be discussed later/anytime. this is an ongoing unfinished project :) 15:45:57 but we can discuss that later - don't want to be offtopic here. 15:46:16 vagrantc: i dont see how this its related to the release cycle? 15:46:43 h01ger: the amount of data stored changes less during freezes? 15:46:50 few last words, I'm improving the API results sto help rebuilder properly finding package 15:47:08 vagrantc: i'm actually not convinced, there might be more uploads. but thats a guess too :) 15:47:20 because right now, snapshot.d.o just tell you in which archive a package is, not really the dist 15:47:31 h01ger: i get emails daily about all the .buildinfo files ... it has slowed to a crawl :) 15:47:46 so in rebuilder, if you may want to rebuild only specific suite, it helps 15:47:47 :) 15:48:12 fepitre: we have 'rebuilder' as a seperate topic.. :) 15:48:18 fepitre: thanks so much for your work on this ... it unblocks numerous other projects :) 15:48:19 (sorry :)) 15:48:24 shall we move on? 15:48:33 +1 15:48:44 and yes, fepitre: *many* thanks for implementing this!! 15:48:52 sure, you are welcome 15:48:58 #topic live-builds (rclobus) 15:49:12 Hi, I wrote a mail yesterday about the status. 15:49:27 In short: the very first live image is now being built by Jenkins 15:49:38 (Many) more to come 15:49:38 #info https://lists.reproducible-builds.org/pipermail/rb-general/2021-June/002288.html 15:49:42 fepitre: just to be extra sure, that github link over there is where it all lives 15:50:18 rclobus: and those/that image is reproducible too! \o/ 15:50:23 sangy, for the whole snapshot service (data management, api etc, yes) 15:50:35 So soon, all major live images will be checked by Jenkins for reproducibility. 15:50:39 somehow we didnt hit hackernews with that ;) 15:51:15 That is, the live images generated by live-build. The current official Debian live images are created by live-wrapper, but that's a totally different topic. 15:51:47 rclobus: do you have plans to work on those too? 15:51:48 rclobus: care to give a rundown on how live-build used to introduce non-determinism? or is this more of a "a base group of packages in the default live-build config are all repro" 15:52:02 h01ger: debian's been doing RB for so long it's no longer "news"? :p 15:52:26 sangy: +1 15:52:33 #info https://wiki.debian.org/ReproducibleInstalls/LiveImages 15:52:36 sangy: ^ 15:52:41 live-wrapper used vmdebootstrap, which requires Python 2.7, which is not present any more on Bullseye. A suitable solution must be found... 15:52:52 rclobus: ah 15:53:26 Hm. 15:53:28 The main changes I did for live-build are e.g. adding '-a' to 'cp' commands, but the most tricky one was the generation of the UEFI image. 15:54:06 rclobus: so are there currently no live-wrapper images for bullseye or are they being built on buster? 15:54:24 I only have 'hacks' for 2 packages at the moment (they already have their bug ticket) to make their configuration files reproducible. 15:54:30 rclobus: did you try to submit those changes to live-build? 15:54:36 ah, cool 15:54:42 > # Use LD_PRELOAD to replace uuid_generate_random with a less random version 15:54:45 ah I see! 15:54:48 Live-wrapper is currently used for buster and bullseye 15:55:10 I'm in contact with live-build, the git repo is the latest and greatest. 15:55:32 LD_PRELOAD was a really ugly hack, but it works :-) 15:55:42 hehe 15:55:58 hey, it's better than ln -s /dev/zero /dev/random orso :P 15:56:30 been there, done that 15:56:32 :) 15:56:48 Perhaps eventually the small C-file could be properly packaged to avoid compiling while building the image. 15:56:48 rclobus: or anyone: anything else or should we move on? 15:57:00 I see there are some "successors" to vmdebootstrap but I can't quickly find whether they handle RB 15:57:04 rclobus: i guess so. or added to some existing package 15:57:08 That it. Thanks for your attention. 15:57:25 thanks for the work and keeping us informed! 15:57:32 yes, nice! 15:57:53 rclobus: do you know if the vmdebootstrap "successors" handle RB? 15:57:56 Why do you need a successor, can't it be ported to python3? 15:58:07 "upstream discontinued" 15:58:11 * h01ger waits with changing topic 15:58:24 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=907751 15:58:25 Oh lol, okay then 15:58:27 https://github.com/bmwiedemann/reproducible-faketools/blob/master/reproducible-faketools.spec#L146 15:59:06 So far, I managed with the regular 'debootstrap' 15:59:34 * h01ger recommends mmdebstrap 15:59:54 * vagrantc likes mmdebstrap too 16:00:02 but ... maybe next topic? :) 16:00:07 sure 16:00:16 #topic Debian: let's do a rebuilder now (h01ger) 16:00:27 :) 16:00:38 let's! :) 16:00:54 \o/ 16:01:07 \o/! 16:01:09 catch up with those *other* distros :) 16:01:12 so as some of you know since forever^w2015 we have been rebuilding debian packages twice... and compared them and made stats and send patches and everything... 16:01:57 but, we (on tests.r-b.o) never did rebuilders and compared against the packages released via ftp.d.o 16:02:15 https://github.com/fepitre/debrebuild - fepitre do you want to say few words? 16:02:21 then there have been several rebuilder implementations, surprisingly many from different Arch Linux folks :) 16:02:28 I think we have two very good candidate projects to do just that :) 16:02:30 marmarek: gimme a sec 16:02:46 i'm well aware and want to give the big(ger) picture 16:03:13 * marmarek nods 16:03:21 and some of these rebuilders are forks of projects which have then been rewritten. 16:03:47 * h01ger waves to josch, sangy, Foxboron, fepitre and now someone will tell me whom i forgot 16:03:56 kpcyrd! :) 16:03:59 kpcyrd: is not here, but 16:04:01 yeah, there! 16:04:30 o/ 16:04:45 and i've also started something.. but got stuck at snapshot.d.o, which fepitre (with some help from josch and me..) unstuck 16:04:50 so 16:04:51 now 16:05:05 * elibrokeit waves at h01ger 16:05:30 i'd like to setup something.. on/for tests.r-b.o which later hopefully can also move to debian.org, dunno, dont care right now 16:05:58 and yes, one bit i'm lost is what codebase to choose now 16:05:59 yeah, I think that'd be ideal. I'm also setting up another rebuilder, and we already ahve one at NYU for arch, which we can definitely upgrade 16:07:17 well, I have a gsoc student working on in-toto + rebuilderd stuff, but I don't think we need to have just one implementation to exist (in fact, I like we have some diversity there) 16:07:32 not sure if i should start a wiki page listing all the different rebuilder implementations there are (for Debian only, i mean) 16:07:35 I think debrebuild is a little bit more mature for te deb ecosystem (I think it works with the in-toto transport right now) 16:07:36 ah, I'm here, was just afk a moment doing laundry 16:08:01 h01ger: are you about integrating it into jenkins (as the overall oversight), or standalone? 16:08:08 sangy: thats debrebuild coming from the descripts source package, right? 16:08:13 If you need a standalone rebuilder orchestrator there is what I developed: https://github.com/fepitre/package-rebuilder/tree/devel110221 16:08:33 which currently use https://github.com/fepitre/debrebuild as Debian rebuilder tool 16:08:37 but others can be integrated 16:08:42 marmarek: either, shrugs. running on some of the same hosts 16:09:09 h01ger: no ,I mean fepitre's 16:09:17 fepitre: and thats not using debrebuild from src:devscripts, right? 16:09:17 we've started establishing the terms "rebuilder" and "rebuilder-backend" to distinguish between actually building and orchestration 16:09:20 I'm asking specifically because having standalone would be easier for others to run it too (the more rebuilders the better, no?) 16:09:21 I think its useful to differentiate between the rebuilder tools and the orchestrators (even though they are often integrated in some implementations) 16:09:28 kpcyrd: +1 16:09:31 h01ger, no it is using my python implementation 16:09:32 kpcyrd: Ah, I see there is already a term. :) 16:10:05 * h01ger agrees on differentiating between those two terms 16:10:14 i'd first like to talk about the rebuilding tool 16:10:32 kpcyrd: which of the two is doing the orchestration? 16:10:35 without it, orchestration is a bit like swimming without water 16:10:36 h01ger: so I think rebuilderd is mostly an orchestration 16:10:52 yup 16:10:52 kpcyrd: ...right? 16:11:00 on my side, package-rebuilder is the orchestrator 16:11:03 sangy: yes, it's building on top of archlinux-repro by Foxboron 16:11:04 sangy: it reads the repository data and schedules packages 16:11:19 its really hard to follow this conversation, because these similar tools have similar names 16:11:36 right, and my understanding is that tasks are mostly containerized, so we could use fepitre's debrebuild to do the magic? 16:11:37 it passes the parameters to the rebuilder backend and then expects it to output the rebuild package, then picks up from there again 16:11:47 *rebuilt package 16:12:06 h01ger: what about we make a wiki page for rebuilder implementations to start settling terminology and such 16:12:14 +1 16:12:18 kpcyrd: where does that source code live? (eg github url) 16:12:24 sangy: YES 16:12:30 that's a very good idea 16:12:33 sangy: $(figlet YES) even 16:12:38 h01ger: https://github.com/kpcyrd/rebuilderd and https://github.com/archlinux/archlinux-repro 16:12:44 kpcyrd: thanks 16:12:58 any other urls to be shared here which should be part of the wiki page? 16:13:03 almost seems like we could have a whole meeting talking about different rebuild implementations :) 16:13:10 there's also a frontend by jelle https://gitlab.archlinux.org/archlinux/rebuilderd-website 16:13:13 vagrantc: almost yes 16:13:34 I think as an actionp oint to actually populate this wiki page... 16:13:44 what's a good place to put it? r-b.org? 16:14:05 https://salsa.debian.org/debian/devscripts/-/blob/master/scripts/debrebuild.pl is the one from src:devscripts so available in bullseye right now 16:14:15 sangy: there's no wiki 16:14:21 then there's two proof of concepts that consume the results from the api, https://github.com/kpcyrd/ismyarchverifiedyet and https://github.com/archlinux/arch-repro-status 16:14:25 fair, should we do it in markdown? 16:14:47 sangy: wiki.d.o seems natural here and now, but i see how you want to extend this to not only cover debian ;) 16:14:50 https://github.com/bmwiedemann/reproducibleopensuse/blob/master/nachbau is the build portion of my rebuilder/verifier 16:15:23 sangy: i suppose a git wiki/repo on salsa/reproducible-builds is a good idea 16:15:33 * kushal has one small update from the SecureDrop project, if anyone can please ping him if we have time at the end. 16:15:39 kushal: noted 16:15:55 h01ger, I think wiki on salsa is fine and easy for everyone? 16:16:01 yes 16:16:07 probably just on the website 16:16:17 i think thats a good idea 16:16:35 a subpage on https://reproducible-builds.org/docs/ 16:16:44 on the website, I guess you can include the wiki page itself from salsa 16:17:01 the website is markdown so whats the diff to a wiki anyway :) 16:17:16 yes any solution is fine 16:17:40 broader edit access for a random wiki, than to a website? 16:17:54 #agreed we shall document the different existing rebuilder and orchestration tools for debian 16:18:23 #agreed (and then probably include other distros too if the rebuilders can rebuild more than one distro...) 16:18:37 #info h01ger suggests to use https://reproducible-builds.org/docs/ 16:18:53 a fair number of people have edit access to the website, and merge requests can be done by nearly anymore 16:18:56 anyone 16:19:07 marmarek: we are adding anyone asking to the salsa project, so i guess thats fine. there's hardly any free to edit wiki left too 16:20:22 a working copy could be on a pad, but that's hardly permanent place 16:20:46 h01ger: +1 for either salsa or the website 16:21:29 if enough people has write access, then the website sounds good indeed 16:21:33 the idea is that the rebuilder-backends are very distro specific but the orchestrators don't necessarily have to 16:21:52 #info https://pad.riseup.net/p/rebuilders%2Borchestration-tools 16:22:05 please go ahead :) 16:22:21 i'll "later" (maybe in a week or 2) turn that into website.git 16:22:25 #save 16:23:01 kpcyrd: right 16:23:36 * h01ger is quite very happy how productive this topic has become and suggests to end it here and move on to the next :) 16:23:46 +1 16:26:02 #topic SecureDrop 16:26:08 kushal: ^ 16:26:43 Till now SecureDrop Workstation project had all packages as reproducible and also the internal Python wheels https://github.com/freedomofpress/securedrop-debian-packaging 16:27:25 There were some changes in the upstream pip, that required us to redo the toolings to keep getting reproducible wheels in future. 16:27:57 cool! 16:28:06 Now, based on that work and as SecureDrop runs on a modern OS (not too modern), aka Ubuntu Focal, we are working on making the primary server side package also reproducible. 16:28:14 * sangy needs to drop, but this was fun! 16:28:22 sangy: o/ 16:28:38 Following the same packaging guidelines and tools in the repo linked above. 16:28:45 Hopefully next week. 16:28:52 s/week/release. 16:29:04 kushal: nice! thanks for the update! 16:29:14 16:29:18 * h01ger guesses we can move on? 16:29:21 h01ger, thank you :) 16:29:33 * obfusk would like to ask two small questions at the end if there's time 16:30:06 #topic Any Other Business (AOB) 16:30:17 kushal: Thanks for the update. Just wondering; are you seeing issues with strip-nondeterminism? You appear to working around some problem/issue in your debian/rules 16:30:56 lamby, we had some trouble, I will get back to you what they were. I totally forgot now (also I am sleepy). 16:31:15 sure 16:31:16 * h01ger notices any (on-topic) topic can be discussed now but suggests we try to keep it at one topic at a time :) 16:31:21 lamby, ah now remember. 16:31:51 kushal: do you have a link for the pip changes? 16:32:05 lamby, as we are installing Python dependencies from the wheels we build, they contain dates and other details which we had to remove. 16:32:14 obfusk, yes, give me a minute please 16:32:59 * fepitre has to leave, don't hesitate to ask questions now or later I would answer ASAP. 16:33:02 * obfusk is not in a hurry 16:33:07 fepitre: o/ 16:33:24 cheers _o/ 16:33:35 obfusk, https://github.com/pypa/pip/issues/9604 16:33:54 kushal: thx! 16:33:55 kushal: (Ah, in that case I think there would be a "more Debian" way of achieving that, but it's no big deal...) 16:34:13 lamby, perfect, I will ping you after meeting. 16:35:09 obfusk: your two other questions were 16:35:10 ? 16:35:49 a few days ago I mentioned here that "I want to create a tool for creating, provisioning, and managing declarative and reproducible (at least as close I can reasonably get at first) VMs (starting w/ libvirt & Debian, more backends & OSs later)." 16:35:49 since this seems to be "any topic goes", there's also progress on reproducible alpine: https://twitter.com/sn0int/status/1408853977106718724 16:36:12 so the snapshots mirrors seem very useful for that 16:36:27 #info progress on reproducible alpine: https://twitter.com/sn0int/status/1408853977106718724 16:36:38 and so would a vmdebootstrap successor 16:37:21 obfusk: seen https://tracker.debian.org/pkg/bdebstrap ? a YAML config based multi-mirror Debian chroot creation tool 16:37:32 if anyone familiar with vmdebootstrap successors or building reproducible kvm images has any tips, that would be appreciated. 16:37:35 kpcyrd: thanks for that update! 16:37:52 if anyone has an interest in this tool I'm planning to build, feedback is welcome. 16:38:00 that's #1 16:38:06 obfusk: isnt there vmdebootstrap2 also? 16:38:38 h01ger: there's vmdb2, but I can't find anything quickly about RB 16:38:38 obfusk: I looked into roughtly this for Tails -- see https://public-redmine-archive.tails.boum.org/code/issues/15349/ and https://gitlab.tails.boum.org/tails/tails/-/issues/15349 16:39:06 They had/have the same problem re. ongoing viability of using vmdebootstrap 16:39:24 obfusk: and again i'd recommend looking at https://tracker.debian.org/pkg/bdebstrap ? 16:39:50 kpcyrd: wow "that tweet" has much more content than expected! :) very nice work too! (and diffoscope pics!) 16:40:03 lamby: thx. but does tails need to create reproducible ext4 images? 16:40:43 Not that I'm aware of... Was mostly just linking to overlapping problem spaces. 16:41:14 are there other 'AOB' topics? it feels like we're getting into too deep details now - or maybe i'm just tired after 100 minutes :) 16:41:17 lamby: ack and thx! 16:41:41 h01ger: I had one more small question :) 16:41:47 obfusk: go ahead! 16:41:53 i don't have anything more on topic to add right now, will probably have some for future meetings tho! 16:41:54 h01ger: there's been more progress after that thread with help from Ariadne in #reproducible-alpine, unfortunately editing takes forever due to the 280 char limit on twitter 16:42:06 * vagrantc suspects 15UTC will work out at least for summer, come winter it will be increasingly difficult :) 16:42:12 kpcyrd: *g* 16:42:26 rgdd: cool, looking forward! 16:43:07 vagrantc: i can understand your desire for winter right now ;) (and surely we can discuss time changes.. just please not monthly ;) 16:43:47 so... I'd like to help out more with making packages reproducible, but unless I happen to come across a bug that looks like something I can help with, it's kind of hard to find something that *I* can help with. 16:44:00 obfusk: packages where? 16:44:14 h01ger: i was more saying "this is working!" 16:44:18 h01ger: mostly Debian. 16:44:23 vagrantc: ah cool! :) 16:45:05 obfusk: you've read https://reproducible-builds.org/contribute/debian/ ? (it probably still needs more updating but its a start) 16:45:12 obfusk: happy to discuss strategies to find and work on packages 16:45:37 obfusk: i at least have a workflow that works for me ... i tend to do bursts of just finding and submitting patches 16:46:11 * h01ger is happy to give this 5min now but would like to close the meeting then. or close now and you discuss then? 16:46:11 h01ger: yes, but it's hard to narrow down the number of bugs to someting I can help with. 16:46:21 * obfusk is fine w/ either 16:47:34 * h01ger supposes vagrantc is typing and waits :) 16:48:31 vagrantc mentioned something about python sphynx docs but looking for that I mostly found packages w/ lots of other issues I would not be able to easily help with. 16:49:06 Assuming the meeting is ending shortly, I'm going to have to go afk for a bit -- thanks for the meeting; am looking forward to the next one. o/ 16:49:17 lamby: o/ 16:49:20 o/ 16:49:30 * h01ger is giving vagrantc two more minutes before wrapping up 16:49:38 thanks for the meeting 16:49:42 you can start saying good-bye now :) 16:49:44 many new links for me to digest, v. helpful! 16:50:19 * h01ger is happy and thankful for the meeting and all the nice info and time! 16:51:14 was a nice meeting. We might even talk more to each other before the next one :-D 16:52:01 :) 16:52:05 alright 16:52:11 thank you all! 16:52:15 #endmeeting