18:00:03 <lamby> #startmeeting
18:00:03 <MeetBot> Meeting started Tue Jan  3 18:00:03 2017 UTC.  The chair is lamby. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:03 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
18:00:08 <lamby> Evening all.
18:00:15 <emaste_> Hello!
18:00:21 <bmwiedemann> hello
18:00:22 <lamby> #topic Introductions
18:00:34 <lamby> So, who's about? :)
18:00:38 * emaste_ waves from FreeBSD-landia
18:00:46 * vagrantc waves
18:00:51 <lamby> Hi bmwiedemann
18:00:51 * brett is here.
18:01:04 <lamby> Hey vagrantc, brett, emaste_
18:01:11 * bmwiedemann waved the openSUSE flag
18:01:34 <lamby> Good grief, the Debian folks might finally be outnumbered…
18:01:51 <siamezzze> Hello.
18:01:58 <emaste_> Heh. That's probably a good thing!
18:02:01 <lamby> Indeed.
18:02:02 <brett> lamby: Good problems to have.  :)
18:02:11 * StevenC99 waves
18:02:23 <lamby> (FYI I'm on a train; so if I cut out for a long time, that's probably why & continue without me)
18:02:29 <lamby> Hey siamezzze & StevenC99
18:02:45 <lamby> We'll give it a couple more minutes then we'll crack on.
18:03:05 <lamby> The agenda is here: https://pad.riseup.net/p/reproducible-irc-meeting-6
18:04:21 <vagrantc> lamby: should someone else presumably work meetbot, then?
18:04:36 * vagrantc forgets if multiple people can control it
18:06:20 <infinity0> hello
18:06:48 <vagrantc> if people can be sure to put there name next to proposed agenda items, that would help ensure we have a point-person to talk to to drive the conversation
18:08:10 <danielsh> ... is lamby here ?
18:08:14 <lamby> vagrantc: I'll keep it for now.
18:08:22 <lamby> Hi danielsh
18:08:24 <lamby> Let's go.
18:08:29 <danielsh> evening
18:08:43 <lamby> #topic What's the right procedure to submit non-bug improvements to diffoscope? (brett)
18:08:47 <lamby> brett: ^ gogo
18:08:55 <brett> Sure.
18:09:01 * dkg waves
18:09:23 <brett> So I've gotten some bugfixes into diffoscope by sending patches to the bugs but when I'm doing more subjective stuff I'm not sure what the right process is.
18:09:36 <lamby> There are currently two ways; either by emailing the diffoscope mailing list (yes, there is one), or by creating bugs in Debian.
18:09:39 <lamby> "Subjective2
18:09:45 <lamby> * "Subjective" ?
18:09:57 <brett> Recently I was reading the --help output and got a little confused by it, so I used the source and then tweaked it to my taste, but admittedly it's subjective.
18:10:17 <brett> Since it's mostly editing English rather than code.  :)
18:10:26 <danielsh> English bugs are still bugs. :)
18:10:47 <bmwiedemann> can still improve the user experience - similar to a good UI
18:10:51 <lamby> The "best" thing would just be to file a bug against the Debian package. Very very unlikely to get lost there.
18:11:07 <brett> It's also multiple commits (two! so technically).  Patches still work but I wasn't sure if that's how people want to review them.
18:11:14 <vagrantc> a bug can always be closed if it's deemed invalid or whatever.
18:11:17 <lamby> On the BTS is fine.
18:11:28 <brett> Sure, that works for me, will do.  Thanks.
18:11:32 <danielsh> brett, I would do 'git format-patch' and attach the two resulting files to a bug.
18:11:36 <lamby> It's a meta-problem that you are not aware and/or were blocked on this. Perhaps we should add something to the --help - anyone want to take that
18:11:49 <brett> Sure, I can add that to the branch.  :)
18:11:54 <lamby> thanks
18:12:14 <lamby> #action brett to add "reporting bugs" (or similar…) to --help output of diffoscope
18:12:22 <lamby> #topic jenkins.debian.org psuedopackage
18:12:37 <lamby> So, we have a new pseudopackage in the BTS for reporting issues against the testing framework.
18:13:10 <lamby> So, if you want something added/fixed on tests.reproducible-builds.org, you can/should file it against the "jenkins.debian.org" package in the Debian BTS.
18:13:20 <lamby> thanks to mapreri for pushing this!
18:13:53 <dkg> woo, nice!
18:14:13 <lamby> (More of an announcement really)
18:14:14 <danielsh> ... that should be said somewhere on t.r-b.o, too.
18:14:23 <lamby> danielsh: good idea. fancy taking an action item to add that?
18:14:50 <danielsh> lamby, already looking at it... the footer points to jenkins-dev@
18:14:56 <danielsh> if action is needed sure, at me
18:14:57 <lamby> cool thanks
18:15:01 <lamby> #action danielsh to mention on t.r-b.o how to file bugs using the new pseudopackage (in the footer…?)
18:15:36 <lamby> #topic CI guidelines for setting reproducibility workarounds (e.g. S_D_E) (emaste)
18:15:40 <lamby> emaste_: ^ gogo
18:16:03 <emaste_> For ongoing CI we want to build the latest out of some VCS, and there might not be a convenient date to use as input to a build.
18:16:09 <emaste_> I wonder where folks feel the line is between things that may be set in the CI infrastructure itself to achieve reproducibility, vs being an integral part of the project under test.
18:16:15 <emaste_> I'll use FreeBSD as an example here although the idea/question applies generally.
18:16:18 <emaste_> Almost all of the binaries in the FreeBSD base system now build reproducibly, but their timestamps in the resulting packages are not reproducible.
18:16:22 <emaste_> For release builds we may have a conveinent release date that can be used as a SOURCE_DATE_EPOCH for the build.
18:16:25 <emaste_> For ongoing CI is it reasonable to set SOURCE_DATE_EPOCH in the CI environment?
18:17:05 <vagrantc> is there a VCS commit you could use the date of?
18:17:13 <lamby> ("For release builds" → You use the release date?)
18:17:17 <bmwiedemann> emaste_: some people always set SOURCE_DATE_EPOCH=1 because they dont care
18:17:55 <emaste_> vagrantc: Debian's Jenkins building FreeBSD fetches the source from git and could pick up the date of the last commit there
18:18:03 <StevenC99> if you set S_D_E you ought to state somewhere the value used
18:18:16 <lamby> (I would say it was fine as a final fallback, but you surely have *some* kind of date somewhere…? Like in Git as mentioned.)
18:18:18 <StevenC99> ... so that others can reproduce it
18:18:26 <emaste_> lamby: we don't yet have a release that's using the packaged base
18:19:05 <lamby> okay
18:19:11 <vagrantc> does the buildinfo capture SOURCE_DATE_EPOCH?
18:19:16 <emaste_> so it's "will use the release date"
18:19:49 <bmwiedemann> vagrantc: yes as part of the env
18:20:08 <StevenC99> sorry - do FreeBSD builds generate a buildinfo...?
18:20:19 <emaste_> vagrantc: in FreeBSD now we haven't used buildinfo because it is implicitly provided by the build infrastructure (FreeBSD base system and ports svn rev captures the toolchain, all dependencies, etc.)
18:20:31 <lamby> StevenC99: If not, at least just echo it during build… :)
18:21:06 <StevenC99> I feel that some kind of buildinfo should explain what value was used for S_D_E
18:21:14 <lamby> sure
18:21:27 <StevenC99> even if that's the only thing you put in the buildinfo right now
18:21:33 <vagrantc> you could construct a .buildinfo oujt of the freebsd logs or something as a big hack
18:21:46 <emaste_> I'll bring up specifics on the mailing list, but it's more of a philosophical question: how much should CI report on waht is reproducible out of the box, vs what can be reproduced (with the appropriate buildinfo, etc.)
18:21:54 <lamby> nod nod
18:22:01 <danielsh> h01ger, j.d.n [PATCH] 6d703e05b0b6 thanks :)
18:22:03 <lamby> Good idea to take specifics to the list; lets do that.
18:22:32 <lamby> #topic is there a proper download URL for strip-nondeterminism tar? (bmwiedemann)
18:22:35 <danielsh> emaste_, you should include svn branch in the buildinfo too, no?
18:22:49 <bmwiedemann> emaste_: for comparison the proposed rpm-buildinfo generator https://ftp.qubes-os.org/~woju/rb/rpmbuildinfo
18:22:50 <h01ger> danielsh: thanks, deploying
18:22:50 <emaste_> danielsh: yes
18:23:13 <emaste_> bmwiedemann: thank you
18:23:32 <bmwiedemann> for strip-nondeterminism, I found that it is nice to have a proper download url where the tarball does not vanish once the next version arrives
18:24:17 <bmwiedemann> is there such a thing?
18:24:20 <vagrantc> you could probably use snapshot.debian.org, presuming there's no other "upstream"
18:24:53 <bmwiedemann> would also be nice to make a proper upstream, even if it is re-using the debian bugtracker and MLs
18:25:43 <bmwiedemann> somewhat similar to the earlier diffoscope question.
18:26:08 <vagrantc> could just use a proxy-redirector that uses snapshot.debian.org as a backend...
18:26:12 <lamby> bmwiedemann: Canonically, the tarball is in git via pristine-tar.
18:26:16 <danielsh> cgit doesn't seem to have a tarball/zip link ?
18:26:18 <vagrantc> oooh.
18:26:45 <lamby> This seems like a specific pain point of the more general problem of generic tools being a bit too "Debian".
18:27:06 <danielsh> pristine-tar still doesn't seem to have a "download .tar" link anywhere
18:27:13 <dkg> danielsh: "git archive --format=tar --prefix=${pkgname}-${upstreamversion} | gzip -9n"
18:27:31 <bmwiedemann> in general, if you want people to use your tools, make it easy to find, get, patch, build, contribute etc
18:27:32 <danielsh> dkg, Sure.  But that's still not a link :)
18:27:46 <lamby> bmwiedemann: Oh, I completely understand that :3
18:27:50 <vagrantc> dkg: does that build reproducibly?
18:27:58 <dkg> vagrantc: it appears to
18:28:05 <danielsh> So long as you use the same gzip version
18:28:08 <dkg> i don't know whether git upstream has committed to that being reproducible
18:28:13 <danielsh> (we've had problems in the past with bzip2 differing across vendors)
18:28:28 <lamby> Can we leave the git-archive specifics for the mailing list or similar? :)
18:28:35 <dkg> sure sure, sorry!
18:29:00 <lamby> bmwiedemann: As a stop-gap, would publishing the tarballs in some HTTP directory be okay for you?
18:29:15 <lamby> If so, I'll do that and adjust all release docs.
18:29:21 <bmwiedemann> would be nice, if it was some future-proof URL
18:29:29 <lamby> ack
18:29:53 <lamby> #action lamby to publish generic tools somewhere (future-proof URL) and to update release docs
18:29:59 <bmwiedemann> also, where is that git repo?
18:30:13 <lamby> https://anonscm.debian.org/git/reproducible/strip-nondeterminism.git
18:30:25 <bmwiedemann> cool. thanks.
18:30:31 <lamby> #topic upstreaming reproducibility fixes (e.g. avfs done by bmwiedemann)
18:31:22 <bmwiedemann> I have been re-building our 8000 openSUSE packages once again and found packages like avfs that have S_D_E patched on debian, but not upstream, so I upstreamed the debian patch
18:31:32 <bmwiedemann> and wondered why nobody had done that before
18:31:39 * vagrantc cheers on bmwiedemann
18:32:20 <lamby> Basically, because we are writing a *lot* of pathces. :/
18:32:20 <vagrantc> it might be as simple as someone having not done it
18:32:26 <infinity0> sometimes we were a bit lazy with upstreaming S_D_E patches, or upstream were slow at accepting them, or various other reasons https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal#Reading_the_variable
18:32:27 <lamby> I'm upstreaming them more and more these days.
18:32:52 <bmwiedemann> one reason might be that the upstream was not exactly easy to find a communication channel.
18:33:09 <emaste_> do we have a sensible way to track outstanding upstreamable Debian patches?
18:33:10 <bmwiedemann> might scale better, if we ask package maintainers to upstream patches?
18:33:57 <vagrantc> in the past, we removed entries in notes.git when it was fixed in debian ... maybe we should wait till it's fixed upstream?
18:34:40 <bmwiedemann> vagrantc: or at least change the note to fixed-in-debian and only remove when upstream took the patch
18:34:41 <vagrantc> in a similar vein, moving notes.git into the multi-distro format?
18:35:29 <danielsh> I assume all of the above applies to non-reproducibility debian package patches, too
18:35:48 <danielsh> the debian qa guys may have (at least partly) solved this problem... ?
18:36:04 <bmwiedemann> btw: this is often quite helpful: https://anonscm.debian.org/git/reproducible/notes.git/tree/packages.yml
18:36:19 <lamby> Maintainers should be sending patches upstream, yes. And avfs is likely just one example of … 500-600 packages in the same situation, alas.
18:37:19 <lamby> Does anyone have any concrete suggestions right now beyond "lets just upstream them whenever possible" ?
18:37:39 <danielsh> "ask debian-qa if they have tooling to solve this which we can piggyback on"
18:37:46 <vagrantc> inevitably, maintains aren't going to submit all patches upstream all of the time, so if we want to push reproducible-builds... we might have to come up with that ourselves
18:37:50 <bmwiedemann> lamby: tracking in notes patches in debian vs patches upstream
18:37:57 <danielsh> (track patches without "Forwarded: yes" DEP-3 metadata, by age, and so forth)
18:38:08 <lamby> danielsh: Fancy looking into that?
18:38:23 <danielsh> lamby, would rather not shepherd this one, sorry
18:38:42 <lamby> No worries
18:39:25 <lamby> #idea < danielsh> (track patches without "Forwarded: yes" DEP-3 metadata, by age, and so forth)
18:39:37 <lamby> (so we don't lose it)
18:39:46 <danielsh> (DEP-3 is a distro-agnostic patch metadata format: http://dep.debian.net/deps/dep3/)
18:40:34 <lamby> The "D" /totally/ stands for "distro-agnostic" … :)
18:40:47 <vagrantc> #link http://dep.debian.net/deps/dep3/
18:40:49 <emaste_> :)
18:41:09 <lamby> a Anything else on this? I feel it's somewhat of "lets try and be 'good' more" when writing patches.
18:41:11 <vagrantc> and don't let the description of DEP-3 fool you...
18:41:24 <lamby> #topic Any other business?
18:41:44 <bmwiedemann> you skipped one topic
18:41:50 <bmwiedemann> That topic is slightly related to the previous one. I found a package (forgot which one) that used a custom env var to override build date and debian build scripts were using S_D_E to set that, but openSUSE like everyone else would need to duplicate this. So I wondered if we should try to convert these to S_D_E upstream
18:42:17 <lamby> Yes, definitely.
18:42:21 <lamby> More upstream the better.
18:42:28 <lamby> :)
18:42:49 <StevenC99> RWS2 reimbursement instructions; h01ger: ping?
18:42:49 <bmwiedemann> so we try to make S_D_E the one and only way to override build date
18:42:50 <emaste_> bmwiedemann: as in a package has some sort of "OVERRIDE_THE_DATE" existing env var?
18:42:52 <danielsh> If the package uses S_D_E I assume whoever wrote it that way had a reason
18:42:58 <bmwiedemann> emaste_: yes
18:43:03 <danielsh> (because upstream declined S_D_E, or had a precedent for a different name, whatever)
18:43:43 <vagrantc> then ideally we at least can convince them to support compatibility with S_D_E as well ?
18:43:50 <bmwiedemann> danielsh: maybe it was done before S_D_E was standardized?
18:43:55 <emaste_> I think it is worth suggesting upstream adopt S_D_E, perhaps as a fallback if the existing/primary one is not set
18:44:28 <danielsh> I think we're all in agreement.
18:44:34 <danielsh> Upstream should be asked to use S_D_E if they haven't already been asked to.
18:44:59 <vagrantc> and if they're stubborn and refuse ... do we need to track it somewhere cross-distro?
18:45:06 <bmwiedemann> asking again doesnt hurt to suggest there are more people interested in it ;-)
18:45:12 <vagrantc> or do we just assume we will eventually wear them down?
18:45:39 <vagrantc> indeed
18:45:40 <bmwiedemann> vagrantc: at least have a hint in notes
18:45:44 <lamby> agreed
18:46:02 <lamby> Hint what, exactly? That it should be patched upstream…?
18:46:26 <bmwiedemann> that currently there is a custom-date-override needed in debian
18:46:37 <vagrantc> if upstream refused the patch to support S_D_E, we should keep track of that, so we know previous attempts to upstream it?
18:46:42 <danielsh> might as well start a distropatches.org for sharing all kinds of distro patches, not just S_D_E :)
18:46:50 <bmwiedemann> true
18:47:05 <bmwiedemann> same for dead upstreams
18:47:07 <emaste_> interesting idea
18:47:41 <bmwiedemann> e.g. django rejected my backports to EOL branches, which prevents re-use by others.
18:47:45 <vagrantc> it's certainly been discussed before...
18:48:03 <vagrantc> it kind of comes down to who will set up the service?
18:48:43 <bmwiedemann> I guess, much of the time a github org / repo could already do the trick
18:48:50 <danielsh> +1
18:49:09 <danielsh> create a repository, decide on a mapping between short names and upstream project names, wait for pull requests
18:49:11 <StevenC99> or fork the existing upstream github project
18:49:24 <vagrantc> StevenC99: under a single organization?
18:49:30 <lamby> So "easy" :)
18:49:34 <danielsh> github.com/patches <- is it taken? :)
18:49:56 <bmwiedemann> github.com/distropatches is free
18:49:57 <emaste_> joined apr 27 2010
18:50:00 <StevenC99> vagrantc: if it's possible, I don't know
18:50:08 * vagrantc suspects so
18:50:28 <vagrantc> that could/would cover a lot of projects...
18:50:30 <lamby> Okay all, as chair I'm going to end the meeting here as we're off on a different (but interesting and v. useful concept!)… and I must leave.
18:50:37 <vagrantc> the danger is someone starts using it as upstream :)
18:50:41 <emaste_> AoB?
18:50:45 <lamby> Thanks all for being here o/
18:50:51 * vagrantc thanks lamby
18:51:14 <danielsh> lamby, One more thing before adjournment -
18:51:21 <lamby> go ahead
18:51:27 <danielsh> next meeting it would be great if agenda topics were added more time ahead of the meeting
18:51:31 <danielsh> FIN
18:51:32 <danielsh> :)
18:51:35 <vagrantc> yes.
18:52:01 <StevenC99> #idea github organisation to collate r-b patches against github-hosted upstreams
18:52:03 <bmwiedemann> that would need the URL published earlier ;-)
18:52:13 <danielsh> meetbot tracks all URLs
18:52:13 <MeetBot> danielsh: Error: "tracks" is not a valid command.
18:52:21 <danielsh> meetbot parses too leniently
18:52:21 <MeetBot> danielsh: Error: "parses" is not a valid command.
18:52:28 <lamby> Mm, yes, please add them earlier everyone. :)
18:52:41 * emaste_ guilty, will try next time :)
18:52:50 <lamby> #endmeeting