14:29:30 <karsten> #startmeeting metrics team
14:29:30 <MeetBot> Meeting started Tue Aug 29 14:29:30 2017 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:29:30 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:29:35 <karsten> https://storm.torproject.org/shared/Ou-1QRctynWbF4yedi-MfDsjImFMFSIEP20fbVGCPRa <- agenda pad
14:30:47 <karsten> okay, shall we start?
14:32:02 <karsten> iwakeh: ^
14:32:16 <iwakeh> yes :-)
14:32:21 <karsten> okay!
14:32:25 <karsten> * Proposal 280 Privcount discussion on September 12, 8pm EDT (= Sep 13, 00:00 UTC)
14:32:36 <karsten> we should contribute to that discussion somehow.
14:32:40 <iwakeh> sure,
14:32:42 <karsten> I'm aware that it's at 2am.
14:33:01 <karsten> I don't have a good plan for that part yet.
14:33:11 <iwakeh> maybe, we can hand in written questions?
14:33:16 <karsten> maybe it means we should at least contribute input beforehand.
14:33:19 <karsten> yes.
14:33:40 <karsten> depending on our input I might be able to make the meeting anyway.
14:33:50 <karsten> if our input is "yes, sure, sounds all fine" that wouldn't be necessary.
14:33:51 <iwakeh> have it on the reading list.
14:33:57 <karsten> good!
14:34:06 <nickm> (sorry about the timing; we needed to get a time when teor could make it too)
14:34:17 <karsten> figured that. :)
14:34:26 <iwakeh> all fine, we find some way to contribute.
14:34:52 <karsten> okay.
14:35:04 <karsten> next?
14:35:09 <iwakeh> yep.
14:35:18 <karsten> * Onionoo 4.1 release by Thursday?
14:35:24 <karsten> and maybe related:
14:35:27 <karsten> * Onionoo backend rotation
14:35:38 <karsten> so, the question is: which should we do first?
14:35:40 <iwakeh> first update then rotation?
14:35:46 <iwakeh> yes :-)
14:35:51 <iwakeh> that's the question.
14:35:57 <karsten> I don't feel strongly.
14:36:04 <karsten> update then rotation works for me.
14:36:16 <karsten> I can review #14201 later today.
14:36:18 <iwakeh> I don't know what constraints there are for the rotation task.
14:36:36 <karsten> just that we should be around afterwards to handle issues.
14:36:50 <karsten> which has been a constraint for me in the past half week or so.
14:36:55 <iwakeh> Maybe first update?
14:37:02 <karsten> sure.
14:37:19 <iwakeh> Then we know the update didn#t cause a problem.
14:37:21 <karsten> what other tickets need to go in?
14:37:29 <iwakeh> just the already
14:37:37 <iwakeh> closed ones and the merge ready.
14:37:45 <karsten> do you have a list?
14:37:50 <karsten> for the merge-ready ones?
14:37:57 * iwakeh lokking
14:38:02 <iwakeh> looking
14:38:57 <iwakeh> https://trac.torproject.org/projects/tor/query?status=accepted&status=assigned&status=closed&status=merge_ready&status=needs_information&status=needs_review&status=needs_revision&status=new&status=reopened&milestone=Onionoo-1.4.0&group=status&order=priority
14:39:32 <iwakeh> the review pending ticket is not that urgent.
14:39:35 <karsten> #22287 is really needs_review, right?
14:39:46 <karsten> (didn't you say it needs another review?)
14:40:04 <iwakeh> yep.
14:40:10 <karsten> which is fine.
14:40:24 <iwakeh> just a small testing
14:40:31 <iwakeh> round locally.
14:40:35 <karsten> okay.
14:40:41 <karsten> but things like #16513 can wait?
14:40:43 <iwakeh> the branch
14:40:49 <iwakeh> is already rebased.
14:40:52 <karsten> ok.
14:41:07 <iwakeh> that is new.
14:41:13 <karsten> ok.
14:41:21 <iwakeh> the deterministic out dir stuff.
14:41:28 <iwakeh> have that in 1.5.0
14:41:32 <karsten> so, #22287, #23297, #14201, #23339.
14:41:36 <iwakeh> right
14:42:02 <iwakeh> except 23339
14:42:18 <iwakeh> document new search filter, is easy.
14:42:30 <karsten> similar to #23297.
14:42:44 <karsten> those two can happen after the release. but should happen shortly after.
14:43:00 <iwakeh> true.
14:43:13 <iwakeh> around deployment.
14:43:38 <karsten> yep. I can deploy on omeiense when you say the pre-release tarball looks good.
14:43:59 <iwakeh> ok, should I wait with oo-h-03?
14:44:18 <iwakeh> well, it'll be before rotation.
14:44:35 <karsten> yes, deploying there sounds fine.
14:44:38 <karsten> once it's released.
14:44:42 <iwakeh> ok
14:44:58 <karsten> okay.
14:45:02 <karsten> next topic?
14:45:21 <iwakeh> shall I take the doc tickets for onionoo?
14:45:33 <iwakeh> 23339 mostly?
14:45:47 <karsten> sure.
14:46:09 <iwakeh> I mean, you don't have a documentation prepared anywhere?   ;-)
14:46:18 <karsten> well,
14:46:30 <karsten> I vaguely remember there was a snippet on a ticket somewhere. but I could be wrong.
14:46:45 <iwakeh> ok, I take a look.
14:46:53 <karsten> https://trac.torproject.org/projects/tor/ticket/21427#comment:1
14:47:04 <iwakeh> Just post on ticket, if you find anything.
14:47:15 <karsten> that's what I found.
14:47:28 <iwakeh> ok.
14:47:36 <karsten> (already linked from the ticket)
14:47:45 <iwakeh> yes, I did that :-)
14:47:47 <karsten> hehe
14:47:56 <karsten> alright, metrics-lib/metrics-base?
14:48:12 <iwakeh> ok
14:48:22 <karsten> you put something on the pad last thursday.
14:48:32 <iwakeh> on pad discussion?
14:48:57 <karsten> well, we can discuss here.
14:49:03 <iwakeh> just a summary and some remarks.
14:49:17 <karsten> so, you identified two areas there.
14:49:29 <karsten> one is that collector needs functionality in ml to produce sanitized descriptors.
14:49:34 <karsten> I think we agree there.
14:49:41 <iwakeh> fine
14:49:48 <karsten> the part that still needs discussion is what we do with index.json.
14:49:56 <iwakeh> well, and
14:50:13 <iwakeh> the functionality for compression/decompression.
14:50:19 <karsten> including that, yes.
14:50:36 <karsten> what about option c)?
14:50:43 <karsten> c): We move the org.torproject.descriptor.index package from metrics-lib to metrics-base where we call it org.torproject.metrics.index and re-use those classes in all code bases relying on metrics-base.
14:50:52 <karsten> you write:     (This might cause trouble: metrics-base is designed as basis for the build environment, which means that code doesn't depend on a certain commit in here.  Changing code would immediately force updates to all development environments b/c the build process checks out HEAD.)
14:50:58 <karsten> can we work around that somehow?
14:51:00 <iwakeh> trouble ahead ..
14:51:10 <iwakeh> not really, because
14:51:12 <karsten> like, can we disable automatic fetching of updates?
14:51:22 <iwakeh> we don't want to start releasing metrics-base.
14:51:28 <karsten> agreed!
14:51:41 <iwakeh> not fetching every time
14:51:54 <iwakeh> amounts to internal releases of certain commits and
14:52:03 <iwakeh> in addition the purpose
14:52:13 <iwakeh> of metrics-base would be redefined.
14:52:19 <karsten> extended, yes.
14:52:20 <karsten> so,
14:52:29 <karsten> we do have commits like this:
14:52:30 <iwakeh> b
14:52:33 <karsten> Update to latest metrics-base.
14:52:46 <iwakeh> unnecessary.
14:52:53 <karsten> oh, why?
14:53:09 <karsten> I'm updating every time I notice differences between local and remote.
14:53:11 <iwakeh> we fetch the latest.
14:53:36 <karsten> if there are updates, that shows up as difference.
14:53:50 <karsten> and when I do `git commit -va`, they'll be merged automatically.
14:54:04 <karsten> so I'm doing that as separate commit.
14:54:10 <iwakeh> hmm
14:54:29 <iwakeh> It is intended to
14:54:46 <iwakeh> fetch the latest commit (if it doesn't I need to check)
14:54:53 <karsten> yes, it does.
14:55:02 <karsten> and it updates the local submodule.
14:55:03 <iwakeh> thus, the development happens with HEAD of metrics-base.
14:55:08 <karsten> yep.
14:55:16 <karsten> but then `git status` shows a change.
14:55:16 <iwakeh> and, not a special commit.
14:55:24 <iwakeh> no problem.
14:55:30 <karsten> this is not a problem.
14:55:49 <karsten> but it's a change that will be included in the next commit when running `git commit -a`.
14:55:59 <karsten> so, going back one step:
14:56:20 <karsten> how bad would it be to include code in this repository and pull that in automatically?
14:56:36 <karsten> we'd still have releases for dependent repositories.
14:56:40 <iwakeh> very bad, b/c indirect release.
14:56:53 <iwakeh> We'd have releases
14:56:54 <karsten> this only affects our own code bases.
14:57:14 <iwakeh> that depend on a moving/volatile/unreleased repository.
14:57:29 <karsten> we'd notice changes there at least.
14:57:32 <iwakeh> and with the new scope that could break existing things.
14:57:40 <karsten> because `git status` says that there are changes.
14:58:15 <karsten> hmm, okay, how do we move forward there.
14:58:22 <iwakeh> hmm,
14:58:34 <iwakeh> could we try b) ?
14:59:08 <karsten> it feels dirty.
14:59:19 <iwakeh> This depends on released versions and is only internal.
14:59:22 <karsten> because it extends metrics-lib beyond parsing/handling descriptors.
14:59:45 <karsten> that's true.
14:59:51 <iwakeh> well, the code needs to live somewhere.
15:00:03 <karsten> but, what if we need to put out a release for a purely internal patch?
15:00:14 <karsten> what do we write in the change log/announcement?
15:00:16 <iwakeh> calling it unofficial prevents other uses.
15:00:24 <karsten> nothing has changed that you should worry about. ;)
15:00:37 <iwakeh> well, the
15:00:59 <iwakeh> code patched in unofficial has an effect somewhere, even in m-lib.
15:01:19 <karsten> not necessarily.
15:01:25 <karsten> not necessarily in m-lib, that is.
15:01:38 <iwakeh> yes, but we'd have
15:01:51 <karsten> and, let's imagine we add more code there that is used by multiple metrics code bases.
15:01:54 <iwakeh> a release that repairs file-io for example.
15:02:03 <iwakeh> at some point
15:02:33 <iwakeh> we could decide that there is enough for an (unofficial) internal or official new product.
15:02:50 <karsten> ok.
15:02:56 <karsten> it's a compromise.
15:03:03 <iwakeh> yes,
15:03:03 <karsten> but we can do b).
15:03:15 <iwakeh> but better than multiplying the same code.
15:03:33 <karsten> maybe. :)
15:03:49 <karsten> okay.
15:04:01 <iwakeh> that was the reason once upon a time to create metrics-base.
15:04:40 <iwakeh> i.e., to remove multiplying code.
15:04:58 <karsten> yes.
15:05:02 <karsten> and that was helpful.
15:05:10 <karsten> there are still more steps towards that goal.
15:05:15 <iwakeh> true
15:05:18 <karsten> including cleaning up metrics-web.
15:05:31 <iwakeh> definitely.
15:05:32 <karsten> but, one thing at a time.
15:05:46 <iwakeh> step by step :-)
15:05:46 <karsten> okay, now that this topic is unblocked I think we can move forward.
15:05:51 <karsten> yes!
15:05:59 <iwakeh> I think there
15:06:05 <karsten> #16596 is on your list, that's good!
15:06:07 <iwakeh> are many pending reviews?
15:06:18 <karsten> okay, let's go through them.
15:06:23 <iwakeh> yes, 16596
15:07:04 <karsten> ah, the collector ones..
15:07:14 <karsten> those are tricky.
15:07:32 <iwakeh> yes a little
15:07:46 <iwakeh> maybe just
15:08:15 <iwakeh> look at the metrics-lib webstats first
15:08:24 <karsten> ah, right, I started that 1 hour ago.
15:08:29 <karsten> didn't finish before the meeting though.
15:08:35 <karsten> that one is rather simple.
15:08:53 <iwakeh> after that the collector ones need to be adapted or consolidated.
15:09:06 <karsten> I really wonder what to do with collector sync.
15:09:09 <iwakeh> I think, now
15:09:21 <karsten> it's incredibly useful for relaydescs.
15:09:32 <iwakeh> with the decision/compromise these all we work out fine.
15:09:33 <karsten> but the need to do it for other modules is rather limited.
15:10:09 <karsten> and with our recent decision to focus on our own collector instances has shifted focus a bit, too.
15:10:18 <iwakeh> right.
15:10:23 <karsten> because it affects how much we can trust other collectors.
15:11:06 <karsten> I'll try to review them, but I'll need to think about that topic a bit more. if you have ideas, please share them, too.
15:11:18 <iwakeh> fine.
15:11:34 <iwakeh> and, this is important with the focus shift
15:11:45 <iwakeh> to only have Tor-hosted instances.
15:12:01 <iwakeh> or only use such instances.
15:12:37 <karsten> okay! are there other things where we're blocking on each other?
15:12:55 <iwakeh> all cleared.
15:13:04 <karsten> great! you're afk later this week?
15:13:34 <iwakeh> just a little less
15:13:57 <iwakeh> online connectivity.
15:14:02 <karsten> ah, okay.
15:14:14 <iwakeh> but plenty time for reading and coding.
15:14:19 <karsten> heh
15:14:27 <karsten> okay, and we sync again next thursday?
15:14:43 <iwakeh> for the release, yes.
15:14:50 <karsten> ah.
15:14:57 <iwakeh> this thrsday.
15:15:08 <karsten> I meant the next meeting.
15:15:15 <iwakeh> sure.
15:15:21 <karsten> the pre-release and release can happen via email as usual, right?
15:15:30 <iwakeh> yes, that'll be fine.
15:16:00 <karsten> alright. thanks, and let's talk more via email and then next week. bye, bye! :)
15:16:18 <iwakeh> thanks & bye, bye!
15:16:23 <karsten> #endmeeting