14:29:34 <karsten> #startmeeting metrics team
14:29:34 <MeetBot> Meeting started Thu Oct 26 14:29:34 2017 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:29:34 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:29:37 <karsten> hello!
14:30:02 <irl> hello
14:30:09 <karsten> hi irl!
14:30:31 <karsten> iwakeh: ping
14:31:51 <karsten> irl: we'll need iwakeh for the first topic. should we start with the atlas/onionoo topic?
14:31:58 <irl> we can do
14:32:09 <karsten> * plan for implementing aggregations in Atlas (#23517, #23509)
14:32:12 <iwakeh> Hi there!
14:32:19 <karsten> hi! :)
14:32:26 <irl> hi iwakeh (:
14:32:33 <iwakeh> :-)
14:32:47 <karsten> so, my plan there was to keep using compass with a JSON interface,
14:33:00 <karsten> and implement this in onionoo later.
14:33:10 <karsten> I'm not sure if that's a good plan, though.
14:33:15 <irl> my thoughts are that onionoo already has all the data required to do compass things
14:33:31 <karsten> true.
14:33:35 <iwakeh> what speaks against?
14:33:41 <irl> it doesn't mean adding extra data, just a new document type with some aggregations in it
14:34:03 <irl> i'm not exactly certain what that document would look like yet
14:34:04 <karsten> it's just a very different kind of operation that onionoo would do.
14:34:17 <karsten> the aggregation part.
14:34:24 <irl> it is a lot more logic than i imagine is in existing document generation
14:34:31 <irl> but it's mostly just adding numbers together
14:34:38 <irl> and filtering
14:34:51 <irl> java streams could be great at this
14:34:57 <irl> maybe
14:34:58 <iwakeh> What timeframe do we have?
14:35:23 <irl> it's an M in the short-term list
14:35:23 <iwakeh> I assume, the step to let compass provide JSON is small.
14:35:25 <karsten> well, we said Q4/2017, but we can easily say Q3/2018.
14:35:37 <karsten> iwakeh: yep.
14:35:51 <iwakeh> So, let's do this interim
14:36:08 <iwakeh> and also decide what to do next.  I also
14:36:32 <iwakeh> think that another JSON document from Onionoo seems ok.
14:37:00 <karsten> it's a question of how much effort we want to put into this now.
14:37:11 <karsten> maybe this is easier after switching to a datbase.
14:37:13 <karsten> database*
14:37:16 <iwakeh> From 10000 feet: Onionoo is a JSON service.
14:37:22 <irl> not doing it in onionoo does mean that we can't shut down compass
14:37:40 <karsten> irl: true.
14:37:40 <iwakeh> We're first reducing compass at least.
14:37:41 <irl> and it would mean we can't advertise this to other users, as we do onionoo
14:37:48 <irl> because we're only going to break it anyway
14:38:03 <karsten> okay.
14:38:14 <karsten> maybe we can't decide this today.
14:38:18 <iwakeh> We'd need to define the JSON doc?
14:38:27 <irl> yep
14:38:52 <iwakeh> That would stay the same being transferred to onionoo.
14:39:01 <karsten> right.
14:39:13 <irl> if we call the compass implementation a prototype, then i'm happier
14:39:14 <iwakeh> so, we wouldn't break, just redirect.
14:39:21 <iwakeh> fine.
14:39:38 <iwakeh> it would be actually good, to find out the right JSON format.
14:39:51 <karsten> maybe we should start with the doc.
14:39:57 <karsten> think about what we need.
14:39:57 <iwakeh> right.
14:40:08 <iwakeh> How much code is already existing?
14:40:12 <karsten> and then either do it by changing the existing compass a bit or by implementing it in onionoo.
14:40:24 <karsten> how much compass code?
14:40:31 <irl> the aggregation code already exists in python
14:40:32 <iwakeh> yes
14:40:46 <irl> but we need to swap out the html template for a jsonify
14:40:53 <iwakeh> let's start with the doc and look further then.
14:40:57 <irl> ok
14:41:00 <karsten> ok.
14:41:32 <karsten> alright, let's talk about the most important topic for today.
14:41:48 <karsten> most important, because we need to finish it today.
14:41:51 <karsten> it == roadmap
14:41:57 <karsten> * roadmap (karsten)
14:42:04 <iwakeh> I read the mail thread and postponed my reply to here.
14:42:12 <karsten> after this meeting we need to share our roadmap with the other teams.
14:42:26 <karsten> let's discuss all open points now.
14:42:27 <iwakeh> which is, I'm fine with the recent suggestions.
14:42:31 <karsten> cool!
14:42:48 <karsten> I also added (XY team might care)
14:43:06 <karsten> so that the other teams can look out for whatever parts they should be reading.
14:43:06 <iwakeh> XY?
14:43:18 <karsten> XY = network, applications, ooni, etc.
14:43:26 <karsten> Switch ExoneraTor and metrics-web from Tomcat to embedded Jetty, and harmonize directory structures (M) (admin team might care)
14:43:58 <karsten> how about we go through the open suggestions on the doc?
14:44:03 <karsten> and change/approve them.
14:44:21 <iwakeh> ok.
14:44:35 <karsten> Make Compass part of Atlas in order to provide aggregated search results and details pages on Atlas and to finally shut down Compass (M)
14:44:37 <irl> oh hang on "make a plan" turned into "do a thing"
14:44:41 <karsten> yes.
14:44:47 <karsten> (UX team might care)
14:44:59 <karsten> (that was part of the paste)
14:45:17 <irl> so i can't really start on compass until atlas is merged into the website
14:45:28 <irl> because otherwise i'm maintaining two atlases
14:45:32 <karsten> note that we don't have to complete this by end of 2017.
14:45:39 <irl> ah ok
14:45:43 <karsten> The following list contains all goals that we‘d like to start or even complete within the remaining months of 2017.
14:45:44 * irl got scared
14:45:56 <irl> yes, starting it is ok
14:46:08 <karsten> there's no way how we can finish 12 M tasks in 2017 and another 15 M tasks in Q1-Q3/2018.
14:46:21 <iwakeh> true
14:46:33 <iwakeh> maybe underline 'start' ;-)
14:46:36 <karsten> hehe
14:46:59 <karsten> let's just point at it if somebody is disappointed later. ;)
14:47:06 <irl> (:
14:47:16 <karsten> so, is that one good to go in?
14:47:20 <iwakeh> yep, smallprint.
14:47:24 <karsten> heh
14:47:31 <irl> yes, that one is good
14:47:47 <iwakeh> fine.
14:48:03 <karsten> the next one is a format suggestion:
14:48:11 <karsten> do we really need the ongoing and on-demand goals category?
14:48:24 <karsten> it made more sense when there were more goals under that.
14:48:47 <karsten> but I could imagine just merging those goals with medium-term goals.
14:48:50 <karsten> what do you think?
14:48:53 <irl> i think it's good to keep some text that says that there are ongoing and on-demand tasks
14:48:59 <irl> but it doesn't need to be a seperate section
14:49:07 <iwakeh> right.
14:49:22 <karsten> want to edit the text under medium-term goals?
14:49:27 <karsten> edit/extend?
14:49:53 <karsten> or should we just copy over everything?
14:49:57 <iwakeh> or, simply remove the additional headline?
14:50:19 <iwakeh> keep: 'We realize ..."
14:50:20 <karsten> yes, but then we should also put the goals under the categories.
14:50:26 <karsten> above.
14:50:39 <irl> made a suggestion
14:50:52 <irl> with that, drop the headline and move the below tasks above
14:51:26 <iwakeh> yes.
14:51:34 <karsten> how about that?
14:53:33 <iwakeh> looks good to me.
14:53:49 <irl> me too
14:54:11 <irl> ah, no, those goals are 12M and we reserve 3M
14:54:24 <irl> that works too
14:54:39 <karsten> ok.
14:55:45 <karsten> looks a bit easier to understand.
14:55:52 <irl> indeed
14:55:53 <karsten> next one is onionoo and votes.
14:56:31 <iwakeh> as agreed before.
14:56:54 <karsten> and we have two more that I think we agreed on before.
14:57:03 <karsten> in the long-term list, so what could go wrong.
14:57:29 <iwakeh> yes.
14:57:33 <irl> looks good
14:57:39 <karsten> okay!
14:57:57 <karsten> I think that's it.
14:58:05 <iwakeh> yep!
14:58:10 <karsten> we should share it with the other teams.
14:58:25 <karsten> which will share their roadmaps with us.
14:58:27 <iwakeh> don't forget to remove the notes below.
14:58:35 <iwakeh> other than that fine.
14:58:47 <iwakeh> ,
14:58:57 <karsten> yes, will do. that is, print only pages 1--8 to a pdf.
14:59:17 <karsten> next step after that: we might receive feedback.
14:59:20 <karsten> from other teams.
14:59:29 <karsten> either things they need us to do or do differently,
14:59:47 <karsten> or from other teams' roadmaps saying what else we need to be involved in.
15:00:36 <iwakeh> maybe, just remove the next steps?
15:00:43 <karsten> yes, that's what I just thought, too.
15:00:55 <karsten> it's isabela's process that we'll follow.
15:00:56 <iwakeh> they're not actually part of the roadmap.
15:01:03 <karsten> not next steps we need to keep in mind ourselves.
15:01:19 <karsten> the roadmap will still be in a draft state for another few weeks.
15:01:28 <karsten> but not because we need more time to think about goals.
15:01:29 <iwakeh> good!
15:01:47 <karsten> so, I'll remove next steps?
15:02:00 <irl> we did all the steps
15:02:04 <irl> they're previous steps now
15:02:11 <karsten> yep!
15:02:14 <iwakeh> true
15:02:21 <karsten> okay, removing.
15:02:59 <karsten> great! that's this agenda topic then.
15:03:06 <karsten> moving to the next?
15:03:11 <iwakeh> +1
15:03:15 <karsten> * robustness for collector and onionoo
15:03:35 <karsten> - Extend Onionoo's nagios script to all known frontends (#23984)
15:03:38 <karsten> - Are there other things we should monitor?
15:03:42 <karsten> there are.
15:03:45 <iwakeh> who will work on the nagios script?
15:03:49 <karsten> fine question.
15:04:04 <irl> the nagios script looks simple enough
15:04:10 <karsten> where the first step might be to make a list of all the things we want to be notified about.
15:04:17 <karsten> ah, the onionoo one.
15:04:18 <iwakeh> yes
15:04:49 <karsten> checking the other onionoo frontends is just one step towards making that check even better.
15:04:53 <karsten> it has been very useful in the past.
15:05:05 <karsten> but there are 20 other things we should be checking automatically.
15:05:21 <karsten> we even have a roadmap item for this: Deploy better notification system for operational issues (M) (admin team might care)
15:05:48 <irl> i was thinking we probably have enough systems to monitor it would justify us having our own monitoring system
15:06:02 <karsten> should we hold a pad meeting to brainstorm possible things to monitor?
15:06:07 <iwakeh> yes.
15:06:10 <irl> ok
15:06:19 <karsten> irl: what's the advantage over using tor's nagios?
15:06:32 <irl> karsten: i can't add checks to tor's nagios
15:06:52 <iwakeh> there are parts only the tor system has access to, like filesystem etc.
15:07:05 <karsten> I can't do that either, but we can ask the admins to do it for us.
15:07:51 <irl> ok, let's make a list first and see what it is we need to monitor
15:07:55 <karsten> sounds good.
15:07:59 <iwakeh> fine.
15:08:05 <karsten> how about mid next week?
15:08:21 <iwakeh> fine for me.
15:08:24 <karsten> maybe before the team meeting?
15:08:49 <karsten> or wednesday to make it a separate thing?
15:08:55 <irl> wednesday is good for me
15:09:01 <irl> thursday is my meetings day, i have no extra room
15:09:08 <karsten> ah! :)
15:09:12 <iwakeh> I don't have a preference, either is fine.
15:09:20 <karsten> let's do it wednesday then.
15:09:26 <karsten> same time as this meeting?
15:09:34 <karsten> (daylight ends this weekend)
15:09:46 <karsten> (or saving of)
15:09:48 <iwakeh> finally back to normal ;-)
15:10:04 <irl> sounds good
15:10:13 <irl> this meeting is defined in terms of UTC?
15:10:22 <iwakeh> 14:30UTC
15:10:26 <karsten> yep.
15:10:39 <irl> cool
15:11:03 <karsten> great!
15:11:16 <karsten> next topic?
15:11:25 <iwakeh> yes.
15:11:30 <karsten> * onionoo release
15:11:39 <karsten> that would be the hotfix release.
15:11:44 <iwakeh> still testing.
15:12:02 <iwakeh> #24012
15:12:04 <karsten> okay.
15:12:22 <karsten> I can create a pre-release tarball this evening.
15:12:28 <iwakeh> trying to find the reason for illegalstateexc.
15:12:32 <karsten> !
15:12:49 <karsten> if you can't find it anymore, that's fine.
15:12:52 <iwakeh> when would you need the ok for the branch or patch?
15:12:57 <karsten> we can always put out another patch release if needed.
15:13:05 <iwakeh> true.
15:13:25 <karsten> 3 hours from now?
15:13:25 <iwakeh> deployment, when?
15:13:43 <karsten> tomorrow at 10:00 cest?
15:14:07 <iwakeh> ok for both.
15:14:31 <karsten> great!
15:14:37 <karsten> there's also a bug in metrics-web.
15:14:45 <karsten> preventing it from updating.
15:15:00 <iwakeh> did you find the reason?
15:15:30 <karsten> I have a patch. I just don't know why the test does not finish.
15:15:53 <karsten> so, might take longer than today to fix it.
15:16:13 <iwakeh> ok.
15:16:27 <karsten> ah, speaking of onionoo,
15:16:35 <karsten> how's your review queue these days?
15:16:53 <karsten> there are a couple of relatively small onionoo patches in needs_review.
15:16:55 <iwakeh> depends
15:17:01 <karsten> ok.
15:17:03 <iwakeh> ah, ok.
15:17:06 <karsten> not urgent.
15:17:40 <iwakeh> I take the onionoo reviews on first then.
15:17:50 <karsten> cool!
15:18:06 <karsten> I think we're done.
15:18:19 <karsten> unless there's more to discuss?
15:18:26 <iwakeh> I'm fine.
15:18:29 <irl> i think that's all
15:18:37 <karsten> great. thank you!
15:18:50 <karsten> bye, bye!
15:18:52 <iwakeh> Thanks! And, bye, bye!
15:19:50 <karsten> #endmeeting