15:01:59 <karsten> #startmeeting metrics team meeting 15:01:59 <MeetBot> Meeting started Thu Feb 13 15:01:59 2020 UTC. The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:59 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic. 15:02:04 <gaba> :) 15:02:05 <gaba> thanks 15:02:28 <karsten> I added a few more topics to gaba's topics. 15:02:30 <karsten> do we have more? 15:03:34 <acute> hi everyone! 15:03:39 <karsten> hi acute! 15:04:06 <karsten> if not, shall we start? 15:04:17 <gaba> sure 15:04:23 <karsten> Review tasks from roadmap session (ticket creation, old cards in trello) 15:04:34 <karsten> what remains to be done? 15:04:37 <irl> i see this topic is so important we will do it twice 15:04:49 <karsten> just to be sure, yes. 15:04:54 <gaba> :) 15:05:03 <irl> i have created the exit scanner tickets, but did not set the keyword, i can do that after meeting 15:05:11 <karsten> I created tickets with the keyword. 15:05:12 <dennis_jackson> I have added a topic for Metrics w.r.t network health / metrics of our metrics 15:05:19 <gaba> i can st the keyword for them as I did not set the keyword for the others 15:05:21 <karsten> sounds good, dennis_jackson. 15:05:25 <gaba> I can do it for all of them 15:05:29 <gaba> irl ^ 15:05:47 <irl> gaba: ok, but only the ones on the roadmap 15:05:54 <irl> not the extra ambitious ones we are not doing 15:05:58 <gaba> right 15:06:14 <gaba> I will do the tasks in that list today 15:06:36 <gaba> irl: you created all tickets that says NeedsTicket in https://pad.riseup.net/p/0K2Q5uuN0CbjbOGxJQsL-keep 15:06:37 <irl> ok, i will review tomorrow and let you know if i think anything is still out of sync 15:06:39 <gaba> right ? 15:07:08 <gaba> sorry, that is karsten 15:07:25 <karsten> I created tickets for O3.1 and O3.2. 15:07:33 <irl> acute has been creating tickets for onionperf work 15:07:43 <gaba> ok 15:07:47 <irl> and updating the pad with ticket numbers 15:08:09 <acute> gaba: I have been creating tickets for O1/O2 15:08:15 <irl> but again, i suggested not to add roadmap keywords because i wasn't sure what the most helpful keyword would be 15:08:33 <gaba> ok. I will go through them 15:08:37 <irl> ideally it is something that helps gaba to keep track 15:08:52 <karsten> acute: did you create tickets for all lines saying NeedsTicket? 15:09:00 <irl> would it be a good idea for acute to ping you once all of the tickets are made gaba? 15:09:13 <gaba> I will create tickets for the objectives and keep track of all this proposal tickets in the wiki page so far: https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor59 15:09:19 <gaba> yes irl, acute. thanks 15:09:35 <acute> karsten: no, not yet - it would be good to decide where we create the OP tickets (trac or gitlab) 15:09:51 <irl> i think we abandon gitlab 15:09:53 <karsten> okay, that's also a topic for today, but we can talk about it now. 15:10:04 <gaba> acute: for now we will do everything in trac until transition is oficial 15:10:09 <gaba> we can talk later about it 15:10:10 <gaba> yes 15:10:57 <karsten> everything happy with this? 15:11:01 <karsten> everyone* 15:11:07 <gaba> \o/ happy 15:11:07 <acute> yes! 15:11:10 <irl> (: 15:11:19 <karsten> great! :) 15:11:33 <karsten> anything else on this topic? 15:11:48 <karsten> oh, old cards 15:12:00 <irl> we should review it again next week and make sure we did make all the tickets and that gaba is happy with the keyword distribution 15:12:21 <gaba> yes 15:12:33 <karsten> should we clean up the trello thing and remove everything we're _not_ going to do in Q1? 15:12:48 <irl> i didn't touch the trello yet apart from existing onionperf tickets 15:12:52 <gaba> karsten: yes. I will do it and we can review next week 15:12:59 <irl> ok 15:13:02 <karsten> okay. 15:13:12 <karsten> next topic? 15:13:18 <gaba> yes 15:13:23 <karsten> Check on tickets from network health 15:13:46 <karsten> those tickets look like placeholders to me. 15:13:56 <gaba> I want to see if there is any work that metrics needs to do there 15:14:09 <gaba> this is something geko will be working on 15:14:14 <irl> the problem is that network health is still a poorly defined team 15:14:30 <irl> and we need to have a clearer idea of the demarcation line between metrics and network health 15:14:33 <gaba> irl: what do you mean? 15:14:37 <karsten> it's unclear from these tickets whether there's going to be something for the metrics team to do. 15:14:44 <gaba> ok 15:14:49 <dennis_jackson> I am curious about the overlap / split as well. 15:14:59 <karsten> but should we still add more work to Q1 at this point? 15:15:11 <gaba> karsten: I do not think so 15:15:30 <karsten> let's watch these tickets and see what work they produce for Q2 or later. 15:15:40 <gaba> ok 15:16:21 <gaba> related to network health team work I think geko will coordinate but tehre is work from other teams there. like dgoulet from network team and ggus are involved in it. 15:17:32 <karsten> alright. moving on? 15:17:50 <gaba> yes 15:17:58 <karsten> Proposal 313 - https://lists.torproject.org/pipermail/tor-dev/2020-February/014158.html 15:18:26 <karsten> I replied with some thoughts. 15:18:51 <karsten> are there open questions to us? 15:19:07 <karsten> or what's the question? 15:19:29 <gaba> mostly to see if anything related to that needs to be discuussed here or we are fine 15:19:37 <karsten> I think we're fine. 15:19:41 <gaba> ok 15:20:13 <karsten> Metric queries for GoodBadISPs https://dip.torproject.org/torproject/web/community/issues/72#note_483 15:20:38 <karsten> looks like Relay Search questions. 15:20:46 <irl> i was in this meeting 15:20:54 <karsten> ah, indeed, it says irl. 15:21:06 <irl> no one has sent an email yet, so i assume they are doing other stuff 15:21:16 <irl> last update there was before the meeting, so i guess no update 15:21:42 <gaba> ok 15:22:13 <karsten> okay. moving on then? 15:22:13 <irl> that PR doesn't implement any of my suggestions 15:22:19 <karsten> ah, sorry. 15:22:36 <irl> we need better data, not just blocks of english text 15:22:55 <irl> otherwise we have to build natural language processing and sentiment analysis systems to know if an isp is good or bad 15:23:18 <irl> but ok, this is not on our roadmap and maybe they are working on this 15:23:32 <gaba> yep 15:23:36 <karsten> sounds good. 15:23:39 <irl> i'm ok moving on 15:23:54 <karsten> moving on. 15:23:55 <karsten> Monthly reports, are they worth the effort? 15:24:16 <dennis_jackson> Uh, there are monthly reports? On metrics? 15:24:27 <karsten> there have been, until mid-2019. 15:24:43 <karsten> they took a few hours to write. 15:24:48 <irl> i struggle to keep up with tor-project@ 15:24:52 <dennis_jackson> Ah okay, I was worried I was missing a mailing list. That's exactly when I became engaged with Tor 15:25:01 <irl> i don't think i've read a full week's worth of meeting reports in over a year 15:25:28 <karsten> same here, I'm not reading very many tor-project@ reports. 15:25:44 <irl> i guess we are the wrong people to know if they are useful or not 15:25:45 <karsten> so, my suggestion was: 15:25:47 <gaba> but I think other people are reading reports 15:25:59 <gaba> and they are available online as a way to understand what metrics is doing 15:26:06 <karsten> gaba and I were talking about changing the roadmap cycle from 3 months to 1 month. 15:26:31 <karsten> where we would not need weeks for creating a roadmap, obviously. 15:27:03 <karsten> that would be for the time after Q1 has ended. not affecting the current roadmap. 15:27:25 <irl> i think with only two people both over-capacity, 3 months is a long time to plan for 15:27:44 <karsten> now, when we have a roadmap for 1 month, it might be easier to write a quick summary for that month. 15:27:55 <irl> yeah that makes sense 15:28:13 <dennis_jackson> I just looked at an old report from July 2016. To me, it seems really nice to have. Even if its just tickets opened / closed / worked on (and what's coming next) It helps people find where the coal face is for the team. 15:28:51 <karsten> good to hear that they have been useful. 15:28:54 <gaba> yes 15:29:15 <gaba> and we can add stuff to the report the last meeting of the month 15:29:23 <gaba> not having to do it only one person 15:29:30 <irl> i used to do these but apparently haven't done them for a long time https://iain.learmonth.me/blog/2018/2018w33/ 15:29:54 <gaba> I can take the lead on setting up the space for the report each month 15:30:15 <irl> for thiings like the feedback process being able to remember what i did and when i did it, such reports would be useful too 15:30:52 <gaba> ok 15:31:07 <irl> interesting that my last weekly report was just before i did a week of travel, i guess i never caught up after i got back 15:31:10 <karsten> how about we write a 3-month summary for Q1 and then monthly reports starting in april? 15:31:16 <irl> ok 15:31:20 <gaba> ok 15:31:22 <gaba> sounds good 15:32:12 <karsten> alright. moving on? 15:32:20 <irl> ok 15:32:34 <karsten> GitLab migration and whether OnionPerf should stay or not 15:32:42 <karsten> already discussed, I think. 15:32:48 <irl> gitlab doesn't even work 15:32:57 <irl> you can't make merge requests 15:33:05 <irl> so i think there are no arguments for gitlab at this stage 15:33:19 <karsten> okay. 15:33:41 <karsten> I'm very much looking forward to a trac replacement, but I'm happy to wait. 15:34:04 <irl> right 15:34:08 <gaba> hiro is working on new server/configuration today 15:34:17 <karsten> thanks, hiro. :) 15:34:18 <gaba> and yes, I agree that is better to wait 15:34:29 <gaba> until we have something stable and migrated 15:34:32 <irl> do we need to rescue the issues back to trac quickly? 15:34:54 <irl> gitlab still has the onionperf issues 15:35:07 <irl> we are in progress of moving them back to trac 15:35:10 <gaba> irl: yes, I think we should use trac still 15:35:16 <gaba> otherwise it will be hard to migrate 15:35:19 <irl> i mean, is the gitlab data going to disappear? 15:35:25 <acute> ok, so remaining tickets that need to be created for OP should go in trac 15:35:26 <irl> like when hiro works on it today 15:35:30 <gaba> if we have data there we will need to move it... 15:35:41 <irl> do we have hours before we lose everything? 15:35:45 <irl> is it already gone? 15:35:51 <gaba> ha, no 15:36:19 <irl> acute: we might just reopen the closed issues for the ones that we migrated 15:36:26 <gaba> it will not be gon w without talking with people that may be using it 15:36:32 <irl> ok cool 15:36:48 <karsten> can you put the Trac component back, or do you need help with that? 15:36:51 <gaba> sorry about this. I should have more clear from the beginning not to use gitlab for trac replacement yet 15:37:06 <irl> i think i can do it 15:37:12 <irl> everyone is a trac admin these days 15:37:20 <karsten> makes sense, right? 15:37:31 <karsten> okay. 15:37:36 <karsten> next topic? 15:37:44 <irl> (component is back) 15:37:45 <irl> yes 15:37:54 <acute> thanks! 15:37:58 <karsten> Alternative to MaxMind's GeoLite2 database (#32978) 15:38:10 <karsten> I think I found something that we can use. 15:39:02 <karsten> I did a quick comparison of our current data to the new data source, and that was promising. 15:39:23 <karsten> I got feedback from a few relay operators, mostly positive. 15:39:39 <karsten> now, I'm not sure about legal questions and licenses and annoyances like that. 15:40:04 <karsten> I'll bring this up at the next vegas meeting to ask for legal stuff. 15:40:27 <gaba> ok 15:40:30 <irl> it seems to be similar enough, i guess we will get to know where it has its pros and cons 15:40:35 <karsten> if that doesn't bring up major concerns, I'll move this forward on the ticket. 15:41:03 <irl> do you have the database now? 15:41:09 <karsten> yes. 15:41:22 <irl> what is 44.190.21.1? 15:41:49 <karsten> 750654721, right? 15:42:05 <irl> huh? 15:42:11 <karsten> integer 15:42:23 <karsten> just hoping I didn't screw up the conversion. 15:42:38 <irl> i have never configured an ip address by integer 15:43:11 <karsten> US. 15:43:17 <irl> err no 15:43:29 <irl> wrong continent 15:43:31 <karsten> heh 15:43:44 <irl> but ok this is a strange one 15:44:15 <karsten> I can share details with you. I just don't want to announce this publicly until we have a rough idea whether we can use it at all. 15:44:28 <irl> ok, by email is fine and i will see if i can take a look 15:44:34 <karsten> great! 15:44:51 <karsten> hopefully I can move this all back to the ticket by the weekend. 15:45:06 <karsten> or early next week. 15:45:09 <karsten> okay, moving on? 15:45:16 <irl> ok cool 15:45:29 <karsten> Onionoo 8.0 release and deployment preparation 15:45:38 <karsten> we said we'd deploy 8.0 in 7 days. 15:45:55 <karsten> can you review the changes before that? 15:46:02 <karsten> they're small, IIRC. 15:46:11 <irl> i am working through reviews 15:46:16 <irl> i would say probably tes 15:46:18 <irl> yes 15:46:48 <karsten> I'm not sure if everything's under review yet. if not, I'll prepare it for early next week. 15:46:56 <karsten> cool! 15:47:15 <karsten> next and last topic? 15:47:40 <karsten> Metrics of Metrics / Network Health 15:47:44 <karsten> dennis_jackson: ^ 15:48:03 <dennis_jackson> This is in part brought on by the discussion in #33076 15:48:11 <dennis_jackson> And also the earlier Network Health team tickets 15:48:58 <dennis_jackson> I realise the roadmap is already pretty intense for the next 3 months - so this is more about ideas and future direction than anything else 15:49:43 <dennis_jackson> So - what kind of metrics do we check about the metrics we gather? 15:50:16 <irl> do you mean like to verify the metrics, or instrumentation of the software we run to generate them? 15:50:57 <dennis_jackson> Both really. For example, the uptime of the four onionperf servers is in the latter category 15:51:09 <dennis_jackson> And the number of successful measurements in the last 24 hours in the former category 15:51:34 <dennis_jackson> Are there targets (of any kind) for these things? Documents with the goals set out? 15:51:48 <irl> both of these are metadata you can derive from the onionperf data 15:51:58 <irl> i am a big fan of in-band metadata 15:52:17 <irl> we don't have anything that looks like an SLA or targets, other than doing the best we can 15:52:25 <dennis_jackson> Yes, absolutely, my point is more about a) is anything / anyone looking at this? b) Does Tor Metrics have any kind of targets for them? 15:52:35 <dennis_jackson> Ah okay 15:52:38 <karsten> we have several checks in collector to warn us if anything looks stale. 15:52:44 <irl> we are working on some operations documentation that starts to document what to do when things go wrong, and what we check 15:52:52 <dennis_jackson> Would something like this be helpful? 15:52:53 <karsten> but, best effort, not systematically. 15:53:01 <acute> sounds like a fun grafana dashboard 15:53:03 <dennis_jackson> I worry the Network Health team want to know what Metrics can offer 15:53:12 <dennis_jackson> And Metrics want to know what Network Health need 15:53:45 <dennis_jackson> Refining some kind of target / spec might be the way to help out both teams. 15:54:10 <dennis_jackson> a way* 15:54:52 <dennis_jackson> Is something that might be useful to move towards? In 3,6 or 12 months time? 15:55:20 <dennis_jackson> I can also see arguments that it might be a distraction, not turn into a useful output, be quickly out of date, etc. 15:55:29 <irl> i don't think this would be useful to metrics 15:55:40 <irl> but that doesn't mean we shouldn't do this, if it would be useful to someone else 15:55:46 <irl> but they have to make a case for that 15:56:13 <karsten> yep. 15:57:04 <karsten> does that make sense? 15:57:16 <karsten> we're approaching the 60 minutes. 15:57:20 <gaba> I think it would be useful to understand how useful the data is. It is also something that may be helpful in the new metrics portal 15:57:28 <dennis_jackson> Okay, I think so. 15:57:46 <dennis_jackson> I am still getting a feel for how Tor works / how sub units relate / manage their time. 15:58:11 <dennis_jackson> And I appreciate how hugely busy the metrics team is with a lot of asks from all over the org 15:58:39 <karsten> :) 15:58:59 <karsten> alright! let's end this meeting in time! 15:59:02 <karsten> #endmeeting