15:04:22 <karsten> #startmeeting metrics team
15:04:22 <MeetBot> Meeting started Thu Aug  8 15:04:22 2019 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:04:22 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:04:41 <karsten> https://storm.torproject.org/shared/5h1Goax5eNusxjXJ_Ty5Wl7hFR1uqCReUiN8xdlBG8T <- agenda pad
15:04:54 <karsten> please add topics that you want to discuss today.
15:06:04 <karsten> let's start with:
15:06:05 <irl> i don't have any extra topics
15:06:14 <karsten> Data portal. Do you all have anything to comment about the proposal?
15:06:16 <karsten> irl: okay.
15:06:47 <karsten> gaba: no further comments from me on that. it looks good to me, as far as I can tell.
15:06:51 <gaba> ok
15:06:58 <irl> my plan is to take one last look this evening, and also invite those that will be at ccc camp to come and visit the scottish consulate and talk about the proposal
15:07:13 <irl> i believe at least one of the people will be around
15:07:16 <gaba> irl: they want to submit it soon
15:07:25 <gaba> the proposal to OTF
15:07:37 <irl> it should not block on me if they have a deadline
15:07:58 <gaba> ok. I will let them know thtey can submit it today or tomorrow then
15:08:03 <irl> cool
15:08:17 <karsten> great!
15:08:27 <karsten> moving on?
15:08:29 <irl> ok
15:08:42 <karsten> Obtaining more context on several roadmap items (karsten)
15:09:02 <irl> might be helpful if mikeperry is around?
15:09:03 <karsten> I listed three roadmap items on the pad that are assigned to me and that I'd have to learn more about before doing something useful.
15:09:21 <karsten> yes, probably.
15:09:44 <karsten> okay, I can ask him at the vegas team meeting today.
15:09:46 <gaba> those are issues that are coming to the metrics table from the scalability roadmap exercise
15:09:48 <irl> the first i believe looks at historical consensus weights and onionperf data to determine if there is correlation
15:10:08 <irl> the second is a pretty high level task, and needs to be broken down
15:10:24 <irl> the third also looks quite high level and needs some specific questions to answer
15:10:48 <karsten> yes, I have a vague idea what these are all about, but that's not enough to start them.
15:10:58 <irl> i think the third one is looking at the live data
15:11:07 <irl> we have some authorities using torflow, and some sbws
15:11:46 <gaba> mikeperry is going to create tickets for all of this
15:11:53 <karsten> ah!
15:12:04 <karsten> good to know.
15:12:16 <irl> excellent
15:13:20 <karsten> that answers my question. not to the extent that I could add something to this week's sprint, but still.
15:13:31 <karsten> which means we can move on to the topic:
15:13:36 <karsten> Roadmap - how are we doing?
15:14:02 <gaba> yes, some other issues need tickets
15:14:08 <gaba> I added a scalability label
15:14:11 <irl> the cloudformation/ansible bits that were blocking other stuff are done, i already moved this to done
15:14:25 <karsten> yay!
15:14:33 <gaba> Right now we are writing a proposal to get funding for this work and much of it will be metrics. I will loop you both in when we have something with Al.
15:14:53 <irl> i've added onionperf ci tasks to in progress, but those are not yet started, will be looking with acute at that and onionperf deployment next
15:14:58 <gaba> :)
15:14:59 <irl> we are also going to organise onionperf tickets in gitlab
15:14:59 <karsten> gaba: so, should I wait for that, rather than prod mike to get more details now?
15:15:12 <gaba> karsten: no, let's still move forward
15:15:16 <karsten> ok.
15:15:23 <gaba> irl: nice
15:15:24 <irl> acute now also has access to AWS to use cloudformation to stand up onionperf dev environments in a disposible way to accelerate the iterations on this
15:15:27 <karsten> regarding roadmap, the ant ivy stuff is done.
15:15:35 <gaba> yeah!
15:15:42 <irl> yeah, ivy works pretty well
15:15:49 <irl> that was easier than expected
15:15:50 <karsten> good to hear!
15:16:21 <karsten> true. the gradle/maven switch will be more painful.
15:16:36 <irl> yeah
15:16:41 <karsten> so, regarding CI, is there anything we can do with metrics-lib or other metrics codebases?
15:16:52 <karsten> anything else we can add to gitlab or make sure it's working as expected?
15:17:06 <irl> yes, we should add tickets to run tests and checks on gitlab ci for all codebases
15:17:17 <gaba> wrt gitlab we are testing it more with roadmaps. I'm not doing anything for metrics because we agreed on using trello and is fine. If we end up having gitlab replacing trac then it will be a good idea to move everything there.
15:17:39 <irl> we're only having onionperf issues in gitlab for now
15:17:43 <karsten> gaba: my main interest is in gitlab's CI.
15:17:46 <irl> yeah
15:17:49 <gaba> ok
15:17:51 <irl> the CI is working nicely
15:18:16 <karsten> is it possible to have non-master branches in gitlab's CI?
15:18:21 <karsten> or even user repositories?
15:18:22 <irl> yes
15:18:31 <irl> i need to read more of the docs though
15:18:38 <irl> we might need one runner per user
15:18:53 <irl> but currently if it's in the metrics group's repo, it will run on all branches
15:19:22 <karsten> I think having some documentation of what's possible would be nice.
15:19:28 <karsten> which could start in a ticket.
15:19:36 <irl> sounds good to me
15:19:49 <irl> first goal though: run tests and checks on master for all codebases
15:19:57 <karsten> sounds great!
15:20:13 <karsten> should I create a ticket (with child tickets), or would you rather want to do that?
15:20:22 <irl> if you have the time, that would be awesome
15:20:29 <karsten> I can create tickets.
15:20:32 <irl> thanks!
15:21:25 <karsten> what else should I do this week?
15:22:00 <irl> we might need to ask what we need to do for #30777
15:22:03 <gaba> anything else from teh backlog that you can do?
15:22:06 <irl> (the answer might be nothing)
15:22:34 <gaba> irl: we can add that question to the anti-censorship meeting that is happening in an hour and a half
15:22:43 <karsten> oh, there's a response on #30777 that I somehow missed...
15:22:46 <irl> gaba: sounds good, there is an item in the backlog
15:23:49 <gaba> ok
15:23:59 * gaba added issue to anti-censorship agenda
15:24:02 <karsten> I wonder if #29461 is actionable.
15:24:19 <irl> i believe the stats are still just written to the log file for now
15:24:40 <karsten> oh, okay.
15:25:05 <karsten> gaba: can you maybe add that to the anti-censorship agenda, too?
15:25:16 <gaba> karsten: what is the specific question?
15:25:24 <karsten> I could start working on the collector part there.
15:25:34 <karsten> but the snowflake side needs to be ready.
15:25:37 <irl> we have the spec of the file
15:25:40 <gaba> Yes. I think #29461 is on metrics plate right now.
15:25:43 <karsten> oh!
15:25:47 <gaba> ok. Will check on that still.
15:25:50 <karsten> if there's a spec I can start doing something.
15:25:54 <gaba> ok
15:26:07 <gaba> Let's totally confirm that we are ready then.
15:26:08 <irl> add to metrics-lib and then collector
15:26:13 <karsten> yes!
15:26:13 * gaba adding it to anti-censorship
15:26:25 <cohosh> (just jumping in to say that everything is ready on the snowflake side)
15:26:37 <karsten> yay!
15:26:38 <cohosh> we are currently collecting the metrics specified in the ticket and saving it to a file
15:26:57 <cohosh> so whenever the collector is ready we can set up the http server part
15:26:58 <irl> cohosh: can we have it on the broker at "/metrics" ?
15:27:03 <gaba> thanks cohosh!
15:27:06 <cohosh> irl: absolutely!
15:27:09 <irl> awesome
15:27:12 <karsten> perfect!
15:27:16 <irl> and this is written every 24 hours?
15:27:22 <karsten> having some real data will make the coding even easier.
15:27:23 <cohosh> i will make a ticket for writing the handler fo rthat
15:27:28 <cohosh> yep, every 24 hours
15:27:31 <irl> awesome thanks!
15:27:49 <cohosh> i can attack a snippet of the log file if that's easier for you to the ticket
15:28:00 <cohosh> or just set up the handler asap so you can fetch it yourself
15:28:16 <karsten> no rush, whatever is easier for you.
15:28:30 <cohosh> is there docs for how the http response should look?
15:28:45 <irl> Content-Type: text/plain; charset=utf-8
15:29:21 <cohosh> cool :)
15:29:37 <cohosh> i am super excited about this
15:29:48 <gaba> hehe :)
15:30:02 <karsten> yes, this sounds great. thanks!
15:30:25 <karsten> okay, I have enough on my plate for this week. :)
15:30:37 <irl> woo
15:31:07 <karsten> anything else about the roadmap topic?
15:31:29 <irl> no more from me
15:31:58 <karsten> okay. moving on to:
15:31:59 <gaba> on more thing. I see the debian metapackages in the icebox
15:32:02 <karsten> oh.
15:32:18 <gaba> irl: does that mean that we are still going to do it and we do not have an ETA or you are dropping it?
15:32:24 <irl> these were a lot more complex than initially considered
15:32:44 <gaba> yes, you think we should close the ticket?
15:32:46 <irl> we are not dropping them because i actually would like to use them for my own relays
15:32:51 <gaba> ok
15:32:54 <irl> i still think they are worthwhile
15:33:16 <irl> but they could actually be a whole gsoc project or something
15:34:06 <gaba> good. pil: ideas for gsoc :)
15:34:09 <gaba> pili ^
15:34:20 <gaba> nothing else from me about roadmap
15:34:27 <karsten> how does this work?
15:34:33 <pili> ack :)
15:34:34 <karsten> do we keep it around on the roadmap anyway?
15:34:42 <irl> i put it at the bottom of the icebox
15:34:50 <karsten> does the order matter?
15:35:01 <gaba> yes, it should be sort out by priority
15:35:16 <irl> at the bottom is does not clutter the view
15:35:24 <karsten> alright.
15:35:33 <irl> if we finish the roadmap early then i'll still do it in this roadmap
15:35:47 <karsten> okay, sounds good!
15:35:58 <karsten> moving on?
15:36:01 <irl> ok
15:36:09 <karsten> Browser Performance Tests
15:36:12 <karsten> djackson: ^
15:36:17 <djackson> Okay so
15:36:25 <djackson> Nearly ready to push the deploy button
15:36:36 <djackson> Have built the VM instances for Google Cloud
15:36:49 <djackson> Can now push button deploy test instances and record network/UX metrics from various locations. Supports dynamic scripts e.g. visiting a website, clicking links, interacting with forms. Final TODO is finishing the orchestration script which spins up instances / runs jobs / kills old instances and so on. ETA Friday.
15:37:04 <djackson> So things are coming along nicely and hoping to have the dataset by the end of next week
15:37:35 <gaba> nice!
15:37:38 <djackson> The study plan is still being finalised, but we hope to quantify the variance in peformance across location, guard selection, browser version/settings and a few other things
15:37:43 <irl> how many new instances would be spun up in an hour?
15:37:48 <djackson> Coming onto that
15:38:14 <djackson> This will give us a good understanding of where Tor is now for the average user, as well as a test bed for future changes to the browser / client.
15:38:37 <djackson> I read the meeting notes from last week and had considered the possible impacts. Back of the Envelope calculations suggest the impact should be negligible. Don't anticpate more than 1000 Tor Browser instances at any one time and total length of testing <3 days.
15:38:47 <djackson> Each browser instance is loading a (relatively light) webpage (only one at a time) and is only navigating to the next page once one page has finished entirely.
15:39:05 <irl> my concern is not so much the web traffic but the directory traffic
15:39:10 <djackson> So the maximum concurrent requests is spread evenly and should be <1000 at any one time.
15:39:19 <djackson> Yes, I wanted to check this with you
15:39:33 <djackson> I have made sure to cache the Tor state between instances
15:39:45 <djackson> However, we are still stopping and starting Tor a lot
15:40:00 <irl> tor will only download a consensus if it doesn't have one that is valid
15:40:11 <irl> so as long as you've kept the consensus around, that's every couple of hours
15:40:18 <djackson> Okay, that's great to hear.
15:40:23 <djackson> That was my hope.
15:40:48 <djackson> So yeah, this should not pose a problem either. We reuse the consensus between invocations, it's not a clean Tor each time
15:41:14 <irl> ok cool
15:41:21 <djackson> We are not worried about cold start / bootstrapping for the moment.
15:41:34 <djackson> A couple of other questions
15:41:39 <djackson> Tor Browser Update pings
15:41:46 <djackson> IIRC they are sent with each start
15:41:56 <djackson> Which means you will see an uptick in the metrics
15:42:01 <djackson> Do you want me to disable these?
15:42:12 <djackson> In some sense they reflect real Tor Browser Instances
15:42:13 <karsten> maybe, yes.
15:42:38 <djackson> So maybe they should remain on, to reflect the increased load. But the fact we are stopping/starting a lot means the count will be inflated.
15:42:39 <karsten> it shouldn't really affect your statistics.
15:42:45 <djackson> It doesn't change mine, no
15:43:19 <karsten> if it's easy to turn them off, maybe do that. not a huge problem, though.
15:43:25 <djackson> Okay, great, thanks
15:43:47 <djackson> There will be in increase in the guard connection counts, but I can't do much about this ofc.
15:44:14 <djackson> Final question - is there any DOS mitigations (that you are aware of) that would kick in at this loading?
15:44:39 <djackson> I couldn't think of any, but maybe frequent Tor stop-starts will trigger something on the guards?
15:44:52 <irl> will they all be using the same guard all the time?
15:45:01 <irl> i would probably disable guards for this sort of experiment
15:45:08 <djackson> No, each instance is using its own guard
15:45:31 <djackson> I specifically want to see how say 1000 users pick a guard and experience the Tor network
15:45:40 <irl> i'm not aware of any mitigation that would kick in as long as you have properly closed the previous tcp connection
15:45:50 <djackson> Okay, great
15:45:53 <irl> that's not to say that relay operators don't have iptables rules
15:46:05 <irl> or magic network appliances
15:46:45 <djackson> Okay, that's fine as a known unknown - we can't easily distinguish the difference
15:46:47 <gaba> djackson: this could also be a question for the Tor-Moz meeting on Tuesday. dgoulet and asn are working on DOS mitigations.
15:47:26 <djackson> Thanks Gaba, I will ask then as well. I have a slot to present this work in more detail in that meeting
15:47:47 <djackson> Hopefully by then I will have the results of the first run and something to show on the slides.
15:47:56 <karsten> cool!
15:48:08 <djackson> And then when the analysis is finished I will post to tor-scaling list as well ofc :)
15:48:19 <djackson> Thanks for having such quick answers to my questions! :) super helpful!
15:48:51 <karsten> great, thanks for working on this experiment!
15:49:00 <djackson> np, it's been fun
15:49:08 <karsten> heh
15:49:17 <karsten> alright, anything else to discuss today?
15:49:30 <djackson> not from me
15:49:35 <gaba> not from me
15:49:48 <irl> on maven/gradle
15:49:57 <irl> i am playing with gradle but not for anything serious
15:50:08 <irl> i might update the ticket from time to time but it doesn't need a response
15:50:11 <irl> just gathering notes
15:50:15 <karsten> yes, perfect!
15:50:28 <karsten> good to collect some ideas on the ticket before actually working on that.
15:50:47 <irl> currently playing with gradle's antlr4 plugin for parsing directory documents
15:51:11 <irl> nothing else from me for the meeting though
15:51:42 <karsten> I have some early thoughts on using antlr for just that somewhere.
15:51:49 <karsten> I'll send them to you.
15:51:55 <irl> ok cool
15:52:19 <karsten> alright! great meeting, everyone. talk to you next week! bye. o/
15:52:23 <irl> bye!
15:52:25 <acute> bye!
15:52:35 <karsten> #endmeeting