15:01:01 <karsten> #startmeeting metrics team meeting 15:01:01 <MeetBot> Meeting started Thu May 21 15:01:01 2020 UTC. The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:01 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic. 15:01:04 <karsten> hi! 15:01:17 * karsten put a few topics on the agenda pad... 15:02:08 <karsten> Public pad URL is: https://pad.riseup.net/p/tor-metricsteam-2020.1-keep 15:02:25 <karsten> please add more topics, and feel free to put them at the top. 15:02:39 <gaba> hi! 15:03:11 <karsten> so much to talk about. 15:03:17 <dennis_jackson> I had a day off today for a public holiday and spent a bit of time on #34257 15:03:25 <karsten> oh! 15:03:37 <dennis_jackson> In the process of writing it up (just got my Trac permissions sorted) 15:03:46 <karsten> neat! 15:04:12 <karsten> added a note to the pad that we should talk about that. 15:04:32 <acute> I was about to update #30586 15:05:08 <karsten> added a note. 15:05:19 <karsten> maybe let's start. 15:05:30 <karsten> we might not have enough time to cover all topics anyway. 15:05:36 <karsten> Update OnionPerf to TGen 1.0.0 (#33974) 15:05:51 <karsten> hi jnewsome! 15:05:59 <jnewsome> karsten: hello! 15:06:07 <karsten> I was just about to talk about #33974. 15:06:21 <karsten> where my main question was: do you need any help here? 15:06:44 <karsten> it's not blocking anything, I'm just trying to unblock you if that is the case. 15:06:58 <jnewsome> looking up the ticket... 15:07:06 <jnewsome> but I assume it's "migrate to tgen 1"? 15:07:09 <karsten> yep. 15:07:46 <jnewsome> main thing that might be helpful is an environment to run in? I think you mentioned getting access to VM(s)? 15:07:56 <karsten> yes. 15:08:00 <karsten> I'll email you about that. 15:08:27 <jnewsome> other than that, I haven't had much time to work on it yet and am mostly learning how things work. I was having some trouble getting the demo working 15:08:46 <karsten> that's fine. 15:08:46 <jnewsome> probably going to poke at it some more today and/or tomorrow 15:09:17 <karsten> okay, great, will email you about getting access to a testing VM in the same place as our other instances are running. 15:09:26 <jnewsome> sounds good. thanks! 15:09:29 <karsten> cool! 15:09:31 <karsten> moving on: 15:09:32 <karsten> requirements are not included in setup.py (#30586) 15:09:41 <karsten> acute: you have an update? 15:10:13 <karsten> my main question there was that we might want to avoid installing packages automatically using pip. 15:10:17 <acute> yes, I was wondering whether there is a distinction here that we need to make - how do {researchers, developers} install and use onionperf on their machines vs how we deploy onionperf in Tor 15:10:38 <karsten> that's a fine question! 15:10:42 <acute> For general purpose use/development, using python-setuptools (which is a standard way of packaging python modules and uses pip to manage dependencies) and then running onionperf on the command line is reasonable, imo 15:10:56 <acute> for 'official' deployments, not so much 15:11:31 <phw> how did we set up onionperf on our deployed instance? 15:11:35 <acute> there, it makes more to install dependencies directly from Debian, and use a systemd service to run Onionperf 15:11:55 <karsten> or, a tmux session... 15:11:57 <acute> this is what the ansible workflow does 15:12:20 <karsten> so, what does that mean to the patch on the ticket? 15:12:26 <jnewsome> I see in the ticket we have a general policy of avoiding such package managers; why is that? (and is it worth maintaining 2 ways of doing it?) 15:12:36 <acute> tmux could work, but it means things get more manual 15:12:40 <acute> and are harder to monitor 15:12:56 <acute> the actual 'python3 setup.py install' step is not even necessary, as you can just run onionperf directly from the git repository 15:12:58 <karsten> acute: yep. it's just the way it is right now. but it totally doesn't have to be that way. 15:13:15 <acute> as long as your PYTHONPATH is set 15:13:24 <karsten> we have avoided using package managers in java code bases. 15:13:33 <karsten> I don't know what the general tor policy is here. 15:14:09 <karsten> and again, I'm not a python person. if this is the way to do it in python, I'm fine with that. 15:14:17 <karsten> I mostly wanted to flag this as potential issue. 15:15:01 <karsten> acute: do you want to write an update on the ticket? 15:15:06 <karsten> with suggestions? 15:15:48 <acute> yes. I think in the long run we should get onionperf in debian 15:15:54 <phw> for what it's worth, bridgedb (also python) uses pip in deployment but our sysadmins would prefer us to use debian packages instead 15:16:07 <karsten> phw: interesting point. 15:16:18 <jnewsome> phw: do you know why they prefer that/ 15:16:40 <karsten> acute: right now it changes too fast to be a useful package in debian, I think. 15:16:47 <phw> jnewsome: you then don't have to worry about vulnerabilities in libraries because debian takes care of that for you 15:17:11 <jnewsome> my understandig is that it's pretty standard to use pip when dealing with a source repo,a nd only use the system package manager if you've packaged it up 15:17:14 <jnewsome> phw: ah fair point 15:17:50 <karsten> okay, let's take this back to the ticket. 15:17:51 <acute> yes, agreed - this is a more of a long-term idea 15:18:16 <karsten> I put a note on the pad that you're writing an update on the ticket. 15:18:58 <karsten> okay, moving on. 15:19:02 <karsten> Analyze unusual distribution of time to extend to first hop in circuit (#34257) 15:19:08 <karsten> dennis_jackson: you had something here? 15:19:24 <dennis_jackson> Yes, quick summary 15:19:44 <dennis_jackson> https://raw.githubusercontent.com/galadran/onionperf-guard-analysis/master/images/histograms/bt1_op-hk2.png 15:19:58 <dennis_jackson> The time to build the first hop for hk has discrete bands 15:20:02 <karsten> heh 15:20:12 <dennis_jackson> But the other distributions do not 15:20:34 <karsten> I ran out of ideas why this is the case. 15:20:50 <dennis_jackson> I then wanted to work out what was going on with the different parameters 15:20:55 <karsten> my current best guess is that there's something strange going on with the network setup. 15:21:06 <karsten> ah, go on. didn't want to interrupt. 15:21:08 <dennis_jackson> so I started looking at start2req, req2fb and the sum of the build times. 15:21:43 <dennis_jackson> There's a strange lack of correlation, at least to my mind 15:21:48 <karsten> yeah... 15:22:04 <dennis_jackson> I would naively expect start2req to be about the same as the sum of the build times 15:22:19 <karsten> circuits might have been built before the request came in. 15:22:29 <dennis_jackson> and req2fb to be correlated with start2req 15:23:03 <karsten> do you know where to find the onionperf analysis files? 15:23:25 <karsten> https://op-hk2.onionperf.torproject.net:8443/htdocs/ 15:23:33 <karsten> just in case you want to dig deeper. 15:23:43 <dennis_jackson> Thanks 15:24:09 <dennis_jackson> Anyway, the correlations are much higher for nl than for hk 15:24:14 <karsten> but yes, I was actually looking for correlations there, too, but didn't find much. 15:24:49 <dennis_jackson> ['op-nl2,public,start2req,total_build', 0.6297887235661617] 15:25:03 <dennis_jackson> ['op-hk2,public,start2req,total_build', 0.4421435991718628] 15:25:11 <dennis_jackson> ['op-us2,public,start2req,total_build', 0.43186586805012234] 15:25:30 <dennis_jackson> Final column is the spearman correlation coefficient 15:25:54 <dennis_jackson> So looks like nl is using the circuits quicker maybe? But unsure without looking at the detailed files 15:26:19 <karsten> there are also the raw logs. 15:26:22 <dennis_jackson> I didn't find much conclusive, just lots of strange things. I plan to look a bit at the onionperf doc to improve my mental model a bit. 15:26:32 <karsten> https://op-hk2.onionperf.torproject.net:8443/tgen-client/log_archive/ 15:26:39 <karsten> https://op-hk2.onionperf.torproject.net:8443/tor-client/log_archive/ 15:27:19 <karsten> okay, let me add to the pad that you're planning to look more. 15:27:31 <karsten> would it make sense to start a second set of measurements nearby? 15:27:57 <dennis_jackson> Can't hurt, provided its not too much hassle. 15:28:09 <karsten> I don't know for hong kong. 15:28:20 <karsten> but we should easily get another machine in florida. 15:28:39 <karsten> and the netherlands. 15:28:44 <karsten> I'll try that. 15:29:07 <karsten> okay! 15:29:10 <dennis_jackson> okay, cool :) 15:29:15 <karsten> thanks for looking into this! 15:29:27 <dennis_jackson> np, its a nice change of pace from contact tracing 15:29:32 <karsten> heh 15:29:40 <karsten> moving on to: 15:29:41 <karsten> Reduce the number of 50 KiB downloads (#34023) 15:30:00 <karsten> what are the general thoughts on doing just the 5 MiB downloads? 15:30:21 <karsten> every five minutes, that is. 15:30:55 <dennis_jackson> I am in favour 15:31:09 <acute> that's 288 measurements per day? 15:31:29 <karsten> yep. 15:32:49 <acute> this is a good thing to do, I wanted to compare the number of measurements with how many we're doing at the moment 15:33:02 <karsten> the number of measurements stays the same. 15:33:10 <karsten> they'll all going to be 5 MiB downloads, though. 15:33:44 <karsten> by the way, the reason why there's no resp2done with the time from first to last byte in #34257 is that we don't have enough data for that. 15:33:57 <karsten> it just doesn't make much sense to do that for 50 KiB. 15:34:30 <karsten> okay, so no major concerns "OMG what are you doing"? 15:35:11 <karsten> sounds like it. 15:35:17 <karsten> great! 15:35:25 <acute> no major concerns here! 15:35:29 <karsten> :) 15:35:33 <karsten> next one is related: 15:35:34 <karsten> Reduce timeout and stallout values (#34024) 15:35:47 <karsten> the change is minor though. 15:35:58 <karsten> the idea is to reduce the timeout to 270 seconds to avoid overlapping measurements. 15:36:05 <karsten> and not use the stallout feature at all. 15:36:23 <karsten> but again, the impact is much smaller. 15:36:44 <karsten> I'd say if this seems like a crazy idea, be sure to shout either here or in the next couple hours on the ticket. 15:36:47 <karsten> otherwise we'll do it. 15:36:52 <dennis_jackson> Your proposal looks good to me. I think it would improve data quality 15:37:11 <karsten> to a very small extent in this case, but yes. 15:37:11 <dennis_jackson> It really just means that we consider failure to be after 3.5 minutes right? 15:37:21 <karsten> 4.5, yes. 15:37:30 <acute> it think it's an improvement 15:38:03 <karsten> cool! 15:38:15 <karsten> added a note. 15:38:17 <karsten> next is: 15:38:18 <karsten> Allow users to select Onion Service version to measure (#33434) 15:38:39 <karsten> basically, the idea is to kill v2 onion service measurements. 15:38:50 <karsten> they're almost exactly the same as v3 onion service measurements. 15:39:00 <karsten> I'd like to hear from dgoulet or asn about this. 15:39:39 <gaba> asn and dgoulet are in other meeting now. 15:40:01 <asn> karsten: just replied 15:40:02 <karsten> but if we do this and kill v2 onion service measurements, we'll have 1/2 measurements to public services and 1/2 to (v3) onion servies. 15:40:04 <gaba> :) 15:40:08 <karsten> oh hey! 15:40:31 <karsten> and you agree, nice. :) 15:40:36 <karsten> thanks for commenting! 15:40:47 <asn> o/ 15:41:22 <karsten> removed the next topic because of time. 15:41:24 <karsten> Remove existing Tor control log visualizations (#34214) 15:41:34 <karsten> err, let me remove that, too. 15:41:49 <karsten> I'll bring them up next week. 15:42:13 <karsten> the next two are candidates for being picked up by somebody looking to write code. 15:42:16 <karsten> Fix message logging and filtering (#29369) 15:42:34 <karsten> I looked into the bug there and suggested a possible fix. 15:42:53 <karsten> now all that remains to be done is to write the code. :) 15:43:08 <karsten> same goes for: 15:43:09 <karsten> Combine multiple analysis files into single data set (#34191) 15:43:28 <karsten> except that that one is an enhancement, not a defect. 15:43:56 <karsten> we don't have to assign them now. 15:44:29 <karsten> just putting this out here if someone's looking for a task that doesn't involve diving deep into the code first. 15:45:13 <karsten> okay, if there are no questions, moving on? 15:45:41 <karsten> doing so. 15:45:42 <karsten> Python style 15:45:59 <karsten> this came up in the latest review. 15:46:11 <karsten> thanks, acute, by the way for putting all those effort into that review!! 15:46:16 <karsten> that* 15:46:41 <acute> :D 15:46:57 <karsten> what are the tools I should be using in the future to write code that complies with generally accepted style guides? 15:47:10 <karsten> what are other teams using? 15:48:02 <phw> karsten: there's flake8, which i use for bridgedb 15:48:20 <jnewsome> I think that's what we use in shadow as well 15:48:25 * jnewsome is checking... 15:48:37 <acute> pylint, yapf, black and pyflakes were mentioned on the ticket 15:49:20 <jnewsome> yeah, we use flake8 for enforcement. https://github.com/shadow/shadow/blob/master/.github/workflows/lint_python.yml 15:49:26 <jnewsome> (we = shadow) 15:49:45 <karsten> how much do they disagree? 15:49:51 <karsten> like, what if we use all of them? 15:49:59 <acute> flake8 seems to be a wrapper on top of pyflakes and pycodestyle, can you use it to reformat code? 15:50:35 * jnewsome doesn't know 15:51:24 <karsten> how about we try flake8 for the moment? 15:51:31 <acute> yapf and black are both formatters, it looks like the other tools are checkers 15:52:02 <karsten> I think black is, yes. 15:52:34 <acute> same for yapf 15:53:10 <karsten> okay. I'll try them out and bring this topic back to the agenda next week. 15:53:24 <karsten> (feel free to do the same.) 15:53:46 <karsten> last topic for today: 15:53:47 <karsten> Reviews 15:54:04 <karsten> I don't think there are any open reviews this week. 15:54:16 <karsten> we covered a few tickets currently under review. 15:54:32 <karsten> the part about cc'ing metrics-team is a minor issue, though. 15:54:56 <karsten> the thing is, if we don't cc metrics-team, updates are not being sent to metrics-bugs@. 15:55:20 <karsten> it's a minor issue, because I'll see updates eventually, but it's the reason why it takes me longer to respond. 15:55:42 <karsten> I hope we'll soon switch to gitlab and this problem will go away. ;) 15:55:49 <dennis_jackson> +1 15:56:04 <karsten> anyway, now you know. if you remember, put metrics-team in the cc field. 15:56:17 <karsten> anything else for today? 15:56:25 <acute> not from me! 15:56:28 <dennis_jackson> nothing from me 15:56:32 <jnewsome> nope 15:56:43 <phw> shall we add metrics-team to trac's auto-cc list? 15:56:52 <phw> for onionperf etc., that is 15:56:52 <karsten> that's for new tickets? 15:57:02 <karsten> we should, yes. 15:57:08 <phw> yes, for new tickets 15:57:21 <karsten> good idea, I'll do that. 15:58:02 <karsten> okay! thanks for a stuffed but very productive meeting, everyone! 15:58:19 <karsten> and thanks for working on making onionperf better!! 15:58:26 <phw> o/ 15:58:29 <jnewsome> o/ 15:58:30 <acute> o/ 15:58:30 <dennis_jackson> :) o/ 15:58:31 <karsten> talk to you next week, same time, same place. bye! o/ 15:58:38 <gaba> o/ 15:58:42 <karsten> #endmeeting