17:00:15 #startmeeting Network team meeting, 20th september 2021 17:00:15 Meeting started Mon Sep 20 17:00:15 2021 UTC. The chair is ahf. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:15 Useful Commands: #action #agreed #help #info #idea #link #topic. 17:00:18 hello hello 17:00:22 o/ 17:00:25 our pad is at https://pad.riseup.net/p/tor-netteam-2021.1-keep 17:00:36 hi! 17:01:34 O/ 17:01:44 ( ) / 17:01:45 o/ 17:01:47 hi 17:01:51 ok! 17:02:00 o/ 17:02:01 https://gitlab.torproject.org/groups/tpo/core/-/boards 17:02:06 how's everybodies' boards looking? 17:02:24 ahf: pretty good, I think 17:02:40 ya 17:03:33 i don't see anything that needs attention 17:04:09 we did the 0.4.7.1-alpha release last week with a minor bump on the way where we hadn't gotten the dirauth's to bump their recommended versions, but it was solved and we released 2 days late 17:04:38 i don't see anything that affects our 0.4.5 and 0.4.6 series in the issue tracker right now 17:04:46 we have some potential backporting to do once 0.4.7 have baked a bit 17:06:35 i take that as a silent nod 17:06:48 we have two discussion points this week: 17:07:00 oh, gaba is afk this week and i am afk on wednesday as an announcement 17:07:06 2021-09-20 [nickm] Plans for this week's arti meeting? 17:07:14 nickm: you wanna go here 17:07:17 ? 17:07:54 sure 17:08:17 I'd like a little advance thinking about what to talk about this week. Just updates and next steps , or any fancier topics we should cover? 17:09:12 i don't remember how these were created, but the end goal with the meetings is that it's a space where we can also coordinate with other developers who uses arti, right? but we don't have any of those yet of course 17:09:55 yeah 17:10:13 also it's a goal to make them interesting so as to encourage interested devs to attend 17:10:52 so it should be less of a status meeting (to avoid duplicated content between the weekly status emails) ? and more about arti related topics 17:11:50 I think that could be a good idea; though I don't want to make permanent changes without gaba 17:12:03 *nod* 17:12:15 what topics would be interesting for this week and have value? 17:13:18 Hm. I can talk about nearly anything, but my working focus is on guards. I guess I could give an overview of some piece of code or I don't know what 17:13:31 Anyone else have any ideas here? 17:14:59 hm, nobody? 17:15:49 the last meeting had 2 externals in them iirc. are they something we need to consider if they make sense to do? i am away on wednesday, so i cannot participate so i am not saying what the topic should/could be here 17:16:38 ok. I'll try to think of something 17:16:46 ok 17:16:47 maybe we could each pick a little example program to work on or something 17:17:50 please check how many people who joins in and then we can maybe follow up on the convo next week about these meetings in general 17:17:53 ok! 17:17:54 Makes sense 17:17:59 2021-09-20 [nickm] Is our 'status update' process below working for us? 17:18:10 I think my next question is moot: most people have in fact done status updates now :) 17:18:13 i go through them each time, i don't feel like i learn anything i didn't know already from them 17:18:14 but 17:18:27 i also don't tell people if they don't update unelss it has been REALLY long time since an update 17:18:29 When I skimmed it before the majority was out-of-date 17:18:34 ah! 17:18:41 i have an alarm for 15 min. to meeting time to write it 17:19:11 I personally don't use the updates at all. With Gitlab and IRC and the fact that I talk to each of you regurlarly, I end up not needing it 17:19:20 i think they are less needed in the current network team format than when we were in different timezones and didn't have the thursday meeting 17:19:23 but I assume people not that close to our dev would think it is useful? 17:19:26 would use it* 17:19:27 and people are really good at also having 1:1 convo's in #tor-dev now 17:19:34 ok, makes sense 17:19:42 dgoulet: i don't think they are read outside this meeting context but i might be wrong 17:20:15 ok added a small line halfway through then we can see if anybody reads it outside the meeting. maybe 17:20:22 i did that with an email at some point and got zero responses 17:20:24 :) 17:20:40 ok! 17:20:45 i think it's s61 time with mikeperry 17:20:52 ok 17:20:53 i don't see other topics or announcements today 17:21:48 so dgoulet and I have been doing the inital tracing and tuning of flow control over onions 17:22:20 I believe the branch is at the point where it can be run in shadow, but there is some things we can still learn from onion testing and tracing 17:23:00 I updated the experiments section in my branch of the spec to describe this proces a bit better, and document more parameter values for tuning, etc: https://gitlab.torproject.org/mikeperry/torspec/-/blob/flow_control_updates_v2/proposals/324-rtt-congestion-control.txt#L914 17:23:37 I had a look at nick's negotiation branch, but I need to dig deeper still 17:24:04 but I also want to get things prepped for shadow, and finish off the rest of the checklist items for flow control asap: https://gitlab.torproject.org/tpo/core/tor/-/issues/40450 17:24:50 jnewsome: how goes the shadow scaling? 17:25:45 Running out of ram 17:25:50 hiro: also, one thing I notced from your graphs of shadow vs live is that there were far less datapoints coming out of shadow 17:26:04 Started doing some ram profiling on Friday 17:26:16 hiro: that's why jnewsome pointed out there's more perf clients than just one: https://gitlab.torproject.org/tpo/network-health/metrics/website/-/issues/40011#note_2751267 17:27:34 it's not something where we can throw more RAM at it? 17:27:42 Starting to wonder if maybe we should plow ahead without 100% client scaling in the meantime 17:27:44 i suspect RAM is cheaper than dev time still. or is it a leak? 17:27:54 We already have 1.5 tb 17:27:58 uf! 17:28:02 jnewsome: so after trying that stack/heap copy avoidance for clients, we could try some value of the client scaling that still uses guards, but with a smaller client multiplication factor 17:28:29 jnewsome: yeah, possible 17:28:50 ok so mikeperry I will aggregate over all the shadow clients and see what I take out of it 17:29:04 The stack/heap setting I was talking about doesn't save much ram, just makes the accounting simpler for profiling 17:29:10 ah 17:29:44 Hiro it'd just be the perf clients right? I'm not saving the logs for the BG traffic clients 17:30:13 I think it is just the perf clients yes 17:31:05 yeah, we just want the perf clients. we may also want some level of exit logs, but I don't see us needing BG clients 17:31:20 ah, finally have matrix up on my laptop instead of just my phone. fighting some hardware issues :P 17:31:48 :-D 17:31:51 i'm saving a few BG clients so we at least have a sample of their logs, but yeah we run out of storage trying to save them all 17:33:10 ok, so i'll see if can get a 10% network size going, still using < 100% client scaling for now. probably 10% 17:33:17 and continue memory profiling in the meantime 17:34:05 from prelim results it looksl ike there might be a small leak in shadow, but most of it is just having thousands of tor clients in the steady-state. 17:35:46 so long as relay utilization looks similar between shadow and live, and perf characteristics match, that should be enough 17:36:28 guards are important still ofc, because we do want those clients to avoid overloaded relays through CBT, but beyond that, it is likely OK if there is a multiplier on their activity 17:36:44 ok, great 17:39:05 anything else? :-) 17:39:12 GeKo: anything from you? 17:39:37 not much 17:39:45 i was mostly in okr mode last week 17:39:56 and then needed to help tor browser folks 17:39:56 ok 17:40:10 i which will continue this week (the browser folks stuff) 17:40:25 but i looked over the overload parts for the support portal 17:40:36 and an upcoming mail to tor-relays@ 17:40:47 yeah I have that ready to go as soon as it is merged 17:40:48 and created a key for our network-report@ help alias 17:41:09 so, we are finally in shape, i think, for flipping the switch 17:41:21 to show some overload on relay search 17:41:32 progress :) 17:42:05 i hope i can get back this week at least to start analyzing the overload i am now collecting for a while 17:42:13 to start notifying relay operators 17:42:16 we'll see 17:42:28 that's all from me 17:42:42 nice, ok 17:43:25 faravahar moved to sbws now, too 17:43:36 we need to keep an eye on that as well 17:43:55 only gabelmoo and moria are left on torflow now 17:44:23 cool. very cool 17:44:31 yeah 17:45:03 are we otherwise done? anything else? 17:45:15 I am good 17:45:35 excellent 17:45:45 thanks all 17:45:47 #endmeeting