17:00:04 <ahf> #startmeeting Network team meeting, 27th september 2021
17:00:04 <MeetBot> Meeting started Mon Sep 27 17:00:04 2021 UTC.  The chair is ahf. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:04 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
17:00:06 <ahf> hello hello hello
17:00:14 <ahf> pad at https://pad.riseup.net/p/tor-netteam-2021.1-keep
17:00:26 <GeKo> o/
17:00:28 <juga> o/
17:00:32 <dgoulet> yello
17:00:35 <ahf> there is privchat running in parallel to that so i could imagine people being a bit slow too
17:00:36 <jnewsome> o/
17:00:58 <mikeperry> o/
17:01:13 <ahf> o/
17:01:25 <ahf> nickm: are you in the house?
17:01:39 <ahf> how are folks' boards looking: https://gitlab.torproject.org/groups/tpo/core/-/boards ?
17:01:49 <nickm> hihi!
17:01:53 <nickm> sorry, was making tea
17:02:00 <ahf> very nice
17:02:03 <hiro> o/
17:02:17 <nickm> hi hiro!
17:02:29 <nickm> my board looks okay; my reviews are a bit much :/
17:02:41 <ahf> i have no reviews right now - you can pass one or two to me?
17:02:46 <hiro> hello nickm
17:03:01 <nickm> dgoulet, mikeperry: will you have a chance to look at tor#40461 this week? I want to fix issues and answer questions before I forget what I did
17:03:27 <dgoulet> oh I thought mikeperry answered you .. hmm
17:03:44 <mikeperry> oh I had some initial comments on irc. I need to look deeper, probably comment on the MR
17:03:50 <nickm> I don't see any comments on tor!440
17:03:53 <mikeperry> well more like questions
17:04:13 <nickm> okay; can you put them on the bugtracker?
17:04:18 <mikeperry> yeah ok
17:04:22 <nickm> ahf: Thanks; I'll pass something on to you
17:04:30 <ahf> nickm: sounds good
17:04:57 <nickm> mikeperry: I hope you weren't waiting long :/
17:05:39 <mikeperry> no it's ok. not blocked on this; currently working to get the flow control code tuned with dgoulet so we can sim it. that can be done without negotiation, just force-on
17:06:04 <ahf> ok! it looks like we are doing OK with the releases. haven't seen any catastrophes being reported so far on the alpha
17:06:05 <mikeperry> that's also why I didn't look deep at this branch yet
17:07:08 <ahf> i see no announcements. remember NT folks that on thursday our face-to-face sync is a roadmap session
17:07:40 <ahf> GeKo, hiro: if you have something we have to do in tor in the near future that may not be tracked on one of our sponsors it could make sense for us to talk about that over the week
17:07:48 <ahf> i assume you're also doing some q4 roadmap activity right now?
17:07:59 <hiro> yes on wed
17:08:12 <ahf> very good
17:08:30 <ahf> we have no announcements. no discussion points right now. so i think mikeperry and s61 can takeover the meeting
17:08:47 <mikeperry> ok
17:08:55 <GeKo> yeah, we'll get back to you once we think there is some dep we need helpw tih
17:08:58 <GeKo> *with
17:09:12 <ahf> sounds good
17:09:39 <mikeperry> so yeah, the main haps with s61 has been me and dgoulet working to fine-tuune flow control
17:10:10 <mikeperry> I think we have enough results to fix up the branch and get it ready for both review and sim in shadow now
17:10:16 <mikeperry> I hope to have that done this week
17:10:21 <ahf> exciting!
17:10:45 <dgoulet> yeah it is pretty cool
17:11:06 <ahf> i am so impressed with some of the tracing work that have happened as part of this work. i don't fully understand all of it yet, but it is nice to follow the discussions on this work
17:11:41 <mikeperry> yeah, dgoulet pulled out his ninja lttng skills and got kernel info and tor outbuf info all lined up
17:11:41 <nickm> I'm happy about everybody levelling up on their understanding of scheduling and stuff, so we can implement it all in rust :)
17:12:15 <ahf> round #2 :-)
17:12:44 <ahf> anything else we need to discuss?
17:12:48 <mikeperry> jnewsome,hiro: how's the latest on benchmarking stock tor in shadow?
17:13:10 <hiro> I am hooking onionperf visualize to sporksmith[m] pipeline
17:13:18 <mikeperry> did we get those extra datapoints in? and is the mem usage struggle still continuing?
17:13:21 <mikeperry> nice
17:13:38 <hiro> but I have had some bash issue that have been taking me a bit too long to solve because the simulation is a bit long
17:13:39 <jnewsome> i've hit a (hopefully small) snag trying to run a 10% network - tornettools is generating invalid configs https://github.com/shadow/tornettools/issues/38
17:13:47 <hiro> if everything goes well I will have it ready to merge this week
17:14:13 <jnewsome> and the machine is a bit in contention now with hiro7 also running sims :)
17:14:36 <jnewsome> hiro7 maybe we should take a look at configuring a smaller sim for faster iteration on post-processing work?
17:14:44 <jnewsome> there are a couple easy knobs there
17:15:01 <mikeperry> yeah, that pinning to run two at once on each set of numa cores comes to mind
17:15:18 <mikeperry> that will be useful long-term I bet
17:15:29 <mikeperry> in addition to etting you both test at once
17:15:32 <hiro> I tried that with sporksmith[m] but I haven't found the right spot... some of the files weren't generated?
17:15:46 <hiro> I tried 1 trial and 7 minutes
17:15:50 <hiro> and was missing the tgen files
17:16:03 <jnewsome> yeah, i think you need at least 10 minutes
17:16:07 <hiro> sorry for clogging yoru server
17:16:13 <hiro> ok i'll do that then
17:16:17 <jnewsome> no worries, it's your server too :)
17:16:25 <hiro> yeah but I don't really need the sim
17:16:41 <hiro> so all that power is wasted really
17:16:48 <jnewsome> yeah, i guess to the degree you can iterate on the post processing locally without running fresh sims, that'd be helpful
17:17:34 <jnewsome> meanwhile i'm working on further profiling and reducing memory usage, which i can now do locally
17:18:02 <hiro> yeah it's a little tweaking with gitlab-ci paths and then I am done. all on my side
17:18:52 <mikeperry> cool
17:19:09 <mikeperry> I am excited. we are getting close :)
17:19:14 <jnewsome> i wonder if it's worth splitting into two separate gitlab workflows
17:19:25 <jnewsome> where a post-processing workflow takes the output of the simulation workflow
17:19:32 <jnewsome> so that we can more easily iterate on the post-processing
17:19:43 <jnewsome> IDK how hard that is to set up in gitlab
17:19:59 <hiro> uhm
17:20:09 <hiro> I could access the artifacts I guess with a link
17:21:06 <jnewsome> maybe not worth messing with if you think you're close to done anyway, but an option to think about if we end up needing to iterate more
17:22:20 <hiro> the thing is the artifact links change at every run if I am not mistaken unless you publishes it somewhere
17:22:26 <hiro> like with gitlab pages
17:22:36 <jnewsome> yeah, not sure there's an easy way to grab the "latest"
17:23:22 <jnewsome> i guess we could upload somewhere else instead of using artifacts, but that's starting to sound like a bigger project
17:23:56 <hiro> yeah well I'll find a way
17:24:43 <jnewsome> ok, well anyway i'm hoping to have a baseline 10% network result soon; this week if things go well
17:25:00 <mikeperry> \o/
17:25:30 <mikeperry> GeKo,hiro: how goes the overload stuff? any other network health stuff related to s61 to discuss?
17:26:13 <GeKo> i've started contacting relay operators to fix their overload
17:26:32 <GeKo> i began with overload-fd as those might be easiest to fix
17:26:47 <mikeperry> yah
17:26:55 <GeKo> other than that we launched the relay-search support
17:27:01 <GeKo> with the support article
17:27:10 <GeKo> and deal now with questions from relay operators
17:27:15 <GeKo> and fix bugs etc.
17:27:24 <GeKo> so, i think overall we are good
17:27:29 <GeKo> and nice to see that finally live
17:28:10 <mikeperry> yep
17:28:19 <GeKo> i plan to get some overview about how the overload develops over time
17:28:35 <GeKo> looking at the data i've accumulated over the past weeks
17:28:41 <mikeperry> yeah, correlating it to rob's experiments will also be interesting
17:28:46 <GeKo> yep
17:29:05 <GeKo> i might have graphs ready by the end of this week
17:29:08 <GeKo> we'll see
17:29:13 <mikeperry> cool
17:29:15 <GeKo> that's all from my side
17:30:22 <mikeperry> ok. I think we're good then
17:30:30 <ahf> excellent
17:30:33 <mikeperry> dgoulet: I will follow up in a bit about spec updates and such
17:30:45 <ahf> thanks all! NT people: remembero ur thursday sync on roadmap things :-)
17:30:49 <ahf> l8r everybody o/
17:30:53 <GeKo> o/
17:30:54 <ahf> #endmeeting