14:29:17 <karsten> #startmeeting metrics team
14:29:17 <MeetBot> Meeting started Thu Nov 15 14:29:17 2018 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:29:17 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:29:41 <karsten> https://storm.torproject.org/shared/5h1Goax5eNusxjXJ_Ty5Wl7hFR1uqCReUiN8xdlBG8T <- agenda pad
14:29:56 * gaba in listening mode in an interview to anti-censorship position. So half here.
14:30:01 <karsten> ok
14:31:36 <karsten> anything else for the agenda?
14:31:45 <irl> not from me
14:31:58 <karsten> ok. let's start.
14:32:01 <karsten> * Onionoo release for #28254 (karsten)
14:32:13 <karsten> how about we put out a new release soon?
14:32:19 <irl> sounds good
14:32:28 <irl> tomorrow after lunch?
14:33:12 <karsten> tomorrow afternoon is bad for me. I could rather deploy on the weekend or on monday.
14:33:26 <irl> monday afternoon?
14:33:26 <karsten> I can prepare a tarball for tomorrow morning.
14:33:40 <karsten> monday afternoon sounds good.
14:33:44 <karsten> 14:30 utc?
14:33:47 <irl> ok
14:34:06 <karsten> cool.
14:34:22 <karsten> sounds like we have a plan.
14:34:24 <karsten> next is:
14:34:26 <karsten> * check and TorDNSEL VM upgrade to Debian buster (irl)
14:34:31 <irl> ln5: are you about?
14:34:51 <irl> so the problem here is that the vm runs jessie and we'd like it to run stretch
14:34:59 <irl> but there may be crazy haskell dependency issues
14:35:15 <irl> we don't actually know, how involved is arlo now? iirc not so involved
14:35:23 <karsten> he's not?
14:35:45 <irl> i remember having discussions about replacing tordnsel because it wasn't maintained
14:36:17 <karsten> not maintained in the sense that there wouldn't be enhancements like ipv6 support.
14:36:31 <karsten> but maintained in the sense of fixing bugs or kicking it when necessary.
14:36:41 <irl> ok, would there be enhancements like fixing things to run on newer versions of debian?
14:36:47 <karsten> not sure how these crazy dependency issues count here.
14:37:06 <karsten> we should ask arlo. I can ask him.
14:37:21 <irl> i think ln5 has already asked him, but i wanted to see if you had more information
14:37:34 <karsten> no, I don't.
14:37:47 <irl> ok, then there's not much else we can do on that
14:37:55 <irl> i did say that we can commit to replacing it before 2020
14:38:00 <karsten> hehe
14:38:01 <irl> which is where debian lts support ends
14:38:14 <karsten> ok.
14:38:15 <irl> (which isn't really debian, but it's better than nothing)
14:38:40 <karsten> alright, so let's first wait for arlo to respond.
14:38:44 <irl> yep, ok
14:38:45 <karsten> and maybe discuss this again next week.
14:39:03 <irl> cool
14:39:05 <karsten> ok.
14:39:06 <karsten> * Progress update for CollecTor prototype replacement (irl)
14:39:18 * karsten is curious!
14:39:27 <irl> i wrote a load of code and found bugs in both tor and stem
14:39:36 <karsten> neat!
14:39:50 <irl> https://github.com/irl/bushel is the code
14:40:05 <irl> https://irl.github.io/bushel/ is some very basic docs
14:40:29 <irl> the current state of it is that it can fetch consensuses and votes and then identify server descriptors and extra info descriptors that are needed
14:40:42 <irl> it's a single-shot thing at the moment though, so you have to re-run it to retry
14:40:59 <irl> but it is storing the files using collector's file system protocol and doesn't download descriptors it already has
14:41:44 <irl> asyncio seems to be a good framework to avoid threading everywhere
14:41:58 <karsten> sounds good. is there anything I can do?
14:42:10 <irl> initially i had decided that advanced python scheduler wasn't really what we wanted, but used with asyncio it looks like it's going to be able to replace any manual scheduling logic we need
14:42:29 <irl> i think at the moment there isn't really, but probably next week i'll ask you to take another pass over the design document
14:42:40 <irl> latest version: https://people.torproject.org/~irl/volatile/modern-collector.pdf
14:42:49 <irl> but probably worth waiting until i ping you
14:43:01 <karsten> sounds good.
14:43:37 <irl> mostly just wanted to let you know what i'm doing, in case i'm doing completely the wrong thing and should switch
14:43:51 <irl> i've been quite deep into this so i'm not sure if i'm missing things going on around me
14:44:19 <karsten> the drafts you showed me earlier were going into the right direction.
14:44:42 <irl> excellent, so i'll ping you probably wednesday next week to ask for some review and maybe even you can try out the code
14:44:45 <karsten> I can't comment on the scheduling parts in python.
14:44:59 <karsten> sounds good!
14:45:17 <karsten> made a note to expect something then.
14:45:24 <irl> cool (: that's all on that then
14:45:35 <karsten> great!
14:45:39 <karsten> * metrics-web progress update and plans: ipv6servers, totalcw (karsten)
14:45:49 <karsten> you already saw the update on #28116.
14:46:01 <irl> yes, hopefully will be able to do this before monday
14:46:19 <karsten> so, when these changes go in we'll have lost some of the oldest metrics code!
14:46:28 <irl> wooo!
14:46:48 <karsten> I'm thinking what to do next.
14:47:01 <karsten> I could easily continue there, but I could also do something else.
14:47:11 <irl> when these changes go in, how easy is it to make the churn by flag graphs?
14:47:13 <karsten> it sounds like you're happily working on the collector thing until mid next week.
14:47:23 <karsten> errrr
14:47:35 <karsten> which graphs?
14:47:43 <irl> https://metrics.torproject.org/networkchurn.html
14:48:24 <karsten> to be honest, I didn't think about those yet.
14:48:39 <karsten> it might be that we'd have to write a new module for those.
14:48:51 <irl> ah ok, just that someone was looking for this data yesterday and if it was easy then it would be useful
14:49:05 <irl> if it's not so easy then we can not do it, we weren't actually asked for it directly
14:49:21 <karsten> I'll think about it. it might be that it's not easy.
14:49:55 <karsten> so, I wonder if I should do the totalcw rewrite next.
14:50:11 <karsten> we're basically being asked for a rewrite there.
14:50:17 <karsten> distributed over several tickets.
14:50:25 <karsten> but what we have is not what we should have made. :(
14:50:41 <irl> there is now an sbws deployed, so this is the time that it is needed i guess
14:50:46 <karsten> I'm just pointing this out, because it's going to take me days to do right.
14:50:56 <karsten> right, that's also my guess.
14:51:43 <karsten> okay, I'll add it to my list then.
14:51:55 <irl> yes, i think it is ok to do that next
14:52:07 <karsten> ok!
14:52:13 <karsten> * Next week's meeting time (karsten)
14:52:18 <irl> oh hang on
14:52:20 <karsten> yes?
14:52:39 <irl> it is possible that we can start archiving bandwidth files very soon
14:52:43 <irl> in collector
14:52:49 <karsten> ah!
14:53:01 <irl> so you should probably make sure you have time for this
14:53:05 <irl> just when it pops up
14:53:14 <karsten> any idea when?
14:53:27 <irl> 1-2 weeks is my understanding
14:53:38 <karsten> okay. totalcw should be done by then.
14:53:54 <irl> coo
14:53:57 <irl> l
14:54:13 <karsten> * Next week's meeting time (karsten)
14:54:24 <karsten> so, my next meeting today is at 17:00 utc.
14:54:34 <karsten> I wouldn't mind a shorter break between the two meetings.
14:54:48 <karsten> but 14:30 utc would still work for me.
14:54:51 <irl> we could change to 15:00
14:55:06 <karsten> 16:00 utc is too late?
14:55:32 <irl> i guess 16:00 is ok
14:55:40 <karsten> cool!
14:55:54 <karsten> let's try out 16:00 utc then. we can always change it if it doesn't work out.
14:56:26 <karsten> I guess we're out of topics.
14:56:28 <irl> ok
14:56:31 <irl> yes
14:56:35 <irl> gaba: did you have anything?
14:57:22 <irl> (maybe gaba is gone)
14:57:22 <gaba> nop
14:57:25 <irl> ah ok
14:57:27 <gaba> I'm here :)
14:57:28 <irl> cool
14:57:29 <karsten> cool!
14:57:49 <karsten> thanks, irl and gaba! let's talk more next week, at 16:00 utc.
14:57:55 <karsten> bye! :)
14:57:57 <irl> bye!
14:58:00 <karsten> #endmeeting