14:59:46 <karsten> #startmeeting metrics team
14:59:46 <MeetBot> Meeting started Thu Apr  6 14:59:46 2017 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:59:46 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:59:57 <karsten> https://pad.riseup.net/p/3M7VyrTVgjlF <- agenda
15:00:09 <karsten> Sebastian: ready to start with your topics?
15:00:19 <Sebastian> ok
15:00:26 <karsten> - a couple of topics that came up during AMS (Sebastian)
15:00:28 <karsten> great!
15:00:38 <karsten> - it'd be great if we had a feature where we can see which dirauth dictated the median for a given relay wrt bandwidth weight (perhaps also with #relays over time)
15:00:43 <hiro> hi!
15:00:48 <karsten> hi hiro!
15:00:55 <Sebastian> people seemed confused about that topic. What's the confusion?
15:01:03 <karsten> what kind of feature?
15:01:12 <karsten> one graph now?
15:01:20 <karsten> something on tor metrics?
15:01:21 <Sebastian> I'm thinking in consensus-health
15:01:27 <karsten> ah!
15:01:27 <Sebastian> where we list the relays
15:01:51 <Sebastian> it could be cool if there were an indication which dirauth(s) is/are the median
15:01:53 <karsten> that would be something for tjr, right?
15:01:57 <Sebastian> I don't know
15:02:03 <Sebastian> is tjr not part of the metrics team?
15:02:13 <karsten> not a regular attendee of these meetings, no.
15:02:41 <Sebastian> number of relays might be something for metrics
15:02:42 <Sebastian> over time
15:03:04 <tjr> But I have highlights!
15:03:07 <tjr> And sometimes I'm at my desk!
15:03:11 <karsten> hehe
15:03:17 <karsten> hi!
15:03:54 <karsten> regarding the number of relays thing, I wonder if such a graph would be too specific for the average tor metrics visitor.
15:04:08 <karsten> but how about this: we can make one graph now and then see how interesting that would be.
15:04:10 <tjr> I think this is something I can get onto consensus health in a weak form fairly easily; and in a strong form over the next week...ish? Don't slave me to that
15:04:23 <Sebastian> maybe. Do we have a better place for stuff that's important for dirauths but boring for the average metrics page visitor?
15:04:23 <tjr> Can you open a ticket and assign it to me just in case?
15:04:38 <Sebastian> tjr: sure
15:04:39 <karsten> Sebastian: consensus-health?
15:05:01 <Sebastian> consensus-health doesn't do history
15:05:14 <Sebastian> it would be cool to analyze historical data for this
15:05:18 <karsten> https://consensus-health.torproject.org/graphs.html
15:05:36 <karsten> so, doing a historical analysis once is easy.
15:05:47 <karsten> finding a place and keeping it updated is harder.
15:06:02 <tjr> Besides the graphs, we also save 2 (or 3?) weeks of detailed pages.
15:06:30 <tjr> My plan was to get the authority who described the bw highlighted today (hopefully) and make a graph showing stack of bwauth 'soon'
15:07:25 <karsten> so, should we do a one-off analysis of the past years? would that be useful?
15:07:54 <karsten> unless you were planning to import years of data, tjr?
15:08:07 <tjr> I was not :)
15:08:22 <karsten> Sebastian: ^ would that be potentially useful?
15:08:56 <karsten> Sebastian: and if so, can you create a ticket for this?
15:09:12 <Sebastian> yeah
15:09:15 <Sebastian> that'd be useful
15:09:25 <Sebastian> ideally we'd have a way to redo the analysis on demand
15:09:31 <karsten> that's easy, too.
15:09:33 <Sebastian> but having it for the past year would already tell us a lot
15:09:42 <Sebastian> will create tickets
15:09:46 <karsten> okay. please put this all in the ticket.
15:09:48 <Sebastian> ok, I think I took up a lot of meeting time already
15:10:05 <karsten> well, let's go through the other two quickly then.
15:10:10 <karsten> - webstats takeover and using them for graphs (mail from isa iirc)
15:10:24 <karsten> I think we already put this on our Q2 list.
15:10:28 <iwakeh> yes.
15:10:30 <Sebastian> I don't really have anything to say for that, I keep maintaining it but the sooner you do it the better :)
15:10:36 <karsten> right.
15:10:42 <iwakeh> we'll hurry
15:10:48 <iwakeh> as much as we can :-)
15:11:01 <karsten> - bwauth feeding servers monitoring
15:11:11 <karsten> the thinking last week was that tor's nagios could do this.
15:11:26 <tjr> This is very interesting to me.
15:11:37 <karsten> hiro: ^
15:11:38 <Sebastian> I was thinking it's like torperf or something
15:11:55 <Sebastian> where we just try to reach the bwauths from non-tor and see if they're very slow.
15:11:58 <hiro> yep I think nagios could do that
15:12:05 <Sebastian> ok
15:12:11 <tjr> I have three checks I perform on my bwauth: one to detect a low number of relays, one to detect if a scanner hasn't completed recently, and one to detect if the relay percentaged measured is too low.
15:12:11 <karsten> can nagios warn if the server is slow?
15:12:16 <hiro> but we should see first
15:12:19 <tjr> Are those the types of things we were thinking about?
15:12:31 <Sebastian> tjr: no this is for the feeding server
15:12:38 <hiro> there are a number of checks... I do not know nagios very well to be honest
15:12:41 <tjr> Oh where the files are hosted?
15:12:44 <tjr> Okay
15:12:49 <Sebastian> yeah
15:12:56 <hiro> maybe nagios is a bit overkill though if all we want to know is if the server is slow or not
15:13:01 <Sebastian> I'm thinking the other checks are very important too, but separate
15:13:15 <Sebastian> hiro: ideally I'd want a notification if it turned out to be slow.
15:13:17 <hiro> we could do a ping check or something of the like
15:13:20 <Sebastian> Like Damian's script
15:13:23 <hiro> yeah
15:13:25 <Sebastian> that downloads consensus from driauths
15:13:26 <Sebastian> dirauths*
15:13:39 <Sebastian> maybe it could download from the feeding server too.
15:13:41 <karsten> sounds plausible, too.
15:13:58 <iwakeh> nagios can run shell scripts; so that should work.
15:14:02 <tjr> Please let me know if/when this happens, as I use Linus' server which should be added (and wouldn't be if I didn't mention it.
15:14:13 <tjr> (FWIW if anyone other bwauth wants me to run the checks against their's I can, we can arrange it outside of this meeting.)
15:14:42 <hiro> if we could define what we want from nagios I could find this out and try to see how we can do it?
15:15:02 <hiro> so would be good for me to know if we should run a set of scripts or implement a number of checks...
15:16:00 <karsten> sounds like a ticket for further discussion. :)
15:16:03 <Sebastian> I'm thinking this might not be a good fit for nagios
15:16:09 <Sebastian> rather I'll talkt to atagar
15:16:12 <karsten> in which case it would be a DocTor ticket.
15:16:12 <Sebastian> and then open the tickets
15:16:19 <karsten> sounds good!
15:16:38 <karsten> alright, moving on to the next topic?
15:16:56 <karsten> 3, 2, 1...  - Onionperf deployment (hiro)
15:17:00 <hiro> yep
15:17:10 <hiro> so today we had a major update from Rob
15:17:15 <karsten> yay!
15:17:21 <hiro> and we should start to see new measurements from tomorrow
15:17:29 <karsten> is there any way to get measurements today?
15:17:37 <karsten> as a sample on the ticket?
15:17:42 <karsten> or, attachment?
15:17:43 <hiro> I can check the logs and generate some yes
15:18:05 <karsten> cool!
15:18:17 <karsten> and we'll have to delete older data again, right?
15:18:31 <hiro> yep if we do not want that we will have to
15:18:47 <karsten> well, 50% is timeouts.
15:18:54 <karsten> we could filter that, but ugh.
15:19:06 <hiro> yeah it is not a big deal do delete data
15:19:08 <karsten> it's surprisingly difficult to add new data sources..
15:19:18 <hiro> yes!
15:19:33 <karsten> thanks for your help there! :)
15:19:49 <karsten> okay, anything else on the onionperf topic for today?
15:20:10 <hiro> i think there isn't much else.. rob is discussing some changes but that won't probably affect us that much
15:20:24 <karsten> yep.
15:20:44 <karsten> okay,
15:20:46 <karsten> - In-memory stats (karsten)
15:20:48 <hiro> when the twistd server will be remove I will just serve the xml w the nginx directly
15:20:55 <hiro> yep moving on :) sorry about it
15:21:02 <karsten> sounds good!
15:21:18 <karsten> alright, in-memory stats.
15:21:37 <iwakeh> simulation?
15:21:40 <karsten> it's likewise surprisingly difficult to find time to focus on this simulation..
15:21:50 <karsten> but I found two bugs in the simulation code.
15:21:51 <iwakeh> yes,
15:22:02 <iwakeh> that's good.
15:22:02 <karsten> one related to binning negative values.
15:22:13 <karsten> and one related to de-binning reported values more than once.
15:22:33 <karsten> but this is still work in progress.
15:22:43 <iwakeh> Can I help?
15:22:53 <iwakeh> Timewise.
15:23:22 <karsten> maybe let me keep this for tomorrow and the weekend and then hand over or discuss what I have on monday.
15:23:31 <iwakeh> fine.
15:23:38 <iwakeh> It's not too urgent.
15:23:58 <karsten> unless we find out that our current plan for adding noise does not work out.
15:24:11 <karsten> in which case we'd need to find an alternative really quickly.
15:24:21 <iwakeh> I think it'll work.
15:24:36 <karsten> I hope so!
15:24:50 <karsten> okay, nothing else on the simulation topic. quite related...
15:24:53 <karsten> - tech report (iwakeh): restructuring and adding the AMS results
15:25:02 <iwakeh> yes, the theory behind
15:25:12 <iwakeh> also work in progress
15:25:18 <iwakeh> but I found a good way
15:25:26 <iwakeh> for re-reading reviewing
15:25:39 <iwakeh> I'm going to use latexdiff, which will
15:25:49 <iwakeh> make changes nicely visible.
15:26:06 <karsten> okay, curious to see that. :)
15:26:27 <iwakeh> yes, it is even helpful for me.
15:26:35 <iwakeh> while writing.
15:26:54 <iwakeh> that's all from my side.
15:26:58 <karsten> what's your timeline?
15:27:17 <iwakeh> next monday for the next doc.
15:27:31 <karsten> neat!
15:27:38 <iwakeh> with latexdiff
15:27:59 <karsten> alright, moving on?
15:28:03 <iwakeh> sure.
15:28:07 <karsten> - Berlin meeting https://pad.riseup.net/p/SFYYn1aEpkPm  (karsten)
15:28:14 <iwakeh> postpone?
15:28:17 <karsten> so, I wonder if we should postpone this meeting.
15:28:33 <iwakeh> we can, sure.
15:28:48 <iwakeh> There are many other tasks right now.
15:28:49 <karsten> the thing is, I won't have much time to prepare it, if it happens in three weeks from now.
15:29:11 <karsten> I'll be busy next week, be on vacation the week after, and then there's the meeting.
15:29:11 <iwakeh> yes, we should get the things done
15:29:18 <iwakeh> we're currently working on.
15:29:30 <karsten> I mean, we should have this meeting in Q2.
15:29:37 <Samdney> +1
15:29:38 <iwakeh> Sure.
15:29:42 <karsten> for handing over operations things, for example.
15:29:55 <iwakeh> true.
15:29:56 <hiro> +1
15:30:26 <iwakeh> May is the middle of q2.
15:30:40 <karsten> yep.
15:30:58 <karsten> mid-May?
15:31:26 <iwakeh> calendar week 15nish
15:31:27 <karsten> I think I can spend more thoughts on planning this meeting in two weeks from now.
15:31:35 <karsten> ah, 2.5 weeks.
15:31:43 <iwakeh> calendar week 20nish
15:32:10 * karsten doesn't have calendar weeks. ;)
15:32:37 <iwakeh> May 17, Wed
15:32:46 <iwakeh> May 18, Thursday
15:32:58 <iwakeh> I can put the day list on the meeting pad.
15:33:03 <iwakeh> for doodling.
15:33:07 <hiro> yep good idea
15:33:12 <karsten> yes, please do!
15:33:35 <iwakeh> Will do right after this meeting.
15:33:49 <karsten> thanks!
15:33:53 <karsten> moving on?
15:34:10 <karsten> - press request (karsten)
15:34:12 <iwakeh> sure, suddenly those topics were added ;-)
15:34:16 <karsten> tada!
15:34:20 <iwakeh> cool,
15:34:27 <iwakeh> which new outlet?
15:34:32 <iwakeh> which news outlet?
15:34:33 <karsten> so, somehow a press request landed on my feet,
15:34:38 <karsten> and I'm trying to shake that off.
15:34:47 <iwakeh> but it sticks?
15:34:48 <karsten> would somebody else want to take it?
15:34:57 <karsten> no, I just didn't find a direction where to shake it.
15:35:03 <iwakeh> what is it?
15:35:10 <Sebastian> "it depends" :P
15:35:13 <iwakeh> Don't we have a communication person?
15:35:24 <karsten> "... story about the increase in the use of privacy tools since the U.S. election ..."
15:35:31 <hiro> iwakeh: +1
15:35:42 <karsten> the communication person somehow managed to drop it on my feet.
15:35:45 <iwakeh> why did it end up in Germany?
15:36:08 <karsten> oh, just because it had to end somewhere in the metrics team.
15:36:19 <karsten> it's from a U.S. news thing.
15:36:26 <iwakeh> ah
15:36:31 <Samdney> ah,too
15:36:38 <hiro> but what's the story there? they need someone to read the graphs for them?
15:36:46 <iwakeh> hehe
15:36:48 <karsten> pretty much.
15:36:52 <Samdney> :)
15:36:53 <iwakeh> uhm.
15:37:14 <karsten> so, my current plan is to let it sit in my inbox and see if it's still alive in a few days.
15:37:20 <karsten> I could imagine doing something smarter.
15:37:36 <karsten> (maybe off-channel)
15:37:44 <iwakeh> there are all those nice graphs on metrics.tp.o ....
15:38:08 <karsten> heh. I think I have a template for such responses somewhere...
15:38:17 <iwakeh> good :-)
15:38:20 <karsten> - new data source (karsten)
15:38:36 <karsten> that's a new request from a research group.
15:38:47 <karsten> which comes with a paper.
15:38:55 <iwakeh> well, interesting
15:39:01 <Samdney> +1
15:39:01 <karsten> and a suggestion to incorporate something into Tor Metrics.
15:39:17 <iwakeh> what is it?
15:39:32 <Samdney> could I read this paper? :)
15:39:46 <karsten> should I fwd this message to you two?
15:39:51 <iwakeh> fine.
15:39:57 <Samdney> yeap
15:40:03 <karsten> okay!
15:40:18 <karsten> alright, I ran out of topics.
15:40:28 <karsten> anything else for today?
15:40:34 <iwakeh> we covered a lot.
15:40:42 <karsten> yep!
15:41:00 <karsten> okay, let's talk more next week. thanks, everyone!
15:41:12 <hiro> "see" you next week
15:41:19 <Samdney> bye bye!
15:41:22 <karsten> indeed. bye!
15:41:25 <karsten> #endmeeting