14:01:57 <karsten> #startmeeting metrics team
14:01:57 <MeetBot> Meeting started Thu Nov 19 14:01:57 2015 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:57 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:02:06 <karsten> hello. who's here for the metrics team meeting?
14:02:11 <yorick> me!
14:02:17 <tomlurge_> me too!
14:02:31 <Letty> me and hi
14:02:40 <karsten> hi, yorick, tomlurge_, Letty!
14:02:49 <karsten> https://pad.riseup.net/p/zUNzEIFRq5S4 <- agenda
14:02:57 <karsten> please add agenda items that you want to discuss today.
14:03:26 <virgil> I exist
14:03:31 <karsten> hi virgil
14:03:57 <flatline> as do I, hello
14:04:20 <qbi> karsten: me
14:04:23 <karsten> hi flatline! (we'll need quick introductions from you and yorick to learn what you're up to. we'll get to that in a minute.)
14:04:27 <karsten> hi qbi!
14:05:12 <karsten> actually, yorick and flatline, want to say a few words what you're interested in regarding tor metrics?
14:05:32 <karsten> (everyone, feel free to add agenda items in the meantime.)
14:05:58 <karsten> yorick, flatline: if you're mostly here to listen, that's fine, too.
14:06:34 <flatline> I am very happy to say a few words about why I'm here
14:06:39 <karsten> great!
14:07:12 <yorick> hi everyone! I came here mostly to ask a question re university project work, I think you guys might have a topic for me
14:07:31 <flatline> I have a fun little hidden service that seemed like it would be neat to show around
14:07:40 <yorick> (possibly not, but it would be cool to be able to help out here and also have a topic for my project)
14:08:28 <karsten> yorick, flatline: great! added to the agenda.
14:08:54 <karsten> okay, let's start.
14:08:57 <karsten> * Vegas team lead meeting at 17:30 UTC; anything to bring up there?
14:09:14 <karsten> so, one remark in berlin was that there's little communication between teams.
14:09:28 <karsten> and that communication may appear unidirectionally.
14:09:35 <karsten> let's fix that. what should I bring up?
14:09:43 <karsten> I'll report back on metrics-team@ later.
14:11:36 <karsten> anything you're happy or unhappy about and that you want other teams to know?
14:11:38 <qbi> Do the other teams have protocols?
14:11:47 <qbi> or some kind of abstract of their last meeting?
14:12:02 <karsten> like StatusBot logs?
14:12:07 <karsten> err, MeetBot
14:12:29 <karsten> or summarized by humans?
14:12:38 <qbi> MeetBot is fine.
14:12:38 <MeetBot> qbi: Error: "is" is not a valid command.
14:12:42 <karsten> (the answer is 'no clue' regardless, I'm trying to figure out what you'd expect.)
14:13:02 <karsten> should I suggest using MeetBot to them?
14:13:14 <qbi> Well, at least for me just reading a log is fine.
14:13:23 <qbi> I wouldn't require anyone to come here and report.
14:13:24 <karsten> well, ask if they're already doing that, and if not, suggest it.
14:13:38 <karsten> okay, but you'd read the log.
14:13:52 <karsten> I guess that's a fine question I should ask.
14:13:53 <qbi> Even an agenda would be good. So one could skim over and ask the other team about single issues.
14:14:04 <karsten> ok.
14:14:12 <karsten> anything else?
14:14:23 <karsten> (I'll write that down on the pad in a minute. thanks!)
14:14:40 <qbi> Information should be pulled, IMHO.
14:14:57 <karsten> hmm?
14:15:13 <qbi> Only if we have questions about some issues we should inite a person to come to "our" meetings and report.
14:15:36 <qbi> But if we just want to know happens in another team I'd suggest to actively look for that information.
14:15:43 <karsten> ok.
14:15:50 <qbi> instead of put the burden on other persons.
14:16:12 <karsten> right, makes sense.
14:16:27 <karsten> turning on MeetBot is not that much of an inconvenience.
14:16:29 <karsten> for most. ;)
14:16:39 <karsten> writing a summary might be.
14:16:45 <karsten> okay, shall we move on?
14:17:10 <karsten> * Analytics server (tomlurge) [max 10m]
14:17:15 <karsten> tomlurge_: want to give an update?
14:17:24 <tomlurge_> yep:
14:17:49 <tomlurge_> status is that I’m almost done with the converters from collector to JSON
14:18:07 <tomlurge_> (if my computer hadn’t crashed last night….)
14:18:24 <karsten> (good thing you had a recent enough backup!)
14:18:41 <tomlurge_> plan is to start ingesting collector data into the analytics server for real coming monday
14:18:53 <karsten> which descriptor types?
14:18:57 <karsten> relays, bridges, others?
14:19:06 <tomlurge_> all of them, practically
14:19:14 <karsten> btw, do you have a link to your github?
14:19:20 <karsten> so that others can follow if they want?
14:19:39 <tomlurge_> github.com/tomlurge/mteam
14:19:56 <karsten> great. and sound like great progress!
14:19:59 <karsten> sounds*
14:20:06 <karsten> anything we can help with?
14:20:19 <tomlurge_> yeah, i was pretty impressed myself :-)
14:20:44 <tomlurge_> not right now. if you have a spare hour beginnig of next week, that could be useful
14:20:54 <karsten> sure, that should work.
14:21:02 <tomlurge_> also I found some missing parts in the metrics-lib library
14:21:05 <karsten> hehe
14:21:07 <karsten> even more?
14:21:12 <karsten> mind filing tickets?
14:21:17 <tomlurge_> yes, another one
14:21:22 <karsten> that's the most convenient way to forg^Whandle them.
14:21:37 <karsten> but more seriously, I'll look into those.
14:21:38 <tomlurge_> i prefer sending you a mail…
14:21:43 <karsten> well, or that.
14:22:04 <tomlurge_> .. there is some more stuff, and a little bug. i’m still collecting...
14:22:06 <karsten> I'm currently jumping from one code base to the other, depending which of them is most on fire.
14:22:14 <karsten> that's good.
14:22:32 <tomlurge_> no fires here, just features, mostly
14:22:35 <karsten> okay, should we talk more early next week and move on here?
14:22:36 <karsten> ok.
14:22:41 <tomlurge_> yep!
14:22:48 <karsten> * Disagreement of directory authorities (Letty) http://letty.io/tor/ [max 10m]
14:23:06 <Letty> Since last meeting, i decide to change the visualization from nested circles into a aster plot (i think that is the name)
14:23:17 <Letty> I hope that improves the the understanding/readability of areas that represent the relay count
14:23:46 <karsten> are you using areas or heights to represent relay counts?
14:24:11 <Letty> In the current version is the flag "Fast" used
14:24:25 <Letty> i use the hight
14:25:01 <karsten> ok. is that because you think it's the right thing to do or because you didn't change that yet?
14:25:09 * karsten isn't sure what's right either.
14:25:19 <virgil> Letty: I like seeing it jiggle
14:25:22 <Letty> i need to find a research paper about this viz, i'm not sure if hight or area is better
14:25:57 <karsten> okay.
14:26:02 <virgil> usually stacked bar chart is the suggested way to show parts of a whole
14:26:17 <karsten> virgil: right, we discussed that as an alternative visualizationt, oo.
14:26:18 <karsten> -t
14:26:26 <qbi> Letty: tbh when I visited your site the meaning of the diagram was not clear to me. Could you add some explanantion?
14:26:47 <karsten> qbi: I was planning to help with writing text once the technical pieces are in place.
14:26:56 <karsten> qbi: but you could help with that if you want. :)
14:27:11 <qbi> First, I need to understand it. :-)
14:27:24 <Letty> i think we have in all viz the same problem. the data is not normal distributed
14:27:30 <karsten> dark green means that a relay got the Fast flag in this case from all authorities,
14:27:37 <Letty> yes
14:27:38 <karsten> lighter green means n-1 authorities.
14:27:45 <karsten> dark brown means 1.
14:27:58 <karsten> left half of the circle -> get into consensus, right half -> not in consensus.
14:28:27 <karsten> qbi: okay, want to help make the initial description that I'll help write better?
14:28:36 <qbi> of course
14:28:48 <karsten> I also want to ask harmony for help with that, because he's just awesome with text.
14:28:53 <karsten> cool.
14:29:06 <karsten> Letty: so, what can we help with?
14:29:17 <karsten> besides discussing alternative visualizations. ;)
14:29:43 <karsten> and if you need more time to discuss things, I'm happy to schedule another time for that, too.
14:30:07 <Letty> i need an idea what kind of UI element i should use for changing the flags ;)
14:30:22 <karsten> ah, ok.
14:30:41 <karsten> (anyone? /me is still thinking)
14:31:01 <ioerror> words?
14:31:16 <Letty> karsten: and i need something like a git. is it a good idea to have it on my normal github account (not private)?
14:31:19 <ioerror> asn: dgoulet - would be happy to talk about the shared random thing i just emailed about
14:31:30 <karsten> Letty: yes, github is just fine.
14:31:33 <Letty> ok
14:31:57 <karsten> so, something like this: http://getbootstrap.com/components/#nav-pills
14:32:06 <karsten> or tabs? is that what you're asking?
14:32:15 <karsten> (tabs are above pills)
14:32:32 <karsten> actually, I think tomlurge_ would be a good person to talk to.
14:32:43 <karsten> ioerror: and yes, words would probably work better than icons.
14:33:32 <karsten> Letty, tomlurge_: want to talk about this after the meeting, maybe?
14:33:49 <tomlurge_> karsten, letty: sure!
14:34:06 <Letty> maybe words are the best thing, everthing else is like the 90's..
14:34:07 <karsten> perfect! (assuming that Letty is also fine with that idea.)
14:34:21 <Letty> tomlurge, karsten: yes!
14:34:24 <karsten> great!
14:34:30 <karsten> okay, let's move on.
14:34:43 <karsten> Letty: ping me if you need anything else, like new data.
14:34:48 <karsten> * Roster update (virgil, Sean) [max 5m -- if there's not much to update]
14:34:54 <karsten> -- Just working on with the BGP graph
14:35:02 <virgil> not much to say.  Just working on the BGP graph.
14:35:03 <virgil> aye
14:35:08 <virgil> working on the diversity measurs
14:35:13 <virgil> diversity measure*
14:35:18 <virgil> and Sean has been busy with linear algebra
14:35:33 <virgil> #undergradlife
14:35:49 <karsten> ok. I saw threads with juris about things not working as they're supposed to, and then getting fixed.
14:36:00 <virgil> yeah various bugfixes
14:36:09 <virgil> most of that seemed to be due to Onionoo issues
14:36:13 <karsten> oh...
14:36:18 <virgil> ha ;P
14:36:21 <karsten> yeah, it's rather unhappy these days.
14:36:28 <virgil> it's cool.  It'll get fixed.
14:36:37 <virgil> it's on my queue
14:36:53 <karsten> ah, then let me briefly explain the plan for fixing it:
14:36:56 <karsten> (well, my plan)
14:37:21 <virgil> sure
14:37:21 <karsten> put all documents in the out/ directory into a psql database, replicate that to front-end hosts, run 2 or 3 of them to handle the load.
14:37:32 <karsten> that shouldn't be terribly hard to implement.
14:37:44 <karsten> because there's already something like a database abstraction layer in onionoo.
14:37:57 <karsten> but I'm saying such things about other coding projects, too, and they're never true.
14:38:04 <karsten> so, should be easy. :)
14:38:35 <karsten> ok.
14:38:55 <karsten> if you come up with other good ideas, happy to consider them, too.
14:39:21 <karsten> should we move on?
14:39:30 <virgil> move on
14:39:40 <karsten> ok.
14:39:44 <karsten> * possible "data mining" projects for university project (yorick) [max 10m]
14:39:54 <karsten> yorick: what do you have in mind, roughly?
14:40:03 <yorick> I am a second-year university-level computer science student, doing a "data mining" course.
14:40:04 <karsten> yorick: and do you know the various things that already exist in the metrics space?
14:40:12 <yorick> This course involves a project (about 60 hours of work split over two people, mostly involving applying algorithms to a dataset and hoping anything interesting shows up) and I am still looking for a good subject
14:40:20 <yorick> some guy suggested I ask here
14:41:14 <yorick> karsten: I'm not all too familiar with the tor-metrics space, but you seem to have some interesting data available
14:41:17 <asn> ioerror: see #tor-project please
14:41:20 <karsten> https://lists.torproject.org/pipermail/metrics-team/2015-November/000002.html
14:41:26 <karsten> https://people.torproject.org/~karsten/volatile/metrics-data-visualizations.pdf
14:41:37 <tomlurge_> oha. couldn’t that be a good use of the analytics server?
14:41:54 <karsten> yorick: I wonder if page 2 in that pdf would be something you'd want to do.
14:41:57 <tomlurge_> yorick, do you have to use a specific programming language?
14:42:00 <karsten> yorick: possibly with the analytics server.
14:42:37 <karsten> though, tomlurge_, it's not ready yet.
14:42:55 <karsten> and maybe this project should start with some raw data.
14:43:03 <karsten> not with everything nicely put into a database.
14:43:06 <tomlurge_> that would be the next question: what’s the timeline
14:43:09 <karsten> (/me is mostly guessing here.)
14:43:24 <yorick> tomlurge_: no specific programming language. timeline is "before jan 20th"
14:43:33 <tomlurge_> well, if it’s about algorithms…
14:44:06 <karsten> yorick: so, assuming this recommended versions thing is remotely interesting to you,
14:44:24 <karsten> I could write you a short introduction to what data and tools you would need.
14:45:27 <karsten> tomlurge_: I think the analytics server is too much for 60 hours for 2 people.
14:45:45 <tomlurge_> too much in what sense?
14:45:49 <karsten> yorick: want to think about it and drop me an email?
14:46:06 <yorick> karsten: sure! data viz isn't quite in the scope, but it can still work
14:46:06 <karsten> tomlurge_: I think you'd need those 60 hours to get started, but then you don't have any useful results yet.
14:46:21 <karsten> yorick: well, it doesn't have to do fancy visualization parts.
14:46:36 <karsten> yorick: it could just provide data for a visualization like that, and use something simple to present them.
14:46:43 <karsten> maybe even just a table.
14:47:06 <yorick> karsten: just counting versions does not sound very challenging
14:47:30 <karsten> I think it gets more interesting when you look how outdated relays are,
14:47:39 <karsten> and what happens when new releases come out, etc.
14:47:49 <karsten> I don't know what excitements you'll encounter while digging.
14:48:20 <karsten> the reason why I'm suggesting it is that the data is not that hard to understand, so that there's little time you need to spend on getting started.
14:48:22 <tomlurge_> the problem with the analytics server is that it’s not finished yet. but when it’s finished it’s whole purpose is that you code algorithms and let them run on it. so it seems like a good fit to me
14:48:44 <karsten> tomlurge_: okay, let's keep that in mind as an option.
14:48:57 <karsten> but, we should move on.
14:48:58 <yorick> karsten: okay, I'll send you that email
14:49:03 <karsten> yorick: great!
14:49:06 <yorick> :)
14:49:11 <karsten> * fun little hidden service (flatline) [max 10m]
14:49:16 <karsten> what's that about?
14:49:44 <flatline> it's about allowing for contact with hidden services
14:50:01 <flatline> while maintaining their anonymity and deniability
14:50:28 <karsten> okay, explain more? (also, how is this related to tor metrics?)
14:50:59 <flatline> implementation is a dead drop encrypted with the permanent key from the hidden service descriptor
14:51:37 <virgil> hidden service operators*
14:52:05 <flatline> and I'm not sure it's directly and immediately related to tor metrics
14:52:08 <virgil> sounds more relevant to network team than metrics team
14:52:13 <virgil> but still cool
14:52:16 <karsten> yep.
14:52:27 <virgil> it's something I've actuallly wanted with ONionLink
14:53:00 <karsten> how about you bring this up again after the meeting, highlighting the usual suspect hidden service developers?
14:53:27 <flatline> that sounds reasonable
14:53:32 <karsten> ok!
14:53:39 <karsten> we just ran out of agenda items.
14:53:46 <karsten> anything else we should discuss in the remaining 5 minutes?
14:54:49 <karsten> ok. no need to wait the full 5 minutes to find out.
14:54:54 <karsten> thanks, everyone, for attending!
14:55:06 <karsten> feel free to bring up questions and issues on metrics-team@.
14:55:11 <karsten> next meeting in two weeks. cheers!
14:55:12 <Letty> thanks for feedback
14:55:21 <karsten> thanks for working on this, Letty!
14:55:24 <karsten> #endmeeting