14:29:42 <karsten> #startmeeting metrics team
14:29:42 <MeetBot> Meeting started Thu May 11 14:29:42 2017 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:29:42 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:29:44 <karsten> hello!
14:29:53 <karsten> https://pad.riseup.net/p/3M7VyrTVgjlF has the agenda.
14:30:00 <karsten> anything else we want to discuss today?
14:30:06 <hiro> hello
14:30:10 <karsten> hi!
14:30:14 <iwakeh> hi hiro!
14:30:17 <Samdney> hi
14:30:19 <hiro> hi all
14:31:12 <karsten> irl: you around?
14:31:46 <Sebastian> I wanted to ask what happened with the webstats stuff, it seems april 6th was the last time it was mentioned. If no ping is needed sorry for the noise :)
14:32:16 <karsten> argh.
14:32:23 <iwakeh> oh!
14:33:47 <karsten> okay, we have a few topics.
14:33:52 <karsten> shall we start?
14:34:08 <hiro> yes!
14:34:17 <karsten> - webstats migration to CollecTor (Sebastian)
14:34:44 <karsten> is everything in everyone's inboxes, or are there some quick links we could put in the action item here?
14:35:01 <iwakeh> Who's inbox?
14:35:10 <karsten> yours? mine?
14:35:17 <karsten> I don't know.
14:35:22 <iwakeh> Me neither.
14:35:45 <iwakeh> Guess, this should be moved to higher prio?
14:36:11 <karsten> Sebastian: where's the current code?
14:36:35 <Sebastian> https://gitweb.torproject.org/webstats.git/
14:37:01 <Sebastian> it's fine if it takes time, I just wondered because nobody asked me about it.
14:37:10 <Sebastian> getting it completely forgotten would be annoying :)
14:37:17 <karsten> it's buried under many other things.
14:37:19 <karsten> it was.
14:37:23 <iwakeh> Now we can look at it
14:37:27 <karsten> it's not much code, right?
14:37:44 <Sebastian> no
14:37:50 <karsten> the awestats stuff is unused?
14:37:54 <karsten> awstats
14:38:08 <Sebastian> like 140 lines
14:38:10 <Sebastian> yes
14:38:17 <karsten> alright.
14:38:20 <karsten> let's prioritize this.
14:38:23 <iwakeh> fine.
14:38:27 <karsten> who does this?
14:38:46 * iwakeh could do that.
14:39:00 <karsten> sounds great!
14:39:06 <karsten> what about sync?
14:39:12 <karsten> is it easier to leave that out for now?
14:39:18 <iwakeh> I'll see.
14:39:20 <karsten> and just download and sanitize?
14:39:38 <karsten> okay. happy to review code in early stages even.
14:39:42 <iwakeh> and will make some plans with optional functionality.
14:39:43 <Sebastian> I imagine syncing could be easy as it's just one file per day, per target
14:39:57 <Sebastian> but I don't know metrics that well.
14:40:02 <karsten> ah, syncing in the collector sense, as in: sync from other collector hosts.
14:40:13 <iwakeh> We have a sync mechanism in place.
14:40:21 <iwakeh> That will be expanded.
14:40:40 <karsten> okay.
14:40:43 <karsten> thanks for the reminder.
14:41:06 <karsten> - longterm or not at all(?): can metrics benefit from easily integrated document parsers etc that get written during the Rust experiment? (Sebastian)
14:41:17 <karsten> let's talk about that next.
14:41:21 <iwakeh> What kind of experiment?
14:41:36 <karsten> that's the rewrite-tor-in-rust experiment, right?
14:41:42 <Sebastian> Tor is doing an experiment of putting some of its code in Rust
14:41:50 <iwakeh> Oh, I'd suggest Java :)
14:42:21 <iwakeh> Fine, and the result is a parser?
14:42:25 <Sebastian> I was thinking that metrics might benefit from re-using some of the rust code we're writing to parse stuff and put it in a database. Or maybe that's crazytalk. Just an idea.
14:42:26 <iwakeh> First result?
14:42:45 <Sebastian> No, this is just wondering whether while we're fleshing out the Rust stuff we should think about metrics at all.
14:43:01 <karsten> so, we were thinking about improving parsers using a different approach:
14:43:03 <karsten> ANTLR.
14:43:13 <karsten> which is similarly long-term.
14:43:16 <iwakeh> Well, a rusty tor would need to produce the same descriptors.
14:43:35 <Sebastian> I'm mostly thinking that as Tor changes, you always need to keep up by implementing the same thing in Java
14:43:47 <Sebastian> and wouldn't it be neat if you could make use of the code already written for tor
14:43:55 <iwakeh> Not really with ANTLR
14:44:15 <iwakeh> antlr uses grammar for generating parsers in many languages.
14:44:19 <karsten> the question is, would you be able to use ANTLR for generating rust code?
14:44:29 <iwakeh> and, grammar would benefit tor again for
14:44:32 <karsten> to avoid writing that code yourself at all.
14:44:35 <iwakeh> defining the protocol.
14:44:53 <karsten> there was a (short) discussion on tor-dev a while ago.
14:45:00 <Sebastian> that's more a question for the C people at this point I think
14:45:02 <iwakeh> so, an even more unifying approach.
14:45:11 <iwakeh> not really.
14:45:11 <Sebastian> if they specify a grammar for documents, we'd generate the parser of course.
14:45:58 <iwakeh> This is more long term. I think.
14:46:11 <iwakeh> But I'd like to look at the code.
14:46:31 <iwakeh> Curious :-)
14:46:46 <karsten> I agree that this is more long-term.
14:47:21 <karsten> I'd say don't worry too much about metrics here for the moment.
14:47:22 <iwakeh> Where is the code to be found?
14:47:42 <karsten> after all, we already have a quite comprehensive parsing library to work with.
14:47:46 <Sebastian> iwakeh: let's discuss that offline? It's an ongoing project with a couple of branches here and there, hopefully soon in tor.git
14:47:47 <iwakeh> true
14:48:06 <iwakeh> Sebastian: fine.
14:48:07 <karsten> okay, great!
14:48:11 <Sebastian> but ok, will not worry about metrics. Thanks
14:48:19 <karsten> thanks for asking. :)
14:48:20 <iwakeh> Thanks!
14:48:30 * karsten moves irl's topic to the end.
14:48:33 <karsten> - metrics-lib "easy" tickets (Samdney)
14:48:44 <Samdney> ah that's me
14:48:50 <karsten> did you enjoy the two easy tickets I mentioned last week? :)
14:49:01 <karsten> anything you need to move forward?
14:49:12 <Samdney> I'm interested in #21751
14:49:14 <iwakeh> How easy?
14:49:23 <karsten> good question.
14:49:33 <karsten> easy as in: does not need tons of metrics-lib context before starting.
14:49:40 <iwakeh> This is rather invvolved, I'd say.
14:49:43 <Samdney> I have to finish something private at first which need more time like I thought
14:49:57 <Samdney> I hope I will find time at the weekend :)
14:49:59 <karsten> can you come up with an easier one?
14:50:01 <iwakeh> Samdney, how easy?
14:50:13 <iwakeh> Does it need to be metrics-lib?
14:50:24 <karsten> no.
14:50:36 <iwakeh> I'd have something to get started in Java coding the Metrics-Team way.
14:50:45 <Samdney> I only looking for something for getting familiar with the code ... :)
14:50:55 <iwakeh> Sure.
14:51:03 * iwakeh looking for the ticket
14:51:38 <iwakeh> This ticket needs a little more work #20546
14:51:51 <iwakeh> I didn't get around reviewing yet.
14:52:09 <iwakeh> the code is modular and will serve
14:52:20 <iwakeh> future improvements of CollecTor.
14:52:37 <karsten> okay.
14:52:50 <Samdney> Will have a look the next days.
14:52:56 <karsten> great!
14:53:03 <iwakeh> I could mail some more info to you, Samdney?
14:53:15 <Samdney> That would be great :D
14:53:21 <iwakeh> About what is left, or post it to the ticket.
14:53:29 <iwakeh> ok, mail then.
14:53:33 <karsten> cool!
14:53:37 <karsten> moving on?
14:53:45 <iwakeh> fine for me.
14:53:51 <Samdney> iwakeh: do you have my email address?
14:54:13 <iwakeh> I think we were on the same thread somewhere.
14:54:28 <Samdney> ok, else ping me ;)
14:54:39 <iwakeh> sure
14:54:42 <karsten> - metrics-lib release planning (karsten)
14:54:47 <iwakeh> 1.7.0
14:54:55 <karsten> so, I touched almost all metrics-lib tickets in the past 1.5 weeks.
14:55:04 <karsten> yes, 1.7.0 and 2.0.0.
14:55:04 <iwakeh> hehe, noticed.
14:55:25 <karsten> I wonder if we should have another meeting where we go through them all and discuss them more synchronously than via email/trac.
14:55:43 <iwakeh> That might be helpful.
14:56:06 <karsten> how's your friday afternoon?
14:56:21 <iwakeh> Later afternoon ought to be fine.
14:56:41 <karsten> 15:00 UTC? like in 24 hours?
14:56:50 <iwakeh> perfect!
14:57:28 <karsten> great!
14:57:38 <karsten> next:
14:57:40 <karsten> - Wrapping up OnionPerf's onion server stats on Metrics (#22112) (karsten)
14:57:47 <karsten> sorry about the renaming confusion.
14:57:49 <karsten> ;)
14:58:00 <karsten> so, if we finish that ticket, we can invoice for the deliverable.
14:58:15 <karsten> and we'll want to invoice for three remaining deliverables in may and june.
14:58:30 <karsten> maybe we can finish this one by the end of this week?
14:59:42 <karsten> still here?
15:00:10 <iwakeh> yes
15:00:14 <karsten> ah. :)
15:01:25 <iwakeh> I replied to 22112
15:02:08 <iwakeh> mixup, not to comment3 yet.
15:02:31 <karsten> so many tickets..
15:02:50 <iwakeh> Well, those permutations of ones and twos everywhere ...
15:02:56 <karsten> hehe
15:03:15 <iwakeh> I'll take a look after this meeting.
15:03:19 <karsten> perfect!
15:03:25 <karsten> - CollecTor operation (karsten)
15:03:37 <karsten> not sure whether there's much to discuss at this point.
15:03:44 <kiki101> Hi, Tor metrics team. I would like to contribute for the summer. How can I get started?
15:03:51 <karsten> just thought I'll put it in as placeholder.
15:04:11 <iwakeh> I'll try to access and that should be fine; now that I know how to get there ;-)
15:04:21 <karsten> hi kiki101, we have something on the wiki for volunteering. let's see where that page is..
15:04:49 <karsten> https://trac.torproject.org/projects/tor/wiki/org/teams/MetricsTeam/Volunteers
15:04:56 <iwakeh> https://trac.torproject.org/projects/tor/wiki/org/teams/MetricsTeam/Documentation
15:05:09 <iwakeh> for more general questions.
15:05:25 <iwakeh> and guide docs on the bottom of the page.
15:05:48 <iwakeh> Samdney: do you have a trac persona?
15:05:52 <kiki101> Thanks, I'll check them out.
15:05:54 <Samdney> yes
15:06:01 <iwakeh> same as here?
15:06:06 <Samdney> yes
15:06:14 <karsten> kiki101: maybe take a look around, and if you find something, come back next week and we can answer questions?
15:06:35 <kiki101> Yep, I will do that.
15:06:41 <karsten> okay, moving on:
15:06:45 <karsten> - OnionPerf deployment state (hiro)
15:06:52 <hiro> sure.
15:07:00 <hiro> I haven't heard back from irl
15:07:08 <hiro> and I can see his op-ab is timing out
15:07:13 <karsten> ok. :/
15:07:14 <hiro> maybe he is having the same issues I was having
15:07:46 <hiro> but I am not sure since I haven't accesso to his setup at the moment
15:08:16 <hiro> and op-se isn't deployed yet as Ln5 reported in the email to tor-team
15:08:33 <hiro> reagarding op-hk, were you able to clean those old data?
15:08:40 <karsten> ah, yes.
15:08:42 <hiro> ah great
15:08:45 <karsten> found a metrics-lib bug while doing so.
15:08:56 <karsten> but the data on metrics.tpo should be good now.
15:08:57 <hiro> perfect
15:09:22 <karsten> and I briefly talked to rob about adding phantomtrain, but I still owe him a reply.
15:09:38 <hiro> yes regarding the old data
15:09:57 <karsten> so, he said he's fine with keeping the server for testing.
15:10:18 <karsten> but he'd also prefer if we'd add past data up to this point.
15:10:32 <karsten> from last year or so.
15:10:41 <karsten> any reasons against doing that that I'm overlooking?
15:10:59 <iwakeh> should be ok.
15:11:23 <karsten> okay. it feels a bit odd to touch all the tarballs to add new data, but I didn't find compelling reasons not to do it.
15:11:34 <hiro> well I do not know about metrics, but if he was measuring without our setup maybe some of his data would have the same issue that the first data I was generating were having
15:11:48 <iwakeh> good point
15:11:48 <hiro> with "our setup" I mean port 80 forwarding and such
15:11:50 <karsten> I think he used port 8080.
15:11:59 <karsten> without port forwarding.
15:12:04 <iwakeh> Could we keep the data separate?
15:12:15 <hiro> I have no idea how much that could change tho :)
15:12:28 <karsten> separate as in: experimental data?
15:12:43 <karsten> I was thinking about such a thing in the past.
15:12:54 <iwakeh> Well, it is measured in a different manner
15:12:57 <karsten> we did a few experiments with torperf and never put the data anywhere.
15:13:00 <iwakeh> slightly at least.
15:13:16 <karsten> the question here is: would we include the data on tor metrics?
15:13:20 <karsten> probably not.
15:13:22 <iwakeh> that should be reflected/recorded in some way.
15:13:50 <karsten> let's think more about that!
15:13:54 <iwakeh> It will be stored on rob's server anyway?
15:14:08 <karsten> well, the value would to mirror it and also document what it is.
15:14:26 <iwakeh> true; needs more thinking.
15:14:33 <karsten> which makes sense if we add 10 other data sets from 5 different people.
15:14:47 <karsten> yes, but it's an interesting idea.
15:15:05 <iwakeh> Yes, and could be the new data we need nextt year.
15:16:01 <karsten> the goal would be slightly different though.
15:16:12 <karsten> but let's think more about this and talk offline.
15:16:23 <iwakeh> fine.
15:16:30 <karsten> - Atlas 1.0 (irl)
15:16:37 <karsten> no sign of irl, I think?
15:16:57 <karsten> anything else for today?
15:17:03 <iwakeh> I'm fine.
15:17:15 <hiro> is the meeting 30 minutes now?
15:17:27 <iwakeh> originally 45min
15:17:32 <hiro> ah that's perfect then
15:17:34 <hiro> always on time
15:17:37 <hiro> :)
15:17:37 <Samdney> :)
15:17:38 <iwakeh> hehe
15:17:38 <karsten> right now we're at 48 minutes.
15:18:08 <karsten> okay, let's talk more about metrics-lib tomorrow at 15 utc and about all things metrics next week!
15:18:13 <karsten> thanks, everyone! bye, bye!
15:18:16 <karsten> #endmeeting