14:00:02 <karsten> #startmeeting metrics team
14:00:02 <MeetBot> Meeting started Thu Jul  7 14:00:02 2016 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:02 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:00:10 <karsten> hello. who's here for the metrics team meeting?
14:00:27 <irl> hello
14:00:44 <karsten> hi irl!
14:01:01 * karsten adds an atlas topic to https://pad.riseup.net/p/zUNzEIFRq5S4
14:01:07 <irl> (:
14:01:14 <oma> hi everybody
14:01:19 <karsten> hi oma!
14:01:19 <irl> on phone, will be back at desk in 10
14:01:25 <irl> put me at the end (:
14:01:29 <karsten> will do
14:02:37 <karsten> let's collect more agenda items until :05.
14:02:54 <karsten> also, we're hoping to finish this (now weekly) meeting by :30 or :45 at the latest.
14:03:21 <irl> cool
14:03:44 <qbi> ln5: I made the change to tor-meeting@ and hope arma2 is fine with it. ;)
14:05:12 <karsten> alright!
14:05:17 <karsten> * metrics-lib release status (karsten)
14:05:23 <karsten> https://trac.torproject.org/projects/tor/ticket/19398#comment:13
14:05:50 <oma> looks good :-)
14:05:54 <karsten> I expect the release to happen either later today or, more likely, tomorrow.
14:06:07 <oma> cool! thanks!
14:06:24 <karsten> but I don't expect any changes to the pre-release tarball at this point, so feel free to use that.
14:06:42 <oma> no hurry
14:06:59 <karsten> okay, any other questions to metrics-lib?
14:07:12 <oma> no, that’s all
14:07:17 <karsten> hi iwakeh! https://pad.riseup.net/p/zUNzEIFRq5S4 <- agenda pad
14:07:17 <iwakeh> hi
14:07:22 <karsten> okay. moving on.
14:07:26 <karsten> * bridge descriptors conversion status (karsten)
14:07:28 <karsten> not good.
14:07:30 <iwakeh> tor is blocked here,
14:07:57 <iwakeh> i had trouble connecting.
14:08:00 <karsten> I converted tarballs year by year, and 2015 took 5 days until I realized it (silently) ran out of inodes.
14:08:05 <karsten> sorry to hear, iwakeh!
14:08:38 <karsten> I now upgraded the internal disk from 500G to 1T and ordered an external 3T drive for the conversion.
14:08:57 <karsten> it'll be another week at least until bridge descriptors are processed.
14:09:06 <oma> karsten: I’m not in a hurry, just wanted to make sure I don’t miss it. sounds like thsi will take at least til next week
14:09:08 <oma> ok
14:09:13 <karsten> at least, yes.
14:09:15 <karsten> okay.
14:09:22 <karsten> moving on.
14:09:27 <oma> okay, no further questions
14:09:28 <karsten> * logging for metrics-lib (iwakeh)
14:09:32 <karsten> oma: ok!
14:09:57 <iwakeh> ok,
14:10:02 <karsten> (reminder: trying to finish until :30, or :45 at the very latest)
14:10:04 <iwakeh> oma mentioned the troubles
14:10:20 <iwakeh> with metrics-lib command line output.
14:10:30 <iwakeh> new issue #19643
14:10:40 <karsten> cool!
14:10:48 <karsten> happy to prioritize that.
14:10:48 <iwakeh> there  are only 4 printStackTrace and
14:10:52 <karsten> for 1.4.0.
14:10:53 <iwakeh> one System.err
14:11:04 <iwakeh> that should be done quickly.
14:11:09 <karsten> heh, yes.
14:11:12 <iwakeh> yes.
14:11:19 <karsten> well, the harder part will be to write useful logging statements.
14:11:25 <iwakeh> 1.4.0 is fine.
14:11:34 <iwakeh> oh well
14:11:40 <oma> nice! will be a real incentive to polish logging in converTor
14:11:58 <iwakeh> @oma: yes :-)
14:12:06 <karsten> (oma: did you know that 1.3.0 has zero (0) checkstyle issues? ;))
14:12:19 <iwakeh> and solves the tunneld-*
14:12:36 <oma> karsten: I saw that today, and tried checkstyle at home. horrifying…
14:12:41 <karsten> :D
14:12:47 <karsten> okay, anything else re: logging?
14:13:02 <iwakeh> no.
14:13:08 <karsten> ok.
14:13:09 <karsten> * naming of ^^^ (iwakeh)
14:13:22 <karsten> tough topic!
14:13:31 <iwakeh> well, i made a suggestion.
14:13:47 <iwakeh> LecTor in #19616
14:14:13 <karsten> nice.
14:14:28 <iwakeh> Looking at the definition by Merriam Webster:
14:14:35 <iwakeh> Lector , n. (Eccl.) A reader of lections; formerly, a person designated to read lessons to the illiterate.
14:14:42 <iwakeh> For the $java-tor-descriptor-parsing-library:
14:14:49 <iwakeh> LecTor helps those that would otherwise not be able to read/process descriptors (using java).
14:15:04 <iwakeh> That would be something for release 2.0.0
14:15:19 <karsten> so, for the 1.3.0 release announcement I thought about doing the same I did for 1.1.0.
14:15:19 <oma> I fear the connotation "Hannibal Lector"
14:15:47 <iwakeh> @oma: No idea?
14:16:16 <karsten> ah, not 1.1.0, but 1.0.0.
14:16:19 <karsten> metrics-lib, which also goes by the name DescripTor, is a Java API ...
14:16:21 <oma> A better idea? No, and I doubt I’ll have one, but I’ll think about
14:16:31 <oma> it
14:16:37 <karsten> I like the suggestion to do the renaming for 2.0.0.
14:16:54 <iwakeh> good :-)
14:17:22 <oma> MeTeor :-)
14:17:22 <karsten> let's discuss more on the ticket.
14:17:24 <iwakeh> Let's wait for atagar's opinion?
14:17:29 <karsten> yep!
14:17:47 <karsten> okay, moving on?
14:17:55 <iwakeh> ok.
14:18:00 <karsten> * converTor code review (oma / question to iwakeh)
14:18:20 <iwakeh> tests are very important :-)
14:18:23 <oma> just wanted to say that I worked through all the issues iwakeh raised
14:18:32 <oma> .. argh .. yes!!!
14:18:35 <karsten> heh
14:18:50 <oma> but not now … still prototyping ;-)
14:19:10 <iwakeh> well, exactly for prototyping.
14:19:12 <oma> no, honestly: I’ll probably start on testing soon
14:19:26 <oma> but only to get an idea
14:19:28 <iwakeh> I made some smaller suggestions where to start.
14:19:41 <oma> then I’ll concentrate on the analytics part
14:19:42 <iwakeh> these tests will be useful when putting the chain together.
14:20:15 <oma> when that’s proving successful and the whole idea seems usable, then I’ll do real testing. that’s my plan
14:20:50 <iwakeh> Well, you could write a test each time you run into a problem setting up analytics?
14:21:13 <iwakeh> Testing is not decoration, it's part of programming.
14:21:20 <oma> yeah, but thats not converTor. I’m talking about converTor here
14:21:33 <karsten> huh.
14:21:36 <iwakeh> hu!
14:21:42 <iwakeh> ?
14:22:02 <oma> hu?
14:22:05 <karsten> 14:21:23 < oma> yeah, but thats not converTor. I’m talking about converTor here
14:22:10 <karsten> converTor or converTor?
14:22:27 <oma> ah.
14:22:59 <oma> i will try to work with tests when developing the analytics part, but for teh converTot it’s postponed
14:23:17 <karsten> ah.
14:23:26 <oma> the converTor is really just the tool that converts from collecTor to JSON, Parquet,Avro
14:23:27 <iwakeh> You do not know that convertor produces the right input.
14:23:57 <iwakeh> if that is somewhat falty, the analytics part has a major problem.
14:24:03 <oma> iwakeh: yes, but it’s good enough as ddummy input for developing analytics scripts
14:24:33 <oma> nothing of this is production ready yet
14:24:36 <iwakeh> even the structure could have a bug here and there.
14:24:43 <oma> yes it could
14:24:55 <iwakeh> How do you currently verify the converted output?
14:25:32 <oma> it just isn’t that importnat right now and I have a motivation problem if I spend another month or 2 with that damn tool without being able to actually play with the data
14:25:33 <karsten> I wonder, would you be more amenable to testing if metrics-lib supported that better?
14:25:39 <oma> by looking at it :)
14:25:50 <iwakeh> the logging.
14:26:04 <karsten> well, I was thinking of generating descriptors.
14:26:12 <iwakeh> oh, ok.
14:26:13 <karsten> which is something that other tools would benefit from, too.
14:26:21 <iwakeh> yes, that's true.
14:26:34 <oma> generated descriptors would be very helpful
14:26:37 <karsten> and we already have some code to generate descriptors, in metrics-lib's own tests.
14:26:55 <karsten> we might consider making that code available to applications.
14:27:17 <oma> still, I don’t understand the problem here
14:27:40 <karsten> the problem of not testing code?
14:27:50 <iwakeh> well, what do we want to find out building the prototype?
14:28:36 <oma> if SPark is reasonably easy to use, if it performs
14:28:40 <iwakeh> from that we can maybe better derive the extend of work to be done?
14:28:50 <oma> if the whole idea makes sense or is just fluff
14:29:13 <oma> and we can do that very well with buggy data as long as it’s not malformed
14:29:24 <karsten> ok.
14:29:35 <karsten> I think we need to move on.
14:29:35 <iwakeh> so, just a spark review sort of?
14:29:42 <karsten> we can revisit this next week.
14:29:48 <iwakeh> ok.
14:29:56 <karsten> oma: ok?
14:30:13 <oma> no, not only spark. that’S my favored tool right now, but also Drill and Hadoop and maybe others.
14:30:21 <oma> yes, let’s revisit
14:30:26 <karsten> ok :)
14:30:27 <karsten> * Atlas improvements (irl)
14:30:30 <karsten> irl: still around?
14:30:32 <irl> yes
14:30:38 <karsten> cool. how's this going?
14:30:41 <irl> so we shut down globe
14:30:44 <karsten> yep
14:30:58 <irl> my next steps are working around #6787
14:31:27 <irl> which i've kind of expanded into a "make it more friendly" bug incorporating also #10883 #9768
14:31:42 <irl> these will all probably end up as a single branch to be merged
14:31:44 <karsten> I saw that you received a lot of feedback on the mailing lists.
14:32:04 <irl> i've not yet had a chance to respond on the mailing lists, but i've been working through the feedback
14:32:26 <irl> some things were already filed as bugs
14:32:36 <oma> I hope I didn’t raise too much points :-)
14:32:37 <irl> others were common sense fixes that i've already applied locally
14:33:08 <irl> for time context, i am presenting at a workshop next week in berlin, along with an EU project plenary meeting
14:33:11 <karsten> did you talk to phw about how to best merge those changes?
14:33:19 <irl> so nothing will really happen next week
14:33:36 <irl> i have not spoken to phw about this yet, but i think i'm the only active person on the code
14:33:51 <irl> i shouldn't be clobbering anyone elses merges and creating conflicts afaik
14:33:56 <karsten> true!
14:34:26 <irl> following the merge of this, the big top priority next thing will be #19452
14:34:30 <karsten> what I meant is that he might have preferences regarding reviewing many branches or a single branch, many commits or one big commit, and so on.
14:34:49 <karsten> okay, happy to help with #19452.
14:35:03 <irl> ah ok, i will email phw when i'm back from berlin
14:35:10 <karsten> sounds good!
14:35:15 <irl> i'm not sure if onionoo will need changes for 19452
14:35:23 <karsten> so, you're not blocking on anyone?
14:35:28 <irl> i think it already does what is needed, just we don't use the api well enough
14:35:35 <irl> i do not believe i am currently blocking
14:35:36 <karsten> sure, but I'm happy to help with creating good onionoo requests.
14:35:47 <irl> if i am, i won't know it for 2 weeks though and i'll bring it to the meeting
14:36:09 <karsten> okay, good. it's great to see that you're picking up the atlas renovation!
14:36:29 <irl> https://irl.github.io/atlas/ - current-ish progress for those interested
14:37:15 <irl> that's i think all i have for progress on that for now
14:38:06 <karsten> here's how I redesigned exonerator a few months (or years...) back:
14:38:39 <karsten> I created a single html page with all possible pages containing sample data, and I included comments why I put those things on the pages.
14:38:40 <iwakeh> @irl: looks cool!
14:38:49 <karsten> that was a single document that people could look at and comment on.
14:39:18 <irl> this sounds like a good approach to take
14:39:19 <karsten> hmm, maybe I can find it..
14:39:55 <karsten> https://people.torproject.org/~karsten/volatile/exonerator-mockup/
14:40:24 <karsten> okay, looks like I lied about the comments. maybe that was for another mockup. but comments would be useful. :)
14:40:38 <iwakeh> mouseovers?
14:40:52 <karsten> sure.
14:41:17 <karsten> okay, just a thought.
14:41:44 <karsten> aaand, there's another agenda item coming..
14:41:50 <karsten> * python style guide (iwakeh)
14:41:52 <iwakeh> sorry :-)
14:41:54 <karsten> heh
14:41:58 <karsten> 3 minutes left
14:42:06 <iwakeh> well, there was a question by anathema
14:42:24 <iwakeh> and I thing using the PEP8 is industry standard there.
14:42:25 <karsten> yep. I'm not a python coder, so I can't really comment on that.
14:42:36 <irl> pep8
14:42:38 <karsten> I believe so, too.
14:42:45 <iwakeh> well, maybe take a poll on our list and
14:42:53 <irl> nooo, pep8
14:43:40 <iwakeh> then finalize on the guide doc.
14:44:06 <karsten> sounds good.
14:44:19 <karsten> if a poll is too much, suggest pep8, and if nobody shouts, it's pep8.
14:44:30 <iwakeh> good idea.
14:44:36 <irl> this is something it's a little too easy for people to have opinions on
14:44:37 <karsten> thanks for extending the documentation stuff!
14:44:39 <irl> be careful
14:44:43 <karsten> hah
14:44:55 <iwakeh> you're right, irl.
14:45:13 <iwakeh> https://www.python.org/dev/peps/pep-0008/
14:45:20 <iwakeh> it's PEP8
14:45:45 <karsten> yay!
14:45:53 <iwakeh> ;-)
14:45:54 <irl> (:
14:45:54 <karsten> okay, we're past the 45 minutes.
14:46:08 <iwakeh> all mentioned.
14:46:10 <oma> i forgot something:
14:46:15 <karsten> yes?
14:46:36 <oma> i wanted to ask you, iwakeh, if you know when you’ll continue with the code review?
14:47:06 <oma> (curiosity killed the cat)
14:47:09 <iwakeh> well, how much code review is necessary?
14:47:16 <iwakeh> I mean what questions
14:47:36 <iwakeh> do you have given that it is a very early prototype?
14:47:54 <oma> I was asking about architecture etc and you said there are still issues coming
14:47:55 <iwakeh> I can look further for things like the logging etc.
14:48:20 <iwakeh> Well, architecture is also testability (sorry ;-) and maintainability and ...
14:48:44 <oma> okay, so nothing in the pipeline right now.
14:48:45 <iwakeh> a lot more that doesn't really apply to quick prototyping.
14:48:49 <oma> (relaxing…)
14:49:01 <iwakeh> That doesn't mean things are fine ;-)
14:49:31 <iwakeh> But I need an objective for a review.
14:49:49 <iwakeh> otherwise I keep harping on tests ...
14:49:54 <karsten> heh
14:50:03 <karsten> okay, I guess that answers the question?
14:50:06 <iwakeh> and soon other things you don't want to do now.
14:50:26 <oma> yes, it does
14:50:33 <karsten> cool. let's end this meeting now.
14:50:34 <iwakeh> ok.
14:50:37 <karsten> we'll have another one in a week!
14:50:40 <karsten> #endmeeting