14:59:48 <karsten> #startmeeting metrics team
14:59:48 <MeetBot> Meeting started Thu Feb 23 14:59:48 2017 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:59:48 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:00:06 <karsten> https://pad.riseup.net/p/3M7VyrTVgjlF <- agenda pad
15:00:15 <karsten> please add anything you want to discuss today.
15:00:37 <iwakeh> all seems to be there.
15:00:48 <karsten> cool!
15:00:54 <hiro> yep
15:01:12 <karsten> let's start then.
15:01:15 <karsten> - Atlas usability project blog post (irl)
15:01:20 <irl_work> hello
15:01:24 <karsten> hi irl_work!
15:01:28 <irl_work> http://paste.debian.net/plainh/293bf266 is the draft text i've got so far
15:02:07 <karsten> you could also ask for contributions/patches.
15:02:18 <irl_work> good point
15:02:30 <irl_work> i'll make a wiki page "how to contribute to atlas" and link to that
15:02:34 <karsten> minor tweak: there won't be a blog post for the onionoo release, just a tor-dev@ post.
15:02:42 <irl_work> ah ok, i'll link to that then
15:02:43 <karsten> sounds great!
15:02:46 <karsten> yep.
15:03:01 <irl_work> (:
15:03:05 <karsten> maybe include a screenshot?
15:03:20 <karsten> not sure how useful that is, but people like visual things.
15:03:26 <irl_work> yep, had planned this, but couldn't put it in the pastebin
15:03:29 <karsten> ok.
15:03:41 <irl_work> i'll find one with cool graphs
15:03:46 <irl_work> lots of colours to keep people interested
15:03:52 <karsten> hehe
15:03:54 <iwakeh> :-)
15:04:11 <karsten> I'm not sure if parts of the post are too technical for the average blog reader.
15:04:34 <irl_work> i think we can drop any mention of JSON
15:04:35 <karsten> I wonder if our new communications person will suggest to simplify that.
15:04:44 <karsten> yes, for example.
15:04:47 <iwakeh> let's test
15:05:06 <iwakeh> It is a good thing to mention that js just fetches data.
15:05:06 <karsten> should I make an introduction to the new communications person via email?
15:05:19 <karsten> yes, explaining that concept would be useful.
15:05:28 <irl_work> i have no idea how to blog, so that would be useful
15:05:34 <karsten> just how that works is beyond what most people would want to know.
15:05:41 <irl_work> i mean, i can write text, but i don't know what the procedure is for publishing it
15:05:44 <karsten> oh, and you'll need an account, right?
15:05:49 <irl_work> i have an ldap one
15:05:51 <irl_work> i guess it's not that
15:05:56 <karsten> no, that's different.
15:06:03 <irl_work> ah ok, so yes, i'll need an account
15:06:11 <karsten> but I'll try to find the process for creating accounts.
15:07:22 <irl_work> ok cool
15:07:30 <irl_work> i think that's all in the agenda pad
15:07:50 <karsten> and we're planning to publish on monday, march .... errr
15:07:55 <karsten> 6
15:07:57 <karsten> right?
15:08:04 <irl_work> sounds great to me
15:08:30 <karsten> okay.
15:08:53 <karsten> that means we can discuss the next draft in a week from now.
15:08:59 <irl_work> yep
15:09:22 <karsten> great! we have a plan. :)
15:09:35 <karsten> thanks for working on this!
15:09:47 <karsten> should we move on?
15:10:03 <irl_work> yep
15:10:10 <karsten> - CollecTor, Onionoo, and metrics-lib websites (RaBe)
15:10:27 <karsten> lots of things have happened there in the past days.
15:10:53 <karsten> RaBe: want to give an update, or should I say more?
15:10:56 <iwakeh> three to one, again.
15:11:26 <karsten> heh, you mean the suggestion. yes, maybe. :)
15:11:32 <RaBe> well, so the collector page and onionoo are already online :)
15:11:34 <iwakeh> which is fine.
15:11:43 <karsten> online, not merged yet.
15:12:00 <karsten> but yes. they look pretty much done.
15:12:03 <karsten> (yay!)
15:12:26 <RaBe> you'll just merge the one pagers as separate pages somewhere to the metrics page...
15:12:46 <karsten> well, I'd like to discuss whether we should do that.
15:12:59 <karsten> but just in case we do it, merging will be pretty simple.
15:13:13 <irl_work> does this mean merging the docs pages?
15:13:25 <iwakeh> merge the web-sites.
15:13:25 <irl_work> and then redirecting the homepages of those domains to metrics.tpo?
15:13:41 <karsten> should I summarize that plan here?
15:13:45 <iwakeh> yes.
15:13:47 <karsten> I only sent it to a few folks via email.
15:14:27 <karsten> okay, we have the CollecTor website, the Onionoo website, and soon a metrics-lib website.
15:14:43 <karsten> which describe the CollecTor service, the Onionoo service, and the metrics-lib library.
15:15:06 <karsten> my suggestion was to move the websites with format specifications to the Tor Metrics website and only keep the services at their current location.
15:15:21 <karsten> so that people would only read about things on https://metrics.torproject.org/xyz
15:15:35 <karsten> we could list available mirrors on those pages.
15:16:15 <karsten> and we could generate our own CollecTor directory listing on Tor Metrics. basically, users would never see something else than https://metrics.torproject.org/ in their browser when reading about stuff.
15:16:22 <karsten> but again, services would remain at their current locations.
15:16:28 <iwakeh> If the mirrors are in tp.o there could just be internal load balancing.
15:16:40 <karsten> yep, good point!
15:17:00 <irl_work> i'll be upset if there are extra keystrokes involved in viewing the onionoo docs, so we need redirections
15:17:18 <iwakeh> sure.
15:17:29 <irl_work> i guess this also means the services are no longer self-documenting, a mirror of the service is only the service and points to tpo for docs
15:17:31 <karsten> do you have existing links in mind?
15:17:51 <karsten> right.
15:18:16 <karsten> there would be such a pointer.
15:18:39 <iwakeh> One thing that needs to be considered are mirrors outside tp.o
15:18:54 <irl_work> if there's different versions of a service running, they would have different docs
15:19:16 <karsten> irl_work: yes, I thought about that, too. I could imagine adding something like "since 3.2" to the docs.
15:19:18 <irl_work> my worry here is removing the documentation from the service, will lead to inconsistent documentation to what actually happens in the service
15:19:25 <iwakeh> If someone just wants to use collector.mirror.org and not be redirected to collector.tp.o
15:20:04 <karsten> regarding the specification, maybe we'll need to keep deprecated items in the spec.
15:20:09 <iwakeh> Right, we intended to put software version on the mirror's web-pages.
15:20:20 <irl_work> what about crazy undocumented features?
15:20:28 <karsten> iwakeh: but they can use collector.mirror.org without being redirected.
15:20:35 <irl_work> for example, if i do pathspider measurements against relays then want to do a demo with that data from onionoo
15:20:36 <karsten> just not for the documentation.
15:20:41 <irl_work> and display it in atlas
15:20:45 <iwakeh> #21414
15:21:13 <iwakeh> if a mirror has an older sw-version running.
15:21:47 <irl_work> adding versioning info for the docs fixes the old version problem, but not the crazy dev features problem
15:21:58 <karsten> what problem is that?
15:22:20 <karsten> you think this will be too crazy?
15:22:22 <iwakeh> well, if I know a mirror operator always runs on master I might take that into account.
15:23:02 <irl_work> i'd like to be able to self-host volatile docs
15:23:03 <iwakeh> master instead of an official release.
15:23:22 <irl_work> not even master, a branch with features that are probably not useful to anyone else
15:23:28 <iwakeh> ok.
15:23:38 <irl_work> i think as long as that option doesn't go away, it's fine
15:23:55 <iwakeh> if your mirror doesn't redirect,
15:24:03 <iwakeh> it'll display your docs.
15:24:10 <karsten> right.
15:24:18 <irl_work> is the metrics website webwml?
15:24:26 <karsten> just html.
15:24:30 <karsten> ah, metrics.
15:24:34 <iwakeh> jsp
15:24:37 <irl_work> oh ok
15:24:38 <karsten> right.
15:24:49 <hiro> yes java :D
15:24:52 <irl_work> in debian we have part of the build process which pulls the latest docs from places and merges it
15:25:09 <irl_work> so the docs are still maintained where they should be, but then the relevant bits get copied in
15:25:17 <irl_work> can we do something like that? to avoid inconsistency
15:25:33 <iwakeh> for collector and onionoo that's the case.
15:25:58 <karsten> so in this case metrics would pull from onionoo?
15:26:04 <iwakeh> web pages are in the release, or did I misunderstand the concern?
15:26:26 <karsten> or what's the suggestion, irl_work?
15:26:38 <irl_work> my concern was that if the web pages weren't maintained with the service, then we get inconsistencies
15:27:01 <irl_work> if we have the tpo onionoo services pointing to metrics.tpo then that's fine, but other people running the service should be able to make changes
15:27:08 <irl_work> and then have a modified copy of the docs
15:27:17 <irl_work> if they add their own extensions that are not anywhere near master
15:28:02 <karsten> okay, I think we can do something like this.
15:28:09 <karsten> this issue didn't come to mind before.
15:28:10 <irl_work> i think that what is being proposed is fine, we're not splitting the codebase/docs up and redirects are optional for each deployment
15:28:31 <iwakeh> we can only redirect for tp.o services.
15:28:40 <iwakeh> and there we have to make sure the
15:28:47 <iwakeh> documentation is correct.
15:28:53 <irl_work> yep, in which case, i have no objections
15:29:01 <karsten> I mean, it might work that we keep the docs in the onionoo repo.
15:29:07 <karsten> and metrics fetches those for its own website.
15:29:16 <irl_work> karsten: yep, that's the debian method
15:29:25 <karsten> and mirror operators disable the short redirect page and include the (modified) spec on their own service.
15:29:54 <iwakeh> could that inclusion be even done on the web-site, not the repo?
15:30:23 <karsten> you mean the metrics website periodically fetches and mirrors the onionoo spec from onionoo.tpo?
15:30:30 <iwakeh> (but that's technical detail)
15:30:33 <karsten> right.
15:30:41 <karsten> all good things to discuss via email or on trac.
15:30:43 <iwakeh> no, in the old days 'frames'
15:30:56 <karsten> ah. ugh.
15:31:07 <iwakeh> hehe, better trac.
15:31:08 <irl_work> please no frames
15:31:20 <iwakeh> no,no, just referring to the type.
15:31:20 <karsten> okay, I'll open a trac ticket for discussion, okay?
15:31:27 <irl_work> do we have a means of detecting of onionoo.tpo was accessed via hidden service so we can redirect to hidden service?
15:31:43 <karsten> oh, another good point.
15:32:04 <iwakeh> it seems the unification needs more thinking.
15:32:09 <karsten> yes :)
15:32:19 <irl_work> the hidden service one is one i need to fix at work, so let me know if you come up with something (:
15:32:19 <karsten> we could provide two links, one direct one via onion service.
15:32:39 <karsten> okay.
15:32:51 <irl_work> web is hard
15:32:53 <karsten> let's take this to trac. great input here.
15:32:58 <iwakeh> true.
15:33:04 <karsten> moving on?
15:33:09 <iwakeh> yep
15:33:30 <karsten> unless there's something else on the redesign topic?
15:33:34 <RaBe> wait
15:33:35 <karsten> RaBe: ^
15:33:42 <iwakeh> the javadocs.
15:34:10 <RaBe> do you want me to work on any of the pages, or should i look into some more atlas tickets? :)
15:34:48 <karsten> fine question.
15:34:56 <RaBe> especially the metrics-lib page isn't done yet
15:35:07 <karsten> right. maybe I should review that one next.
15:35:18 <iwakeh> true, and a bit different due to the javadoc.
15:36:06 <karsten> well, the javadocs could live on metrics.tpo, too, right?
15:36:24 <karsten> or what did you refer to, iwakeh?
15:36:27 <iwakeh> yes, just wanted to make clear these
15:36:41 <iwakeh> won't be on a single page with everything else.
15:36:47 <karsten> true.
15:37:00 <karsten> I wonder if the tutorials will fit on the single page.
15:37:03 <iwakeh> they can be anywhere.
15:37:12 <iwakeh> maybe not.
15:37:25 <karsten> or we need to keep them really short. :)
15:37:28 <iwakeh> depends on how elaborate they become.
15:37:33 <iwakeh> ;-)
15:37:41 <karsten> I mean, who would have thought that all CollecTor format specifications fit on a single page?
15:38:04 <iwakeh> Well, web-pages are endless.
15:38:15 <karsten> the alternative is to revisit the website navigation topic, but I'm so afraid of that...
15:38:22 <karsten> endless they are.
15:38:39 <karsten> okay, moving on?
15:38:42 <RaBe> so... for now, i'll just wait for any response on the three webpage tickets :)
15:38:55 <karsten> RaBe: yes, I hope to have something by tomorrow morning.
15:39:00 <RaBe> great!
15:39:26 <karsten> - Tech reports (iwakeh)
15:39:31 <iwakeh> yes,
15:39:53 <iwakeh> the contents are there now especially
15:40:03 <iwakeh> after I read the privcount paper
15:40:14 <iwakeh> and now it seems a matter of sculpting
15:40:20 <karsten> heh
15:40:24 <iwakeh> the material into one report.
15:40:51 <iwakeh> so, I think the vision for the
15:41:11 <iwakeh> future two topics 4.2. & 4.3 is pretty defined, too.
15:42:03 <karsten> that's the Tor daemon patches?
15:42:11 <iwakeh> right
15:42:28 <iwakeh> the ideas what they will be.
15:42:29 <karsten> great! C hacking!
15:42:41 <iwakeh> uh, make tor java :-)
15:42:51 <iwakeh> yes C for a change .
15:42:54 <karsten> heh
15:43:13 <karsten> looking forward to this!
15:43:23 <iwakeh> fine :-)
15:43:29 <karsten> moving on for now?
15:43:32 <iwakeh> sure.
15:43:38 <karsten> - OnionPerf (hiro)
15:43:57 <karsten> hiro: you posted some graphs... let me find them..
15:43:57 <hiro> sure
15:44:16 <hiro> so I made this board of transfer times: https://github.com/hiromipaw/onionperf-notebook/blob/master/board.md
15:44:19 <hiro> on onionperf
15:44:49 <hiro> it's on a 10 days time period
15:45:02 <karsten> right. one thing I wondered: why is the 5MB download almost faster than the 1MB download?
15:45:05 <hiro> yes
15:45:08 <hiro> that I wonder too
15:45:33 <hiro> I did it this afternoon and I was like why are these the same?
15:46:10 <karsten> and the 50KB is slower than the torperf 50KB one.
15:46:14 <hiro> yep
15:46:19 <hiro> it is a couple of sec slower
15:46:29 <hiro> which is like a lot
15:47:02 <karsten> would it make sense to compare different timestamps, not just end-start?
15:47:23 <hiro> I am doing 0percent to complete
15:47:56 <karsten> which fields on https://collector.torproject.org/#type-torperf ?
15:48:26 <karsten> the torperf graph uses START and DATACOMPLETE, IIRC.
15:48:36 <hiro> DATACOMPLETE
15:49:02 <karsten> yes, START and DATACOMPLETE.
15:49:15 <karsten> do you also use START?
15:49:38 <hiro> and I use DATAPERC0
15:49:52 <hiro> for start because that's available... but I could use start
15:49:53 <karsten> okay, that's not the same.
15:50:03 <iwakeh> x >=10
15:50:03 <hiro> ok I'll change it and update the ticket then
15:50:04 <karsten> that's when you receive the first byte.
15:50:24 <karsten> iwakeh: onionperf adds DATAPERC0, I think. even though torperf doesn't have it.
15:50:28 <karsten> spec extension!!
15:50:34 <iwakeh> yes :-)
15:50:37 <karsten> hiro: thanks, that would be great.
15:51:03 <hiro> sure no problems
15:51:31 <karsten> oh, and I didn't finish the collector code that fetches onionperf files yet.
15:51:36 <karsten> not sure if I get that done tomorrow.
15:51:45 <karsten> but I'm working on it.
15:51:49 <iwakeh> fine.
15:52:00 <hiro> fine w me
15:52:04 <karsten> okay, moving on?
15:52:27 <karsten> - metrics-lib tutorials (karsten)
15:52:50 <karsten> I have been thinking about these tutorials, though I haven't started writing them.
15:53:02 <karsten> but I'd like to discuss very briefly what devs might expect there.
15:53:13 <karsten> let me paste some questions on the pad..
15:53:25 <karsten> or hey, it's just five questions. let me paste them here.
15:53:33 <iwakeh> hehe
15:53:39 <karsten> 1) how many tutorials should there be? 3? more? less?
15:53:39 <karsten> 2) should we rather cover a diverse set of descriptors (relay, bridge, exit lists, torperf) or just the most important ones (relay and bridge)?
15:53:42 <karsten> 3) should we try to pick real-world examples from earlier analysis, or should we make up simple examples?
15:53:45 <karsten> 4) should we focus on code that interacts with metrics-lib, or should we also describe how to process parsed data using Java collection classes, files, or a database?
15:53:48 <karsten> 5) is it a problem if the same results can be achieved with grep, sed, and awk?
15:54:06 <iwakeh> 1) at least one ;-)
15:54:09 <karsten> heh
15:54:18 <iwakeh> 2) important ones.
15:54:38 <iwakeh> 3) one simplle and maybe one more tricky example.
15:54:57 <iwakeh> 4) only focus on metrics-lib, we don't write java tutorials.
15:55:17 <karsten> re 4) we could provide the full examples for download.
15:55:31 <iwakeh> 5) what do you mean exactly?
15:55:34 <karsten> and only describe the metrics-lib related parts on the website.
15:55:40 <iwakeh> yes
15:55:53 <karsten> re 5) here's how you count relays with the Exit flag.
15:55:53 <iwakeh> usually such tutorials focus
15:56:05 <karsten> it's faster to write that using grep.
15:56:07 <iwakeh> on the special library parts and the code contains all.
15:56:22 <karsten> makes sense.
15:56:38 <karsten> okay, that's useful.
15:56:39 <iwakeh> well, the number itself is not the
15:56:48 <iwakeh> reason to employ a library.
15:57:12 <iwakeh> so no problem, but we should have maybe not too simple examples ;-)
15:57:22 <karsten> right. in such a case the main purpose of the tutorial would be to show how to set up everything, including DescriptorReader, etc.
15:57:33 <iwakeh> yes, that'll be fine.
15:58:00 <iwakeh> that's what tutorials do
15:58:01 <karsten> okay, thanks, I can continue thinking about this. not sure when, but this is a fine thing to keep on the back burner.
15:58:24 <iwakeh> tutorials for recursive programming calculate x!
15:58:39 <iwakeh> even though there are plenty of tools around todo that.
15:58:43 <karsten> right.
15:59:00 <karsten> oh, do you have any sample tutorials at hand?
15:59:00 <iwakeh> all answered :-)
15:59:07 <iwakeh> ?
15:59:07 <karsten> somewhat related.
15:59:19 <karsten> any tutorials you found impressively easy to read, fun to try out, etc.
15:59:21 <iwakeh> I can think about htat
15:59:32 <karsten> okay, if you find something, please let me know.
15:59:33 <iwakeh> not at hand but easy to find.
15:59:37 <karsten> cool!
15:59:45 <karsten> done on time? :)
15:59:53 <iwakeh> yes :-)
16:00:01 <karsten> yay. thanks, everyone!
16:00:06 <karsten> #endmeeting