14:29:39 <karsten> #startmeeting metrics team
14:29:40 <MeetBot> Meeting started Thu Aug 10 14:29:39 2017 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:29:40 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:29:42 <karsten> hello.
14:30:33 <iwakeh> hi there!
14:30:44 <karsten> hi iwakeh!
14:30:50 <iwakeh> trac seems down.
14:30:55 <karsten> yep.
14:31:01 <karsten> being worked on.
14:31:16 <iwakeh> yes, the error messages vary :-)
14:31:17 <karsten> https://storm.torproject.org/shared/Ou-1QRctynWbF4yedi-MfDsjImFMFSIEP20fbVGCPRa <- agenda pad
14:31:22 <karsten> heh
14:31:36 <karsten> okay, shall we go through the agenda items?
14:31:42 <iwakeh> fine
14:31:45 <karsten> * Onionoo backup host (iwakeh): running fine (see stats below)
14:31:49 <karsten> yay!
14:32:02 <iwakeh> yes, it helped to use more memory ;-)
14:32:07 <karsten> how much?
14:32:12 <iwakeh> just 4g
14:32:15 <karsten> ah ok.
14:32:20 <karsten> cool!
14:32:20 <iwakeh> but the host has 12g
14:32:28 <karsten> did you compare results?
14:32:38 <karsten> like, does it diverge from omeiense?
14:32:41 <iwakeh> not yet
14:32:51 <karsten> okay, that's one possible next step.
14:33:01 <iwakeh> that's part of the 'next steps' question.
14:33:07 <karsten> right.
14:33:13 <iwakeh> but I#M reluctant
14:33:15 <karsten> it's not accessible from outside, right?
14:33:34 <iwakeh> to run the web b/c it is accessible from outside.
14:33:52 <karsten> hmm?
14:34:02 <iwakeh> Which of course makes it more convenient to
14:34:14 <iwakeh> retrieve the data and shut-down the service again.
14:34:59 <karsten> I'm unclear what the issue is.
14:35:16 <iwakeh> There is no firewall.
14:35:20 <iwakeh> I assume.
14:35:32 <iwakeh> no redirection etc.
14:35:39 <karsten> did you try?
14:35:43 <iwakeh> yes,
14:35:53 <iwakeh> I ran the web and accessed it locally
14:36:04 <iwakeh> and just for fun I tried the
14:36:16 <iwakeh> host:8080 and got the pages.
14:36:19 <karsten> ok.
14:36:24 <karsten> well, not the end of the world.
14:36:40 <iwakeh> and because of that I shut down the web immediately.
14:36:56 <karsten> btw, on omeiense we're running the web udner a different user.
14:37:04 <iwakeh> I noticed :-)
14:37:07 <karsten> onionoo-unpriv
14:37:15 <iwakeh> yes, good idea.
14:37:36 <karsten> okay, do you want to discuss those things (firewall, redirection, etc.) with the admins?
14:37:43 <karsten> (different user, ...)
14:37:55 <iwakeh> same setting as omeiense?
14:38:05 <karsten> that would work, yes.
14:38:48 <karsten> ok.
14:38:59 <karsten> next step after those two: include in rotation.
14:39:12 <iwakeh> ok.
14:39:29 <karsten> do you want to do the configuration stuff before comparing results?
14:39:53 <iwakeh> hmm, and when results are different?
14:40:03 <karsten> well, and before including it in the rotation.
14:40:13 <karsten> though, you could also listen on localhost only.
14:40:20 <iwakeh> I'd like to know it first, but
14:40:36 <iwakeh> it could be done parallel as I need to mail with the admins.
14:40:48 <iwakeh> I guess.
14:40:50 <karsten> right. and if you only open the port on localhost, you should be fine.
14:41:06 <iwakeh> yes, I can set that.
14:41:14 <karsten> ok.
14:41:40 <karsten> that are some good next steps, I think.
14:41:45 <iwakeh> true.
14:41:49 <karsten> moving on?
14:41:56 <iwakeh> yes.
14:42:01 <karsten> * metrics-lib internal&LogDescriptor (iwakeh): #22983
14:42:12 <iwakeh> ready for review.
14:42:15 <karsten> is that the ticket in needs_review from today?
14:42:30 <iwakeh> correct and #22428
14:42:36 <iwakeh> is to follow soon.
14:42:39 <karsten> ok!
14:42:59 <karsten> I'll look tomorrow.
14:43:00 <iwakeh> there are quite a few code 'reductions'
14:43:11 <iwakeh> possible with the new approach.
14:43:18 <karsten> okay. :)
14:43:59 <karsten> let's talk more after the review?
14:44:04 <iwakeh> sure.
14:44:11 <karsten> great!
14:44:18 <karsten> moving on:
14:44:19 <karsten> * Check logs of TB downloads (iwakeh)
14:44:47 <iwakeh> still wondering about the keep  salt in-memory with subsequent
14:44:55 <iwakeh> script runs.
14:45:05 <iwakeh> seems quite tricky.
14:45:25 <karsten> that's why I suggested the logio module.
14:45:38 <karsten> which could answer our question, too.
14:45:38 <iwakeh> which doesn't hash IPs.
14:45:42 <karsten> true.
14:45:47 <karsten> different approach.
14:46:01 <iwakeh> yes, but with less answers. I intend
14:46:17 <iwakeh> to just drop a mail to the admins and maybe they have a solution.
14:46:23 <karsten> sounds good!
14:46:35 <iwakeh> after all a salt removed and hidden from us should work too.
14:47:00 <karsten> maybe.
14:47:15 <iwakeh> well, we all trust root there, don't we.
14:47:50 <karsten> it's the part where non-hashed IPs hit the disk that I'm worried about.
14:48:06 <iwakeh> oh, they shouldn't hit the drive unhashed.
14:48:16 <iwakeh> only the salt is stored during the process.
14:48:35 <iwakeh> and then erased.
14:48:50 <karsten> and read from disk before writing a log line?
14:49:12 <iwakeh> hmm, yes.
14:49:23 <karsten> sounds okay.
14:49:48 <iwakeh> fine.
14:49:52 <karsten> alright. asking the admins sounds like a fine next step.
14:49:56 <karsten> next?
14:50:02 <iwakeh> yep.
14:50:08 <karsten> * putting ExoneraTor on Tor Metrics (karsten)
14:50:14 <karsten> so, I worked a bit on exonerator in the past days.
14:50:30 <iwakeh> is there a ticket?
14:50:46 <karsten> and I have a version with a servlet that produces a JSON file with results and the original servlet using that JSON file rather than querying the database directly.
14:50:54 <karsten> there's no ticket yet. I didn't know how far I'd get.
14:51:01 <karsten> but now I have something running here.
14:51:01 <iwakeh> ok
14:51:16 <iwakeh> fine, a Metrics sub-page.
14:51:24 <karsten> the question is: should we put ExoneraTor on Tor Metrics, say, under https://metrics.torproject.org/exonerator.html?
14:51:43 <karsten> and keep https://exonerator.torproject.org/ running for now, just serving the data?
14:51:52 <iwakeh> hmm,
14:52:00 <iwakeh> maybe a sub-tab?
14:52:13 <iwakeh> It's sort of like the graphs.
14:52:19 <karsten> ah.
14:52:30 <iwakeh> I enter a configuration/data and receive a result.
14:52:41 <karsten> regarding the place on Tor Metrics,
14:52:50 <karsten> I thought about putting it under Home >> Operation >> ExoneraTor.
14:53:15 <karsten> that page would pretty much be what's currently on https://exonerator.torproject.org/, with different header.
14:53:25 <karsten> and it would show results.
14:53:36 <karsten> we'd just keep exonerator.tp.o running for the JSON data.
14:53:47 <karsten> so, Tor Metrics would query that host and get the JSON object with results.
14:53:55 <irl> is this the same model as for collector then?
14:54:00 <karsten> hi irl!
14:54:04 <iwakeh> hi!
14:54:04 <karsten> pretty much, yes.
14:54:10 <iwakeh> well, no
14:54:28 <iwakeh> not exactly, as the user gives an ip and receives a result.
14:55:00 <irl> i mean, in the split of backend vs. frontend it's similar, vaugely in a handwave way
14:55:11 <karsten> yes.
14:55:16 <karsten> that's also what I meant.
14:55:26 <karsten> and a bit like onionoo.
14:55:32 <iwakeh> ah, I was looking from the user perspective, usually
14:55:33 <karsten> where the specification is now on Tor Metrics.
14:55:44 <karsten> and the service is still running on onionoo.tp.o.
14:55:50 <iwakeh> we're 'hiding' the more tachnical data away.
14:56:23 <iwakeh> but, for a start we could just use the onionoo/collector approach.
14:56:38 <karsten> what would be the step after that?
14:56:42 <karsten> (when you say for a start)
14:57:11 <iwakeh> see, if complaints come in and move
14:57:20 <iwakeh> it to some more prominent place.
14:57:45 <karsten> you mean easier to find on Tor Metrics?
14:57:52 <irl> on the metrics website, it's more discoverable
14:57:57 <iwakeh> yes, easier to find for non-techs.
14:58:03 <irl> it's already linked from the operations page, it's just where the link goes that will change
14:58:08 <irl> old location would redirect anyway
14:58:14 <iwakeh> true.
14:58:14 <karsten> true.
14:58:23 <karsten> I mean, if we can make it even easier to discover, let's do that.
14:58:28 <irl> (:
14:58:52 <karsten> right now, we focus mainly on graphs.
14:58:56 <karsten> on the start page.
14:59:11 <karsten> we could change that and also add direct links to services.
14:59:18 <iwakeh> I think, it'll be fine with the links and redirection.
14:59:21 <karsten> well, extend, not necessarily change.
14:59:26 <irl> as a fun thing, "this ip address was last seen as a tor relay on DD-MM-YYYY" link to exonerator in the footer of the metrics website, link to exonerator
14:59:41 <irl> maybe it'll always say "never" for most users, but it's an interesting thing people might click
15:00:00 <karsten> hehe. and it'll totally kill the database to make that many requests. ;)
15:00:09 <irl> ok, maybe not then (:
15:00:13 <karsten> not yet.
15:00:14 <karsten> ;)
15:00:15 <iwakeh> oh, another reason for refacturing ;-)
15:00:25 <iwakeh> the db, that is.
15:00:28 <karsten> yes!
15:00:32 <karsten> I didn't touch that part.
15:00:42 <karsten> could be a bigger project.
15:00:54 <karsten> anyway.
15:00:58 <iwakeh> but for the integration now the plan seems set
15:01:06 <karsten> alright, I'll proceed with this plan.
15:01:14 <karsten> and open a ticket once I have something to share.
15:01:20 <iwakeh> :-)
15:01:50 <karsten> speaking of,
15:02:01 <karsten> should we consider doing something like this with atlas?
15:02:16 <karsten> that is, move it to Home >> Operation >> Atlas ?
15:02:19 <irl> on my roadmap is updating the bootstrap and jquery things
15:02:28 <irl> we could retheme it to the metrics theme, i don't see why not
15:02:38 <irl> we have a ticket for this
15:02:47 <iwakeh> cool!
15:02:50 <karsten> I mean, we could even include it on the page.
15:02:59 <karsten> and avoid updating jquery and bootstrap in 2 places.
15:03:11 <karsten> not suggesting this as short-term thing.
15:03:17 <karsten> but, looking at the next weeks or so.
15:03:23 <irl> yep, i get it, that is what i meant
15:03:35 <irl> instead of updating it, just go for integrating it with metrics website
15:03:45 <irl> which updates it as a side effect
15:03:51 <karsten> yes. worth thinking about!
15:04:00 <irl> i'll update the ticket once trac is back
15:04:20 <karsten> in theory, we could add atlas.git as submodule and let metrics-web pull it in when producing a .war file.
15:04:28 <karsten> not sure what's easiest.
15:04:32 <karsten> okay.
15:04:50 <irl> we can discuss this on the ticket
15:04:56 <irl> i like it though
15:05:07 <karsten> cool!
15:05:10 <karsten> last on the list:
15:05:13 <karsten> * future of Compass (karsten)
15:05:25 <karsten> we should decide what to do with compass.
15:05:32 <karsten> I hear it's still being used.
15:05:38 <irl> is gsathya still around?
15:05:39 <karsten> but it's not really maintained anymore.
15:05:44 <karsten> I don't think so, no.
15:06:08 <karsten> though I believe there's a fork of some sort.
15:06:10 <irl> i know people do still use it
15:06:22 <iwakeh> we could take out the link?
15:06:22 <karsten> created by nusenu who cares a lot about tor network health.
15:06:39 <iwakeh> ah, or find a new active maintainer?
15:06:49 <karsten> if we want to keep it, we might want to review those changes and merge it back.
15:06:55 <karsten> or decide it's the new compass.
15:07:04 <irl> i could look at that, i can python
15:07:25 <karsten> that would be neat!
15:07:36 <karsten> maybe you can talk to nusenu about this, too.
15:07:55 <karsten> I noticed this branch/fork when looking at a compass ticket. let me find that.
15:08:08 <karsten> #10306
15:08:12 <karsten> #10306: Show relays by nickname substring
15:08:20 <karsten> (pretending to be the bot)
15:08:25 <iwakeh> hehe
15:08:28 <irl> bleep bloop
15:08:38 <iwakeh> (full url ;-)
15:08:45 <karsten> okay, do you want to take a look when trac is back?
15:08:50 <karsten> (sorry, I'm still learning..)
15:08:51 <irl> ok, i'll follow up on that
15:08:57 <karsten> cool!
15:09:47 <irl> (:
15:09:48 <karsten> alright.
15:09:53 <karsten> out of items.
15:09:55 <karsten> anything else?
15:10:00 <iwakeh> yep :-) no.
15:10:01 <irl> one quick onionperf
15:10:02 <karsten> (glad to see you back here, irl, by the way!)
15:10:08 <karsten> yes?
15:10:15 <irl> i have restored my instance and i'm doing some debugging
15:10:35 <irl> i'm also looking at writing icinga plugin to make sure it doesn't disappear for extended periods again
15:10:42 <karsten> heh
15:10:43 <irl> will follow up on onionperf deployment ticket if that's still going
15:11:09 <karsten> sure. either the existing ticket or a new one.
15:11:36 <irl> i will also probably be along to the montreal meeting, need to sort out some admin things but should have an idea by the end of the week
15:12:05 <irl> i think that's all my things
15:12:12 <karsten> be along means?
15:12:32 <irl> present in person on the days of the meeting, including the team meeting
15:12:43 <karsten> ah, neat!
15:12:53 <karsten> regarding dates,
15:13:09 <karsten> I'll only be around for 3 days, and iwakeh won't make it at all.
15:13:21 <karsten> see the trac page for details.
15:13:23 <iwakeh> :-(
15:13:28 <karsten> yes, :(
15:13:37 <irl> ah ok
15:13:51 <irl> still, maybe some remote participation can be organised
15:14:06 <irl> irc/skype/jitsi/whatever
15:14:09 <karsten> yes!
15:14:13 <iwakeh> yes.
15:14:32 <karsten> okay.
15:14:39 <karsten> are we done?
15:14:44 <iwakeh> I think so.
15:14:46 <irl> (:
15:14:59 <karsten> great. thanks, iwakeh and irl!
15:15:06 <iwakeh> thanks and bye, bye :-)
15:15:08 <karsten> more via trac once it's back. bye, bye!
15:15:19 <karsten> #endmeeting