13:01:48 <asn_> #startmeeting SponsorR
13:01:48 <MeetBot> Meeting started Tue Sep  8 13:01:48 2015 UTC.  The chair is asn_. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:01:48 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
13:01:54 <asn_> let's use this pad for deliverables: https://storm.torproject.org/grain/CA5H5Ji6EDxnDv3tdiSshY
13:02:11 <syverson> dgoulet: just went to a local (to me) pub's biergarten two nights ago. Very nice.
13:02:14 <dgoulet> asn_: that's not the shared URL
13:02:24 <dgoulet> asn_: you need to create on with a button "Shared link" or something
13:02:37 <dgoulet> syverson: in DC?
13:02:53 <syverson> Silver Spring MD
13:03:20 <syverson> http://denizensbrewingco.com/
13:03:23 <asn_> https://storm.torproject.org/shared/dqn_MKP56HvKeH0kkIWyaueiL6fQqmcAgXWPjBkGfh3
13:03:28 <asn_> ok here we go
13:03:31 <asn_> does this work for you?
13:03:50 <dgoulet> asn_: yes!
13:03:53 <asn_> ok good.
13:03:55 <asn_> so anyway.
13:04:04 <asn_> welcome to a new season of sponsorr
13:04:13 <dgoulet> da best
13:04:19 <asn_> do you want to jump directly to figuring out deliverables
13:04:20 <asn_> ?
13:04:33 <asn_> is there anythign exciting happenign right now we should talk about before deliverables?
13:04:56 <dgoulet> year 2 planning for me is good!
13:05:43 <asn_> some 50% deliverables are quite close (next month)
13:05:52 <asn_> but i think we have stuff to present for those.
13:05:57 <dgoulet> is "onion service" HS or single onion service ?
13:06:11 <syverson> both
13:06:16 <asn_> pretty sure it stands for HS.
13:06:17 <asn_> (or both)
13:06:25 <dgoulet> ok
13:06:41 <syverson> Or do you mean specifically in this doc? I'll have to look.
13:06:46 <dgoulet> in the pad
13:06:57 <asn_> dgoulet: what do we have for task 2.4?
13:07:02 <asn_> as we are i mean
13:07:23 <asn_> controller event logs and stuff
13:07:34 <asn_> pretty sure we can call this 50% already right?
13:07:41 <asn_> from summer's work.
13:07:41 <dgoulet> I have the hs health tool working, maybe not gathering the exact data we would like but working
13:07:57 <syverson> I think it meant double-onion services since that's what's running, but should probably make sure things work for both.
13:07:58 <dgoulet> scripts that parse controller logs is maybe 30% done (did that in July)
13:08:07 <dgoulet> so clearly 50% done by Oct
13:08:12 <asn_> ok
13:08:27 <asn_> ok see task 3.4
13:08:28 <asn_> - Run the HS Health Measurer (from 2.4) more consistently, and start
13:08:28 <asn_> collecting summary data from it. Add the interesting parts as new
13:08:28 <asn_> graphs / data sets on the metrics page.
13:08:38 <asn_> this is part of the metrics deliverable
13:08:53 <asn_> before we agree to this we should make sure there is nice presentable data that can come out of the HS health measurer.
13:08:56 <asn_> maybe.
13:08:57 <dgoulet> asn_: yes and I would *really* like that we discuss that at the dev meeting in person with Roger
13:09:11 <asn_> which that? hs health measurer?
13:09:20 <dgoulet> the hs health part yes and then which graphs to put on metrics
13:09:25 <asn_> you think roger will have additional insights on this?
13:09:46 <asn_> i mean he would, but what specifically are you looking for?
13:10:06 <dgoulet> asn_: that we all agree on the data that hs health is gathering and graph that it creates
13:10:19 <dgoulet> remember last time, you were a bit confused of the usefulness of the graph we had
13:10:20 <asn_> ok sounds like a fine discussion.
13:10:25 <asn_> yes i remember
13:10:35 <dgoulet> so in person we can clearly get all on the same page and organize the work
13:10:35 <asn_> ok sounds fine.
13:10:38 <dgoulet> cool
13:10:43 <asn_> next fun deliverable (cc syverson)
13:10:44 <asn_> - Work with NRL when they do their HS scalability experiments: they're
13:10:44 <asn_> going to simulate n crawlers hitting a Tor network at once, and see
13:10:44 <asn_> what happens to the simulated Tor network as n grows.
13:10:57 <asn_> O_o
13:10:58 <asn_> what is this about?
13:11:10 <asn_> are we afraid that crawlers are going to take down the ntwork?
13:11:15 <syverson> You mean you don't understand the task?
13:11:17 <dgoulet> sounds like shadow simulation
13:11:28 <asn_> yes sounds like shadow simulation. trying to understand what can come out of it.
13:11:29 <syverson> Briefly, yes.
13:11:44 <syverson> Also what fails and how as load increases.
13:11:50 <asn_> hmm ok
13:12:05 <Yawning> hm
13:12:06 <asn_> there are various ways you can crawl onions (you can ask for the descriptor every time, or only once. you can do a single rendezvous, or a thousand)
13:12:20 <syverson> Yep.
13:12:21 <asn_> it seems to me that you will see different results depending on how you implement your crawler
13:12:31 <asn_> which mnight be different from how other people implement their creawlers
13:12:42 <asn_> but i guess you know what's going on here, so it's fine.
13:12:43 <Yawning> most of the really aggressive behaviors will kill the client tor instance
13:12:48 <Yawning> >.>
13:13:33 <syverson> A hope would be to make sure that benign crawling doesn't break things once all those memex partners start doing their own crawls.
13:13:42 <dgoulet> Yawning: this is under shadow so my guess is each crawler has it's own tor instance copy
13:14:46 <Yawning> dgoulet: yeah, just thinking about hwat happened when I ran my boom fork with really aggressive settings
13:16:19 <asn_> ok
13:16:22 <dgoulet> asn_: so what should we do here, triage what we think is good to do or not and for them maybe create task list/tickets ?
13:16:31 <asn_> hmmm
13:16:44 <asn_> i was just making sure that there is not anything terrible in the list.
13:16:58 <dgoulet> good idea
13:17:04 <asn_> but i think i like roger's list :)
13:17:22 <dgoulet> same
13:17:23 <asn_> i agree that the next step would be to match tasks to trac tickets and create any needed tickets.
13:17:32 <dgoulet> particularly 3.3 :D
13:17:58 <dgoulet> you know "fixes" we are considering <cough>224</cough>
13:18:13 <asn_> so task 5.3.1 depends on task 1.3
13:18:18 <asn_> - All the "onion services fall over when you hit them 100 times at once"
13:18:18 <asn_> tickets. Act on the various conclusions from task 1.3 so everything
13:18:18 <asn_> works better. "Describe sources of delays/failures in contacting HS".
13:18:37 <asn_> and task 1.3 is the one with chutney and stuff
13:18:54 <dgoulet> we already have tickets about that from profiling results
13:19:02 <asn_> i think teor was trying to handle the "100 users crash HS" ticket using chutney
13:19:15 <asn_> but it didn't work, because his client script was hammering his PC.
13:19:26 <dgoulet> yup it's quite aggressive
13:19:27 <asn_> which was already hammered by the Tor network running in chutney.
13:19:43 <asn_> so single box investigation doesn't seem super useful.
13:19:54 <asn_> except if you can isolate processes to cores or something insane like that.
13:20:11 <asn_> but during the DoS crisis we did some profiling and stuff
13:20:16 <asn_> so we have info.
13:20:27 <dgoulet> Rob has a cluster at NRL that is quite capable of going crazy on stuff, maybe we could ask him to run a couple of shadow experiement there for HS ;)
13:20:38 <asn_> we ended up making those "implement rendezvous queue" ticket etc.
13:20:51 <asn_> plausible
13:21:40 <syverson> Something I don't see here that I think I talked with dgoulet and maybe Roger about during summer camp. Wade really wanted some sort of signalling (e.g. crawl cell) so that we can identify voluntary indications of crawling. This could be detected at crawling detector onion services and/or at RPs.
13:21:54 * karsten just arrives late to the meeting. sorry!
13:22:18 <asn_> syverson: how would that be used?
13:22:38 <asn_> syverson: it's the equivalent of google setting its User-Agent to Googlebot ?
13:22:59 <asn_> karsten: hello
13:23:04 <karsten> hello asn_
13:23:09 <syverson> To help gather statistics on crawling vs. general onion service traffic, rather than having to try to infer same.
13:23:38 <syverson> asn_: I don't know Googlebot.
13:23:40 <asn_> karsten: you might be interested in task 3.4 from https://storm.torproject.org/grain/CA5H5Ji6EDxnDv3tdiSshY/
13:23:41 <Yawning> isn't that a bad idea from the perspective of the memex people?
13:23:52 <Yawning> because someone will write a patch to kill the circuit at the rp
13:23:53 <syverson> Yawning: how so?
13:24:04 <syverson> Why would they do that?
13:24:05 <Yawning> since enough of the userbase distrusts memex and crawling
13:24:15 <asn_> syverson: the ancient problem with this, is that the bad guys will not send the crawl cell. the good guys will do, and they will get blocked.
13:24:25 <Yawning> it's the equivalent of writing a "hell fuck no, go away google robots.txt"
13:24:45 <syverson> But why should it just be memex that's doing the crawling, anybody might be crawling. Could be ahmia, etc.
13:24:50 <Yawning> what asn said
13:24:52 <Yawning> sure
13:25:00 <karsten> asn_: hmm, it says unauthorized.
13:25:15 <dgoulet> karsten: https://storm.torproject.org/shared/dqn_MKP56HvKeH0kkIWyaueiL6fQqmcAgXWPjBkGfh3
13:25:16 <asn_> karsten: i'm sorry https://storm.torproject.org/shared/dqn_MKP56HvKeH0kkIWyaueiL6fQqmcAgXWPjBkGfh3
13:26:02 <Yawning> IMO there's a disconnect between researchers working on memex/tor and the community and that it's still poorly explained. but that might just be my impression
13:26:03 <karsten> dgoulet, asn_: better, thanks.
13:26:21 <syverson> Yeah but I'm talking about benign crawling. Part of this would also help differentiate that. If somebody doesn't indicate they're crawling but are detected, we can see what that tells us. Think setting my_family.
13:26:35 <Yawning> Sure, I understand why you'd want to do that
13:27:12 <asn_> syverson: i'm fine with adding an item to the list about researching this avenue of thought
13:27:21 <Yawning> just saying that paranoia (unfounded or not) about the project exists
13:27:22 <syverson> I don't think there's any one such thing as "the community". It's important to indicate which "the community" you're talking about.
13:27:32 <asn_> personally I don't find it useful as it stands, but maybe i need to think more.
13:27:58 <Yawning> *shrugs*
13:28:40 <asn_> i mean, probably only ahmia will turn it on (and other ahmias), and then we will be able to gauge how much ahmia is crawling.
13:28:41 <syverson> I think it's a valid concern how robust our infrastructure is if there's e.g. thirty or fifty entities crawling at the rate SRI does.
13:28:52 <asn_> useful? maybe. worth changing the protocol for it? not sure.
13:29:13 <asn_> ok let's add a note about this!~
13:29:14 <karsten> dgoulet: is now a good time to talk about 3.4 (metrics) and 2.4 (hs health measurer)?
13:29:20 <syverson> Also, our c. 3 percent stat was after they started crawling. What if 2 of the 3 percent was SRI, wouldn't you like to know that?
13:29:49 <dgoulet> karsten: I would really want to do that in Berlin if possible (not too far away ~20 days)
13:29:52 <Yawning> asn_: could send a padding cell with "MEMEX PLS IGNORE" or something as the payload
13:29:54 <Yawning> :P
13:30:02 <Yawning> and not change the protocol
13:30:10 <karsten> dgoulet: when do we have to commit to this plan?
13:30:11 <dgoulet> karsten: lots to be said and I think we still need to figure out some details
13:30:25 <asn_> karsten: sure.
13:30:45 <dgoulet> karsten: 50% is by October but we have that already for the hs health and task 3.4 is feb 2016 for 50% for which we already have stuff
13:31:03 <syverson> Yawning: What about non-memex crawling taking place? But yeah something like that.
13:31:31 <karsten> okay, two quick questions: would you run the hs health measurer? and can we write a spec of some sort for the data format it produces?
13:31:48 <dgoulet> Yawning: fyi, memex is open sourcing everything they do so expect people outside of that project using their agressive crawlers... :S
13:31:59 <dgoulet> karsten: yes and yes
13:32:14 <karsten> sounds great. let's talk about the details in berlin. :)
13:32:47 <asn_> ok
13:33:03 <karsten> dgoulet: when do you arrive in berlin?
13:33:10 <asn_> so i think we should look at this list some more during the week, and assign tickets and stuff to each item.
13:33:13 <asn_> maybe we don't need to do it no,.
13:33:16 <asn_> now
13:33:27 <asn_> karsten: i arrive early!
13:33:28 <Yawning> dgoulet: heh
13:33:29 <dgoulet> karsten: only the 27th beginnning of PM
13:34:20 <karsten> dgoulet: ah ok, not much time before the meeting then. asn: great, we should meet on the weekend. my plan is to arrive on that friday.
13:34:33 <dgoulet> asn_: I agree, we can break them down during the week, on the pad or wiki ?
13:34:40 <syverson> dgoulet: from heathrow by any chance?
13:34:52 <asn_> dgoulet: we should eventually transcribe them to https://trac.torproject.org/projects/tor/wiki/org/sponsors/SponsorR
13:34:55 <dgoulet> syverson: I'm in .eu already and will be coming from cdg
13:34:55 <asn_> dgoulet: we can use the pad till then
13:35:01 <syverson> Oh right.
13:35:03 <dgoulet> asn_: good
13:35:12 <asn_> so, weekly meetings or biweekly?
13:35:19 <asn_> do we want to meet next week or the week after?
13:35:22 <dgoulet> karsten: are you staying the whole week in Berlin?
13:35:31 <karsten> dgoulet: yes.
13:35:57 <dgoulet> asn_: I would say biweekly
13:36:12 <dgoulet> karsten: ok great, plenty of opportunities :)
13:36:19 <karsten> dgoulet: sure!
13:36:24 <syverson> More generally, either meet next week or meet in Berlin.
13:36:37 <asn_> ok
13:36:53 <syverson> Can decide in Berlin on some actual schedule for these.
13:36:55 <asn_> so let's meet again on the 22nd?
13:36:56 <asn_> yes indeed.
13:37:07 <asn_> and we the Tor folks will be in IRC all the itme anyway
13:37:08 <dgoulet> yes let's break it down and then consolidate in Berlin
13:37:13 <asn_> so we can have impromptu meetings if needed.
13:37:41 <nickm> re!
13:37:48 <syverson> Same time on the 22nd?
13:38:00 <asn_> fine by me.
13:38:03 <asn_> did you guys like the time?
13:38:04 <dgoulet> +1
13:38:13 <asn_> ok great.
13:38:14 <syverson> Is there anyone in california who we're missing?
13:38:24 <asn_> don't think so
13:38:25 <syverson> Cause it's 0600 there.
13:38:28 <asn_> yeah
13:38:36 <dgoulet> isabela :P
13:38:39 <asn_> hmmm
13:38:46 <asn_> but i think she mainly enjoys the backlog
13:39:00 <nickm> oh was this sponsor R?
13:39:03 <asn_> well we can certainly make the meeting later if she asks for it.
13:39:11 <syverson> nickm: Yes
13:39:18 <asn_> so next meeting the 22nd of September at 13:00UTC .
13:39:30 <syverson> I'm copacetic.
13:39:49 <asn_> i will send an email to roger to tell him that we ACKed his deliverables.
13:39:58 <asn_> and that we will be assigning tickets to it over the next week or so.
13:40:06 <dgoulet> syverson: if you can ask Rob to come by next meeting would be great I think since some shadow stuff is involved :)
13:40:19 <dgoulet> syverson: I at least will have questions for him :)
13:40:30 <dgoulet> asn_: sounds good
13:40:38 <asn_> ok thx for the meeting!
13:40:39 <asn_> #endmeeting