17:15:32 <asn> #startmeeting snowflake meeting
17:15:32 <MeetBot> Meeting started Thu Apr 12 17:15:32 2018 UTC.  The chair is asn. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:15:32 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
17:15:39 <arma4> arlolra: ok. i just made a note at the end of the agenda, to think about what a part-time arlo job might look like
17:15:52 <arlolra> alrighty
17:16:14 <arma4> ok. item 1, how are we doing at the tasks we identified from last time as needing doing?
17:16:32 <arma4> the snowflake.torproject.org website is all set and ready i think. let me know if that isn't so.
17:16:38 <arma4> sukhe, any news on the windows build front?
17:16:54 <sukhe> ok I can go first
17:16:59 <arma4> great
17:17:10 <dcf1> Yes, snowflake.torproject.org is working, and I'm planning to move content there.
17:17:21 <sukhe> so there is progress, but the builds are not ready yet. to be clear, I am mostly building on all the work dcf1 has already done
17:17:28 <dcf1> Longer term, it will be nice to do a style guide-ificiation, like isabela said
17:17:59 <sukhe> so the webrtc builds on Windows are done (again, dcf1, arlolra) and now we are trying to build go-webrtc
17:18:37 <sukhe> there is a linking error, which we think is due to C++ ABI (webrtc is compiled with clang and go-webrtc with mingw-w64)
17:18:46 <antonela> arma4 sukhe dcf1 are you going to add more content at snowflake.torproject.org?
17:18:53 <dcf1> (This is a place where cgo actually is causing a problem, because webrtc wants to compile with clang and cgo wants to use gcc, and they have incompatible ABIs on Windows)
17:18:54 <isabela> dcf1: if you want our help ( antonela help)
17:18:56 <antonela> I'd like to provide the branded version
17:18:59 <isabela> yes
17:19:02 <isabela> ^^ this :)
17:19:08 <antonela> yes
17:19:16 <sukhe> I am currently trying to fix that. it's difficult to come up with an ETA
17:19:19 <isabela> dcf1: if you want you can provide content and anto can organize it in a design
17:19:43 <sukhe> antonela: I will let dcf1 and arlolra answer that one :)
17:19:43 <asn> dcf1: sukhe: and what's the fix wrt cgo webrtc clang/gcc?
17:19:51 <antonela> sukhe :)
17:19:53 <dcf1> antonela: Initially it will just be a copy of https://keroserene.net/snowflake/ (that's where we're migrating the page from)
17:20:06 <antonela> dcf1: yes, i have it saved
17:20:11 <antonela> all it?
17:20:34 <sukhe> asn: either fixing the linking errors or even better, build go-webrtc with clang but that has problems as well
17:20:49 <sukhe> because cgo wants to use gcc, like dcf1 mentioned
17:21:13 <arma4> sukhe: ok. is there anything we can do, in terms of people or etc, to help you there? or is the answer just that you need to beat your head against it for a while more?
17:21:15 <dcf1> antonela: yes, at first. The important part is not the text in the page, it's the JavaScript code. We need to move the JavaScript code there in order to to other tasks.
17:21:15 <sukhe> dcf1: should I focus on https://github.com/keroserene/go-webrtc/pull/69 (the wrapper) or https://github.com/azadi/tor-browser-build/commits/go-webrtc ?
17:21:22 <GeKo> dcf1: what is needed to get go-webrtc compiled with a/the clang toolchain?
17:21:37 <dcf1> sukhe: I have no clue :) I'm at the same impasse you are.
17:21:46 <sukhe> arma4: I think it's mostly that unless I am following the wrong approach but to prevent that from happening, I am keeping dcf1 and arlolra updated
17:22:15 <dcf1> GeKo: not sure what's required. I think that it is not actually supported by Go, as Hooman found out. He found a GitHub issue, let me find it.
17:22:30 <sukhe> arma4: I plan to ask "the experts" since I have tried what I could so that should help
17:22:30 <GeKo> because the medium term plan is to move to a clang/mingw-w64 toolchain anyway
17:22:44 <GeKo> because firefox needs it
17:23:06 <arma4> sukhe: right, bringing in other people seems like a smart move. if you find that we need to ask the twitters for help, for example, steph can totally do that. maybe there are less nuclear options too.
17:23:07 <sukhe> GeKo: https://github.com/golang/go/issues/17014 and https://github.com/golang/go/issues/20982 are the issues with cgo but I haven't read them in detail
17:23:14 <dcf1> GeKo: yes, agreed on clang/mingw-w64.
17:23:20 <antonela> dcf1: cool! I can work with this content. Thanks!
17:23:32 <GeKo> ok, thanks
17:24:22 <isabela> arma4: ssup with the item "- arma: figure out cookies w/ weasel"
17:24:28 <arma4> isabela: it's done
17:24:31 <arma4> all resolved
17:25:00 * arma4 is hunting through last week's pad to try to notice other things we were going to do but have missed (if you know any too, let us know)
17:25:07 <dcf1> Right, sukhe's https://github.com/golang/go/issues/17014 is the issue I was looking for, clang for cgo is not a supported configuration on windows; it's unclear how much work it will be to make it work.
17:25:09 <isabela> cool
17:25:38 <sukhe> so yeah, that's it from me, hoping to resolve this asap
17:25:48 <asn> sukhe: ack thx!
17:25:52 <asn> (kept some notes on pad)
17:26:09 <sukhe> dcf1: yeah, also related is why I didn't try the other stuff (like removing win_sdk from the webrtc builds) because I wanted us to have a working build first, then the enhacements
17:26:13 <arma4> great. we know this is a hard one, so please bring in whatever help you think you need, and/or escalate to us to try to get you that help.
17:26:13 <isabela> #23947
17:26:21 <isabela> ^^ ssup with this
17:26:27 <isabela> i dont think is in the roadmap
17:26:34 <arma4> isabela: that ticket is all done, and the remaining thing is "populate snowflake.tp.o website"
17:26:55 <arma4> and i guess part of populating the website is that they need to move the proxy code over to the new website
17:26:58 <isabela> ok
17:26:59 <arma4> and point new snowflakes at it
17:27:06 <arma4> does that involve a tor browser update too?
17:27:07 * isabela is digging
17:27:22 <isabela> GeKo: ^
17:27:27 <arma4> i think maybe no? since snowflake clients just talk to the broker?
17:27:32 <dcf1> arma4: yes, it will require a change in the client code and a new browser update.
17:27:44 <arma4> ok. why?
17:27:45 <dcf1> But it's graceful, old versions will continue to work under the old location.
17:27:51 <arma4> still trying to get the right mental model here :)
17:28:12 <dcf1> It's in one of Tor Browser's config files
17:28:38 <arma4> don't snowflake clients (a) tell the broker they need a snowflake, and then (b) await a snowflake contacting them?
17:28:45 <dcf1> https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/tor-browser/Bundle-Data/PTConfigs/linux/torrc-defaults-appendix
17:28:56 <dcf1> "-url https://snowflake-reg.appspot.com/" that part needs to change
17:29:13 <asn> ah
17:29:20 <dcf1> arma4: oh wait, I might be confusing something.
17:29:35 <arlolra> https://trac.torproject.org/projects/tor/wiki/doc/Snowflake#Tickets
17:29:53 <arlolra> see the graph
17:29:53 <dcf1> arma4: okay yes, you are right, for just moveing from keroserene.net to snowflake.tpo, we don't need a client update
17:30:09 <arma4> whew. mental model remains intact
17:30:12 <arlolra> we're ultimately trying to change the broker
17:30:13 <mcs> who controls  https://snowflake-reg.appspot.com/? is that also an issue in the long run?
17:30:18 <dcf1> but part of the reason for moving from keroserene.net is to start using our standalone (apart from APp Engine) broker, and that is the part that needs a client update
17:30:28 <mcs> ah, okay
17:30:40 <dcf1> mcs: serene controls https://snowflake-reg.appspot.com/, which is why we're trying to stop using it.
17:30:53 <arma4> so the glorious future involves running a broker not actually on appspot, and then we can use appspot to front to it, but also we can use other things to front to it?
17:31:07 <dcf1> arma4: correct
17:31:10 <arma4> great
17:31:28 <arma4> which ticket is "move the broker to a new more standalone location"?
17:31:33 <dcf1> In fact https://snowflake-reg-test.appspot.com/ (which is mine) is already fronting for https://snowflake-broker.bamsoftware.com/, which is the standalone broker.
17:31:47 <arma4> #22874
17:31:54 <arma4> okeydoke
17:32:13 <arma4> 22 mins. were there other items from last week that we should report progress on? or, on to next items?
17:32:13 <dcf1> Yeah see my comment:7 "I think we're not going to regain access to ​https://snowflake-reg.appspot.com/. I think the way forward is to do #23947; i.e., move the proxy-hosting page away from keroserene.net, and then we configure the proxy on the new host to use the new broker."
17:32:37 <arma4> sounds good. all that remains is to just do the things.
17:33:08 <arma4> so, my item 2 on the agenda. i wanted to understand better our plans for the arms race:
17:33:23 <asn> perhaps we should ask t0mmy what kind of specific info they need for drafting the job posting?
17:33:26 <asn> based on "t0mmy:  started on a job description but need specific info
17:33:36 * arma4 backs up
17:33:37 <arma4> yes ok great
17:33:43 <t0mmy> oh no rush
17:34:04 <t0mmy> the roadmap under (4)
17:34:30 <arma4> what about it :)
17:34:34 <t0mmy> is this what I should include in a dev job description?
17:34:44 <t0mmy> these tasks, I mean
17:34:49 <arma4> ah hm. i think no.
17:34:49 <isabela> no
17:34:56 <arma4> i think these immediate tasks that we know about, we have a pretty good handle on
17:35:05 <isabela> lets look at the pad
17:35:08 <arma4> some of them will be open-ended, like "keep x working" or "make x more robust"
17:35:27 <isabela> https://pad.riseup.net/p/K4cF5vPEU7Rg-censorshipteam-jobposts
17:35:31 <arma4> but what i'm imagining for the snowflake dev is that their main job is to keep snowflake actually working in, say, china
17:35:34 <isabela> we need to write skills
17:35:39 <isabela> so must have and nice to have
17:35:41 <arma4> and that means being on top of censorship that happens
17:35:45 <isabela> not tasks
17:35:49 <t0mmy> arma4 right so it's a reactive position
17:35:50 <isabela> i mean we can list some tasks
17:35:55 <t0mmy> isabela aha, filling this out would be super
17:35:58 <isabela> but general ones that they will be executing at the job
17:36:02 <arma4> yep. skills and abilities.
17:36:20 <arma4> i agree we should fill out this list for the job descriptions. i think we won't succeed at doing it during this meeting though.
17:36:33 <dcf1> We could use some better monitoring/metrics, something automated from the broker
17:36:38 <dcf1> building that could be part of the job
17:36:58 <asn> arma4: agreed
17:37:20 <arma4> ok. two things to do for the future meetings: (a) actually get a handle on all the known tasks and try to organize them into a roadmap,
17:37:21 <dcf1> Another necessary ongoing skill is coping with upstream changes to the webrtc library
17:37:22 * asn adds it to action items in the end
17:37:33 <arma4> and (b) flesh out the two job description visions
17:37:43 <dcf1> It doesn't have a stable API and it's an unpredictable amount of work when there's a new Chromium release.
17:37:50 <arma4> dcf1: ah ha, right, the surprises won't just come from the censor, they will come from chrome too.
17:37:50 <isabela> but we can paste there what was said so far
17:37:53 <t0mmy> okay, so I'll pause on the job descriptions for now?
17:37:54 <isabela> we dont need to loose it :)
17:38:00 <arma4> isabela: yes please
17:38:39 <dcf1> An OONI test for WebRTC could be useful, though there are many details to consider. Let me find a opst.
17:39:03 <isabela> dcf1: ^^ this
17:39:12 <isabela> we can add as taks for ooni at the other spreadsheet
17:39:17 <isabela> k
17:39:18 <dcf1> https://lists.torproject.org/pipermail/ooni-dev/2017-March/000496.html
17:39:32 * isabela leaves the job description pad alone :)
17:39:42 * t0mmy does likewise
17:39:43 <asn> yes. if we want a dev that responds to incidents, then we need a way to detect incidents. otherwise the dev will just sit around and wander.
17:39:44 <isabela> t0mmy: please bug people over the week so folks dont forget about it :)
17:39:54 <arma4> ok. second try at my item 2 from the agenda. :) as i understand it, we use the data channel in webrtc. and not much else does -- hangouts does not, for example. and i think it's possible/easy to tell on the wire whether it's using the data channel or the other channels.
17:40:00 <t0mmy> isabela ah yeah, we can work btw meetings
17:40:01 <arma4> what was our vision for how that arms race proceeds?
17:40:05 <isabela> t0mmy: also we should try to bug all the original thread again with updates on these meetings and these tasks
17:40:07 * dcf1 is trying to help with job description, by brainstorming chronic maintenance tasks
17:40:13 <t0mmy> isabela +1
17:40:19 <arma4> because the bad version of the vision is that we roll this out, and then censor blocks webrtc data channel, and then oops.
17:40:20 <isabela> dcf1: :)
17:40:21 <t0mmy> will do
17:40:34 <isabela> t0mmy: tx
17:41:11 <asn> i think dcf1 and arlolra only can answer the future of arms race discussion
17:41:15 <dcf1> For some reason the rest of that OONI/WebRTC thread is not linked
17:41:16 <dcf1> https://lists.torproject.org/pipermail/ooni-dev/2017-March/000497.html
17:41:26 <asn> i can just shrug and say "let's move from data channel of webrtc, to the one that hangouts uses"
17:41:49 <arma4> asn: if we just spent a bunch of work ripping out the parts of webrtc that aren't the data channel, though, ...
17:41:58 <dcf1> asn: it's not quite that easy
17:42:25 <arlolra> arma4: I did that already in https://trac.torproject.org/projects/tor/ticket/19569#comment:1
17:42:52 <blanu> Detectability of data channel vs AV channel is something that could be tested for with Adversary Lab. Alternatively, you could have someone dissect packet traces.
17:44:02 <arma4> but yes, dcf1 and arlolra are the ones to answer this one. is there a plan already planned? or was it "step one, make the data channel work"?
17:44:17 <blanu> I think using the AV channel for application data will be some significant work.
17:44:20 <dcf1> asn: Hangouts doesn't use a plain data channel or plain media channel, but something else custom (last time I checked)
17:44:37 <asn> ouch
17:45:07 <dcf1> asn: https://arxiv.org/pdf/1605.08805v1.pdf
17:45:09 <asn> is hangouts our main target in terms of hiding behind it?
17:45:11 <dcf1> "Some WebRTC applications that use
17:45:11 <dcf1> SRTP make use of an older type of key exchange called
17:45:14 <dcf1> SDES [2]—in this case no DTLS handshake occurs."
17:45:19 <dcf1> Hangouts is one such application, afaik
17:45:24 <asn> hello paper
17:45:44 <blanu> Oh what a nice paper.
17:46:05 <dcf1> asn, you might also like https://www.bamsoftware.com/papers/thesis/#sec:webrtc-fingerprinting
17:46:11 <Samdney> +1
17:46:32 <arma4> ok, so those look like they argue it's a hard problem
17:46:50 <dcf1> arma4: maybe not so much hard, as unknown at this point
17:46:59 <dcf1> quantifying how hard may actually be the hard problem :)
17:47:05 <arma4> dcf1: can you paint a picture for us, briefly, about how you imagined the next steps going?
17:47:21 <dcf1> arma4: or censor-circumventor interaction?
17:47:32 <arma4> because if we're pretty sure we're going to need to do the future steps, no time like the present to think them through
17:47:39 <arma4> yes. of "they block this so we move to that"
17:48:22 <dcf1> Likely first step (arlolra check my intuition) is the censor blocks the STUN server
17:48:39 <dcf1> https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/tor-browser/Bundle-Data/PTConfigs/linux/torrc-defaults-appendix
17:48:47 <dcf1> "-ice stun:stun.l.google.com:19302" that one
17:48:52 <arma4> poor google
17:49:02 <arma4> (so sad)
17:49:23 <dcf1> We haven't put a whole lot of thought into the STUN server, whether the one we've chosen is actually hard to block, what other applications use it, etc.
17:49:49 <dcf1> It is also the case that you can use any old STUN server, doesn't have to be a specific one--though of course if you have to change frequently
17:49:59 <dcf1> you want something more agile than a torrc file in Tor Browser
17:50:38 <Samdney> what would be a more agile solution?
17:50:52 <dcf1> I see that as the most obvious step for some engineer who gets the instruction to analyze and block it
17:51:00 <asn> load stuff off the network? but then they can block that?
17:51:23 <arma4> if we assume domain fronting continues to work, then we can use it for low bandwidth stuff like getting a fresh stun server suggestion
17:51:32 <dcf1> Samdney: I don't know, a local database of many servers, maybe there's some kind of local client autodetection we can do
17:51:44 <dcf1> arma4: yes, potentially.
17:52:16 <dcf1> arma4: also I'm jazzed about an idea to use DNS-over-HTTPS for this kind of rendezvous information: https://trac.torproject.org/projects/tor/ticket/25594#comment:1
17:52:36 <arma4> fun. like cloudflare's 1111
17:52:49 <dcf1> yeah
17:53:18 <arma4> ok. imagine we solve the stun server question. what then?
17:53:35 <dcf1> So another likely next step, as I see it, is the aforementioned fingerprinting of the WebRTC implementation, or snowflake's use of data channels (the Fifield-Gil Epner tech report)
17:54:27 <dcf1> That one, it's hard to say how it will go, because so much depends on what kind of WebRTC exists in the wild
17:54:46 * arlolra tacitly trusts dcf1's intuition on all this :)
17:54:54 <dcf1> We don't have a https://tlsfingerprint.io/ for WebRTC.
17:55:24 <dcf1> https://tlsfingerprint.io/top/ is maybe more illustrative of what we would love to have
17:55:47 <arma4> and, from the other side, what the censor would love to have too, for their own network
17:56:09 <dcf1> Another thing that is hard to predict, is how diverse the Snowflake proxy IP addresses will actually be
17:56:12 <dcf1> arma4: yes, true
17:56:31 <arma4> ah ha. maybe we have 100 snowflakes and they just get blocked, and nobody else cares.
17:56:42 <arma4> or, maybe they're all on comcast, so comcast gets blocked
17:56:54 <dcf1> Maybe we only have a few hundred, and maybe they are mostly all static proxy-go or Cupcake isntances, and they just die from incremental blocking over time.
17:57:09 <arma4> in theory they can do multi-characteristic blocking too, like "comcast and stun" or "comcast and webrtc"
17:57:18 <dcf1> Yes, the nature of the AS is also a consideration
17:57:38 <dcf1> But: looking at it another way, you have basically the same situation with normal Tor bridges,
17:57:51 <asn> but it's easier to setup snowflake proxies than bridges right?
17:57:54 <dcf1> excapt that it's orders of magnitude easier to start a snowflake proxy
17:58:02 <arma4> right
17:58:03 <t0mmy> heh
17:58:06 <dcf1> (Maybe not easier to actually get people to *keep* them running, though)
17:58:13 <asn> true
17:58:46 <dcf1> So if you make a blog post, maybe you get thousands of new Snowflakes, whereas you might get dozens of new bridges, because people don't have to rent a VPS or whatever.
17:58:46 <asn> seems like someone needs to setup a community of snowflake proxies too. like with blog posts and stuff. not sure if that's the job of the developer or the PM.
17:59:06 <arma4> ok. so let's say webrtc data channels are considered by the censor to be an acceptable loss.
17:59:22 <arma4> asn: it's the community outreach team
17:59:26 * asn needs to go in a bit. i still have #endmeeting rights.
18:00:17 <dcf1> I shudder to mention it,but without data channels you could still conceivably encode a reliable stream into audio/video.
18:00:33 <dcf1> Like https://censorbib.nymity.ch/#McPherson2016a https://censorbib.nymity.ch/#Houmansadr2013a https://censorbib.nymity.ch/#Barradas2017a
18:00:42 <asn> arma4: you want me to #endmeeting, and you start it up again?
18:00:44 <arma4> ok. the remaining items i was hoping to get to are: (a) more understanding of the arms race roadmap. (b) somebody to assure me that all hope isn't lost for a tor browser snowflake on android, and (c) we should decide if we like thursday meetings
18:00:46 <t0mmy> before we wrap up -- is everyone clear on next steps? can we assign the action item tasks to people/groups? does it suit/make sense to meet next Thursday at this time again?
18:00:58 <t0mmy> oh sorry, arma4, go for it
18:01:30 <blanu> There is probably more low-hanging fruit in terms of DPI blocking of snowflake until you get to blocking all data channel traffic.
18:01:33 <arma4> it sounds like we should keep chatting again on the arms race roadmap question. because i am nervous that we don't have it mapped out very far.
18:01:55 <arlolra> yeah, we didn't talk about all the fun ways to mess with the broker
18:01:55 <arma4> blanu: right, meaning, they block on a particular characteristic, so we change that characteristic, and there's some back and forth for a while
18:01:55 <dcf1> I don't have very many ideas beyond what we've discussed
18:02:01 <isabela> arma4: i think we can answer some of these questions next meeting
18:02:33 <arma4> ok. sounds like we need a censorship arms race task force. maybe once we have more experts in our group.
18:02:38 <isabela> maybe we need to 1. agree on the next meeting 2. send a call to it listing these questions and the other tasks we spoke of
18:02:45 <isabela> like the job description pad
18:02:50 <arlolra> re: android, we have an arm build as a start (https://github.com/keroserene/go-webrtc/blob/master/lib/libwebrtc-linux-arm-magic.a)
18:02:59 <isabela> great
18:03:08 <arma4> arlolra: oh good, so it is a thing that we are hoping will work. great.
18:03:26 <asn> ok guys im off. i #endmeeting. sorry about that! keep it up!
18:03:28 <asn> #endmeeting