17:15:32 <asn> #startmeeting snowflake meeting 17:15:32 <MeetBot> Meeting started Thu Apr 12 17:15:32 2018 UTC. The chair is asn. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:15:32 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic. 17:15:39 <arma4> arlolra: ok. i just made a note at the end of the agenda, to think about what a part-time arlo job might look like 17:15:52 <arlolra> alrighty 17:16:14 <arma4> ok. item 1, how are we doing at the tasks we identified from last time as needing doing? 17:16:32 <arma4> the snowflake.torproject.org website is all set and ready i think. let me know if that isn't so. 17:16:38 <arma4> sukhe, any news on the windows build front? 17:16:54 <sukhe> ok I can go first 17:16:59 <arma4> great 17:17:10 <dcf1> Yes, snowflake.torproject.org is working, and I'm planning to move content there. 17:17:21 <sukhe> so there is progress, but the builds are not ready yet. to be clear, I am mostly building on all the work dcf1 has already done 17:17:28 <dcf1> Longer term, it will be nice to do a style guide-ificiation, like isabela said 17:17:59 <sukhe> so the webrtc builds on Windows are done (again, dcf1, arlolra) and now we are trying to build go-webrtc 17:18:37 <sukhe> there is a linking error, which we think is due to C++ ABI (webrtc is compiled with clang and go-webrtc with mingw-w64) 17:18:46 <antonela> arma4 sukhe dcf1 are you going to add more content at snowflake.torproject.org? 17:18:53 <dcf1> (This is a place where cgo actually is causing a problem, because webrtc wants to compile with clang and cgo wants to use gcc, and they have incompatible ABIs on Windows) 17:18:54 <isabela> dcf1: if you want our help ( antonela help) 17:18:56 <antonela> I'd like to provide the branded version 17:18:59 <isabela> yes 17:19:02 <isabela> ^^ this :) 17:19:08 <antonela> yes 17:19:16 <sukhe> I am currently trying to fix that. it's difficult to come up with an ETA 17:19:19 <isabela> dcf1: if you want you can provide content and anto can organize it in a design 17:19:43 <sukhe> antonela: I will let dcf1 and arlolra answer that one :) 17:19:43 <asn> dcf1: sukhe: and what's the fix wrt cgo webrtc clang/gcc? 17:19:51 <antonela> sukhe :) 17:19:53 <dcf1> antonela: Initially it will just be a copy of https://keroserene.net/snowflake/ (that's where we're migrating the page from) 17:20:06 <antonela> dcf1: yes, i have it saved 17:20:11 <antonela> all it? 17:20:34 <sukhe> asn: either fixing the linking errors or even better, build go-webrtc with clang but that has problems as well 17:20:49 <sukhe> because cgo wants to use gcc, like dcf1 mentioned 17:21:13 <arma4> sukhe: ok. is there anything we can do, in terms of people or etc, to help you there? or is the answer just that you need to beat your head against it for a while more? 17:21:15 <dcf1> antonela: yes, at first. The important part is not the text in the page, it's the JavaScript code. We need to move the JavaScript code there in order to to other tasks. 17:21:15 <sukhe> dcf1: should I focus on https://github.com/keroserene/go-webrtc/pull/69 (the wrapper) or https://github.com/azadi/tor-browser-build/commits/go-webrtc ? 17:21:22 <GeKo> dcf1: what is needed to get go-webrtc compiled with a/the clang toolchain? 17:21:37 <dcf1> sukhe: I have no clue :) I'm at the same impasse you are. 17:21:46 <sukhe> arma4: I think it's mostly that unless I am following the wrong approach but to prevent that from happening, I am keeping dcf1 and arlolra updated 17:22:15 <dcf1> GeKo: not sure what's required. I think that it is not actually supported by Go, as Hooman found out. He found a GitHub issue, let me find it. 17:22:30 <sukhe> arma4: I plan to ask "the experts" since I have tried what I could so that should help 17:22:30 <GeKo> because the medium term plan is to move to a clang/mingw-w64 toolchain anyway 17:22:44 <GeKo> because firefox needs it 17:23:06 <arma4> sukhe: right, bringing in other people seems like a smart move. if you find that we need to ask the twitters for help, for example, steph can totally do that. maybe there are less nuclear options too. 17:23:07 <sukhe> GeKo: https://github.com/golang/go/issues/17014 and https://github.com/golang/go/issues/20982 are the issues with cgo but I haven't read them in detail 17:23:14 <dcf1> GeKo: yes, agreed on clang/mingw-w64. 17:23:20 <antonela> dcf1: cool! I can work with this content. Thanks! 17:23:32 <GeKo> ok, thanks 17:24:22 <isabela> arma4: ssup with the item "- arma: figure out cookies w/ weasel" 17:24:28 <arma4> isabela: it's done 17:24:31 <arma4> all resolved 17:25:00 * arma4 is hunting through last week's pad to try to notice other things we were going to do but have missed (if you know any too, let us know) 17:25:07 <dcf1> Right, sukhe's https://github.com/golang/go/issues/17014 is the issue I was looking for, clang for cgo is not a supported configuration on windows; it's unclear how much work it will be to make it work. 17:25:09 <isabela> cool 17:25:38 <sukhe> so yeah, that's it from me, hoping to resolve this asap 17:25:48 <asn> sukhe: ack thx! 17:25:52 <asn> (kept some notes on pad) 17:26:09 <sukhe> dcf1: yeah, also related is why I didn't try the other stuff (like removing win_sdk from the webrtc builds) because I wanted us to have a working build first, then the enhacements 17:26:13 <arma4> great. we know this is a hard one, so please bring in whatever help you think you need, and/or escalate to us to try to get you that help. 17:26:13 <isabela> #23947 17:26:21 <isabela> ^^ ssup with this 17:26:27 <isabela> i dont think is in the roadmap 17:26:34 <arma4> isabela: that ticket is all done, and the remaining thing is "populate snowflake.tp.o website" 17:26:55 <arma4> and i guess part of populating the website is that they need to move the proxy code over to the new website 17:26:58 <isabela> ok 17:26:59 <arma4> and point new snowflakes at it 17:27:06 <arma4> does that involve a tor browser update too? 17:27:07 * isabela is digging 17:27:22 <isabela> GeKo: ^ 17:27:27 <arma4> i think maybe no? since snowflake clients just talk to the broker? 17:27:32 <dcf1> arma4: yes, it will require a change in the client code and a new browser update. 17:27:44 <arma4> ok. why? 17:27:45 <dcf1> But it's graceful, old versions will continue to work under the old location. 17:27:51 <arma4> still trying to get the right mental model here :) 17:28:12 <dcf1> It's in one of Tor Browser's config files 17:28:38 <arma4> don't snowflake clients (a) tell the broker they need a snowflake, and then (b) await a snowflake contacting them? 17:28:45 <dcf1> https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/tor-browser/Bundle-Data/PTConfigs/linux/torrc-defaults-appendix 17:28:56 <dcf1> "-url https://snowflake-reg.appspot.com/" that part needs to change 17:29:13 <asn> ah 17:29:20 <dcf1> arma4: oh wait, I might be confusing something. 17:29:35 <arlolra> https://trac.torproject.org/projects/tor/wiki/doc/Snowflake#Tickets 17:29:53 <arlolra> see the graph 17:29:53 <dcf1> arma4: okay yes, you are right, for just moveing from keroserene.net to snowflake.tpo, we don't need a client update 17:30:09 <arma4> whew. mental model remains intact 17:30:12 <arlolra> we're ultimately trying to change the broker 17:30:13 <mcs> who controls https://snowflake-reg.appspot.com/? is that also an issue in the long run? 17:30:18 <dcf1> but part of the reason for moving from keroserene.net is to start using our standalone (apart from APp Engine) broker, and that is the part that needs a client update 17:30:28 <mcs> ah, okay 17:30:40 <dcf1> mcs: serene controls https://snowflake-reg.appspot.com/, which is why we're trying to stop using it. 17:30:53 <arma4> so the glorious future involves running a broker not actually on appspot, and then we can use appspot to front to it, but also we can use other things to front to it? 17:31:07 <dcf1> arma4: correct 17:31:10 <arma4> great 17:31:28 <arma4> which ticket is "move the broker to a new more standalone location"? 17:31:33 <dcf1> In fact https://snowflake-reg-test.appspot.com/ (which is mine) is already fronting for https://snowflake-broker.bamsoftware.com/, which is the standalone broker. 17:31:47 <arma4> #22874 17:31:54 <arma4> okeydoke 17:32:13 <arma4> 22 mins. were there other items from last week that we should report progress on? or, on to next items? 17:32:13 <dcf1> Yeah see my comment:7 "I think we're not going to regain access to https://snowflake-reg.appspot.com/. I think the way forward is to do #23947; i.e., move the proxy-hosting page away from keroserene.net, and then we configure the proxy on the new host to use the new broker." 17:32:37 <arma4> sounds good. all that remains is to just do the things. 17:33:08 <arma4> so, my item 2 on the agenda. i wanted to understand better our plans for the arms race: 17:33:23 <asn> perhaps we should ask t0mmy what kind of specific info they need for drafting the job posting? 17:33:26 <asn> based on "t0mmy: started on a job description but need specific info 17:33:36 * arma4 backs up 17:33:37 <arma4> yes ok great 17:33:43 <t0mmy> oh no rush 17:34:04 <t0mmy> the roadmap under (4) 17:34:30 <arma4> what about it :) 17:34:34 <t0mmy> is this what I should include in a dev job description? 17:34:44 <t0mmy> these tasks, I mean 17:34:49 <arma4> ah hm. i think no. 17:34:49 <isabela> no 17:34:56 <arma4> i think these immediate tasks that we know about, we have a pretty good handle on 17:35:05 <isabela> lets look at the pad 17:35:08 <arma4> some of them will be open-ended, like "keep x working" or "make x more robust" 17:35:27 <isabela> https://pad.riseup.net/p/K4cF5vPEU7Rg-censorshipteam-jobposts 17:35:31 <arma4> but what i'm imagining for the snowflake dev is that their main job is to keep snowflake actually working in, say, china 17:35:34 <isabela> we need to write skills 17:35:39 <isabela> so must have and nice to have 17:35:41 <arma4> and that means being on top of censorship that happens 17:35:45 <isabela> not tasks 17:35:49 <t0mmy> arma4 right so it's a reactive position 17:35:50 <isabela> i mean we can list some tasks 17:35:55 <t0mmy> isabela aha, filling this out would be super 17:35:58 <isabela> but general ones that they will be executing at the job 17:36:02 <arma4> yep. skills and abilities. 17:36:20 <arma4> i agree we should fill out this list for the job descriptions. i think we won't succeed at doing it during this meeting though. 17:36:33 <dcf1> We could use some better monitoring/metrics, something automated from the broker 17:36:38 <dcf1> building that could be part of the job 17:36:58 <asn> arma4: agreed 17:37:20 <arma4> ok. two things to do for the future meetings: (a) actually get a handle on all the known tasks and try to organize them into a roadmap, 17:37:21 <dcf1> Another necessary ongoing skill is coping with upstream changes to the webrtc library 17:37:22 * asn adds it to action items in the end 17:37:33 <arma4> and (b) flesh out the two job description visions 17:37:43 <dcf1> It doesn't have a stable API and it's an unpredictable amount of work when there's a new Chromium release. 17:37:50 <arma4> dcf1: ah ha, right, the surprises won't just come from the censor, they will come from chrome too. 17:37:50 <isabela> but we can paste there what was said so far 17:37:53 <t0mmy> okay, so I'll pause on the job descriptions for now? 17:37:54 <isabela> we dont need to loose it :) 17:38:00 <arma4> isabela: yes please 17:38:39 <dcf1> An OONI test for WebRTC could be useful, though there are many details to consider. Let me find a opst. 17:39:03 <isabela> dcf1: ^^ this 17:39:12 <isabela> we can add as taks for ooni at the other spreadsheet 17:39:17 <isabela> k 17:39:18 <dcf1> https://lists.torproject.org/pipermail/ooni-dev/2017-March/000496.html 17:39:32 * isabela leaves the job description pad alone :) 17:39:42 * t0mmy does likewise 17:39:43 <asn> yes. if we want a dev that responds to incidents, then we need a way to detect incidents. otherwise the dev will just sit around and wander. 17:39:44 <isabela> t0mmy: please bug people over the week so folks dont forget about it :) 17:39:54 <arma4> ok. second try at my item 2 from the agenda. :) as i understand it, we use the data channel in webrtc. and not much else does -- hangouts does not, for example. and i think it's possible/easy to tell on the wire whether it's using the data channel or the other channels. 17:40:00 <t0mmy> isabela ah yeah, we can work btw meetings 17:40:01 <arma4> what was our vision for how that arms race proceeds? 17:40:05 <isabela> t0mmy: also we should try to bug all the original thread again with updates on these meetings and these tasks 17:40:07 * dcf1 is trying to help with job description, by brainstorming chronic maintenance tasks 17:40:13 <t0mmy> isabela +1 17:40:19 <arma4> because the bad version of the vision is that we roll this out, and then censor blocks webrtc data channel, and then oops. 17:40:20 <isabela> dcf1: :) 17:40:21 <t0mmy> will do 17:40:34 <isabela> t0mmy: tx 17:41:11 <asn> i think dcf1 and arlolra only can answer the future of arms race discussion 17:41:15 <dcf1> For some reason the rest of that OONI/WebRTC thread is not linked 17:41:16 <dcf1> https://lists.torproject.org/pipermail/ooni-dev/2017-March/000497.html 17:41:26 <asn> i can just shrug and say "let's move from data channel of webrtc, to the one that hangouts uses" 17:41:49 <arma4> asn: if we just spent a bunch of work ripping out the parts of webrtc that aren't the data channel, though, ... 17:41:58 <dcf1> asn: it's not quite that easy 17:42:25 <arlolra> arma4: I did that already in https://trac.torproject.org/projects/tor/ticket/19569#comment:1 17:42:52 <blanu> Detectability of data channel vs AV channel is something that could be tested for with Adversary Lab. Alternatively, you could have someone dissect packet traces. 17:44:02 <arma4> but yes, dcf1 and arlolra are the ones to answer this one. is there a plan already planned? or was it "step one, make the data channel work"? 17:44:17 <blanu> I think using the AV channel for application data will be some significant work. 17:44:20 <dcf1> asn: Hangouts doesn't use a plain data channel or plain media channel, but something else custom (last time I checked) 17:44:37 <asn> ouch 17:45:07 <dcf1> asn: https://arxiv.org/pdf/1605.08805v1.pdf 17:45:09 <asn> is hangouts our main target in terms of hiding behind it? 17:45:11 <dcf1> "Some WebRTC applications that use 17:45:11 <dcf1> SRTP make use of an older type of key exchange called 17:45:14 <dcf1> SDES [2]—in this case no DTLS handshake occurs." 17:45:19 <dcf1> Hangouts is one such application, afaik 17:45:24 <asn> hello paper 17:45:44 <blanu> Oh what a nice paper. 17:46:05 <dcf1> asn, you might also like https://www.bamsoftware.com/papers/thesis/#sec:webrtc-fingerprinting 17:46:11 <Samdney> +1 17:46:32 <arma4> ok, so those look like they argue it's a hard problem 17:46:50 <dcf1> arma4: maybe not so much hard, as unknown at this point 17:46:59 <dcf1> quantifying how hard may actually be the hard problem :) 17:47:05 <arma4> dcf1: can you paint a picture for us, briefly, about how you imagined the next steps going? 17:47:21 <dcf1> arma4: or censor-circumventor interaction? 17:47:32 <arma4> because if we're pretty sure we're going to need to do the future steps, no time like the present to think them through 17:47:39 <arma4> yes. of "they block this so we move to that" 17:48:22 <dcf1> Likely first step (arlolra check my intuition) is the censor blocks the STUN server 17:48:39 <dcf1> https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/tor-browser/Bundle-Data/PTConfigs/linux/torrc-defaults-appendix 17:48:47 <dcf1> "-ice stun:stun.l.google.com:19302" that one 17:48:52 <arma4> poor google 17:49:02 <arma4> (so sad) 17:49:23 <dcf1> We haven't put a whole lot of thought into the STUN server, whether the one we've chosen is actually hard to block, what other applications use it, etc. 17:49:49 <dcf1> It is also the case that you can use any old STUN server, doesn't have to be a specific one--though of course if you have to change frequently 17:49:59 <dcf1> you want something more agile than a torrc file in Tor Browser 17:50:38 <Samdney> what would be a more agile solution? 17:50:52 <dcf1> I see that as the most obvious step for some engineer who gets the instruction to analyze and block it 17:51:00 <asn> load stuff off the network? but then they can block that? 17:51:23 <arma4> if we assume domain fronting continues to work, then we can use it for low bandwidth stuff like getting a fresh stun server suggestion 17:51:32 <dcf1> Samdney: I don't know, a local database of many servers, maybe there's some kind of local client autodetection we can do 17:51:44 <dcf1> arma4: yes, potentially. 17:52:16 <dcf1> arma4: also I'm jazzed about an idea to use DNS-over-HTTPS for this kind of rendezvous information: https://trac.torproject.org/projects/tor/ticket/25594#comment:1 17:52:36 <arma4> fun. like cloudflare's 1111 17:52:49 <dcf1> yeah 17:53:18 <arma4> ok. imagine we solve the stun server question. what then? 17:53:35 <dcf1> So another likely next step, as I see it, is the aforementioned fingerprinting of the WebRTC implementation, or snowflake's use of data channels (the Fifield-Gil Epner tech report) 17:54:27 <dcf1> That one, it's hard to say how it will go, because so much depends on what kind of WebRTC exists in the wild 17:54:46 * arlolra tacitly trusts dcf1's intuition on all this :) 17:54:54 <dcf1> We don't have a https://tlsfingerprint.io/ for WebRTC. 17:55:24 <dcf1> https://tlsfingerprint.io/top/ is maybe more illustrative of what we would love to have 17:55:47 <arma4> and, from the other side, what the censor would love to have too, for their own network 17:56:09 <dcf1> Another thing that is hard to predict, is how diverse the Snowflake proxy IP addresses will actually be 17:56:12 <dcf1> arma4: yes, true 17:56:31 <arma4> ah ha. maybe we have 100 snowflakes and they just get blocked, and nobody else cares. 17:56:42 <arma4> or, maybe they're all on comcast, so comcast gets blocked 17:56:54 <dcf1> Maybe we only have a few hundred, and maybe they are mostly all static proxy-go or Cupcake isntances, and they just die from incremental blocking over time. 17:57:09 <arma4> in theory they can do multi-characteristic blocking too, like "comcast and stun" or "comcast and webrtc" 17:57:18 <dcf1> Yes, the nature of the AS is also a consideration 17:57:38 <dcf1> But: looking at it another way, you have basically the same situation with normal Tor bridges, 17:57:51 <asn> but it's easier to setup snowflake proxies than bridges right? 17:57:54 <dcf1> excapt that it's orders of magnitude easier to start a snowflake proxy 17:58:02 <arma4> right 17:58:03 <t0mmy> heh 17:58:06 <dcf1> (Maybe not easier to actually get people to *keep* them running, though) 17:58:13 <asn> true 17:58:46 <dcf1> So if you make a blog post, maybe you get thousands of new Snowflakes, whereas you might get dozens of new bridges, because people don't have to rent a VPS or whatever. 17:58:46 <asn> seems like someone needs to setup a community of snowflake proxies too. like with blog posts and stuff. not sure if that's the job of the developer or the PM. 17:59:06 <arma4> ok. so let's say webrtc data channels are considered by the censor to be an acceptable loss. 17:59:22 <arma4> asn: it's the community outreach team 17:59:26 * asn needs to go in a bit. i still have #endmeeting rights. 18:00:17 <dcf1> I shudder to mention it,but without data channels you could still conceivably encode a reliable stream into audio/video. 18:00:33 <dcf1> Like https://censorbib.nymity.ch/#McPherson2016a https://censorbib.nymity.ch/#Houmansadr2013a https://censorbib.nymity.ch/#Barradas2017a 18:00:42 <asn> arma4: you want me to #endmeeting, and you start it up again? 18:00:44 <arma4> ok. the remaining items i was hoping to get to are: (a) more understanding of the arms race roadmap. (b) somebody to assure me that all hope isn't lost for a tor browser snowflake on android, and (c) we should decide if we like thursday meetings 18:00:46 <t0mmy> before we wrap up -- is everyone clear on next steps? can we assign the action item tasks to people/groups? does it suit/make sense to meet next Thursday at this time again? 18:00:58 <t0mmy> oh sorry, arma4, go for it 18:01:30 <blanu> There is probably more low-hanging fruit in terms of DPI blocking of snowflake until you get to blocking all data channel traffic. 18:01:33 <arma4> it sounds like we should keep chatting again on the arms race roadmap question. because i am nervous that we don't have it mapped out very far. 18:01:55 <arlolra> yeah, we didn't talk about all the fun ways to mess with the broker 18:01:55 <arma4> blanu: right, meaning, they block on a particular characteristic, so we change that characteristic, and there's some back and forth for a while 18:01:55 <dcf1> I don't have very many ideas beyond what we've discussed 18:02:01 <isabela> arma4: i think we can answer some of these questions next meeting 18:02:33 <arma4> ok. sounds like we need a censorship arms race task force. maybe once we have more experts in our group. 18:02:38 <isabela> maybe we need to 1. agree on the next meeting 2. send a call to it listing these questions and the other tasks we spoke of 18:02:45 <isabela> like the job description pad 18:02:50 <arlolra> re: android, we have an arm build as a start (https://github.com/keroserene/go-webrtc/blob/master/lib/libwebrtc-linux-arm-magic.a) 18:02:59 <isabela> great 18:03:08 <arma4> arlolra: oh good, so it is a thing that we are hoping will work. great. 18:03:26 <asn> ok guys im off. i #endmeeting. sorry about that! keep it up! 18:03:28 <asn> #endmeeting