16:00:12 #startmeeting tor anti-censorship meeting 16:00:12 Meeting started Thu Apr 22 16:00:12 2021 UTC. The chair is cohosh. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:12 Useful Commands: #action #agreed #help #info #idea #link #topic. 16:00:22 hi 16:00:22 hi everyone 16:00:40 here is our meeting pad: https://pad.riseup.net/p/tor-anti-censorship-keep 16:01:15 hi 16:02:10 dcf1: looks like that first announcement is yours 16:02:42 The security audit I talked about a few weeks ago is over. There are a few discovered issues affecting Snowflake, but nothing too major. 16:03:24 I've sent the full report to cohosh privately. Some of the issues affected software dependencies, so right now I'm in the process of reporting those upstream. 16:03:35 When that's done I'll make the full report public. 16:04:09 nice \o/ thanks dcf1 for filing those issues 16:04:56 i see one of them also has a fix already 16:05:29 yeah but I think snowflake!31 should be merged first, since they affect the same code 16:05:48 which I only realized yesterday while reviewing !31 16:06:01 nice 16:06:15 cool, i read your review this morning thanks for that 16:07:30 i'm impressed with how thorough the security audit was but i also don't have much experience with them 16:09:09 okay the next discussion item is for the Tor Browser 10.5 stable release 16:09:15 it is looming 16:09:35 and the browser team would like us to make a decision on whether it's time to put snowflake in stable 16:10:14 a major consideration the last time we brought this up is that putting it stable will increase the users by a lot 16:10:50 and perhaps things will break that we didn't notice before because our load wasn't high enough to break them 16:11:07 this was part of the motivation for snowflake#40007 16:11:52 here's some usage metrics for our snowflake bridge: https://metrics.torproject.org/rs.html#search/flakey 16:12:40 and usage over the last 3 months: https://metrics.torproject.org/userstats-bridge-transport.html?start=2021-01-22&end=2021-04-22&transport=snowflake 16:13:02 it has increased by 2 or 3 times what it was the last time we had this discussion 16:13:08 Here's what it looked like when meek moved from alpha to stable (2014-10-15) 16:13:11 https://metrics.torproject.org/userstats-bridge-transport.html?start=2014-07-01&end=2014-12-31&transport=meek 16:13:51 ooo nice to have that to compare against 16:14:21 snowflake has more moving parts than meek for sure 16:15:14 otoh, it is at a place where it's useful for some users and we might lose meek soon 16:15:41 at some point we will have to take the plunge 16:15:54 what are we most worried about in terms of capacity? 16:16:16 is it number of volunteer proxies, bridge bandwidth, anything we can anticipate? 16:16:33 we're definitely short on unrestricted proxies (proxies for users with symmetric NATs) 16:16:54 can we get some more of those by doing an 'install the goflake' campaign? 16:17:07 (sorry, i hope i didn't just rename it ;) 16:17:07 yeah, probably 16:17:10 could that be caused by probetest malfunctioning (snowflake#40039) 16:17:18 lol arma2 too late now :P 16:17:25 dcf1: no i don't think so 16:17:46 i looked at our metrics and they haven't dropped that much during the malfuncion 16:18:02 i also merged some code a bit ago to not have proxies forget their unrestricted status just because probetest fails 16:18:36 is there only one meek bridge? 16:18:40 so proxies will stay unrestricted until probetest is back online and tells them they aren't or enough restricted clients fail to open a datachannel connection 16:19:27 arlolra: yes, there's just one. There used to be one bridge per CDN (meek-google, meek-amazon, meek-azure). 16:19:51 how does the hardware for that bridge compare to snowflake's? 16:20:19 here's the meek bridge: https://metrics.torproject.org/rs.html#details/8F4541EEE3F2306B7B9FEF1795EC302F6B84DAE8 16:20:57 hm, i don't have access to it and I'm not sure of the specs 16:20:59 I don't know. Team Cymru runs it and I don't know the specs. But for a while I ran one of the bridges on an unexceptional VPS and it wasn't a problem. 16:21:32 i'm more worried about the broker than the bridge 16:22:05 since the last time i checked it looked like the broker was getting a lot more requests than i'd expect from the usage counts 16:22:41 requests, like from censored users? meaning more people are *trying* to run snowflake than *succeeding* at running it? 16:22:45 I'm wondering if there are some client deployments out there that use -max 3 or something, requesting a lot of proxies that they don't use. 16:23:16 yeah we also have that open issue where our standalone proxies showing a lot of extremely short connections 16:23:24 maybe this is an enumeration attempt 16:23:32 https://lists.torproject.org/pipermail/tor-dev/2014-September/007482.html [tor-dev] Call for a big fast bridge (to be the meek backend) 16:23:43 but it would also be explained by -max 3 16:24:44 you can always count on some research group doing the enumeration attempt at getting an ndss paper out of "you didn't do ticket 40055 yet". extra sad because when you see an attempt there's a good chance it's a grad student, not an actual censor. :/ 16:24:59 fwiw team cymru was not particularly responsive to use asking them to update software in the last two years 16:25:07 *to us 16:25:17 but we had no other complaints afaik 16:25:30 I guess the question is, if Snowflake isn't in 10.5, do we have an idea of what concrete work would put it in a good position for the release after that 16:25:40 (if we wanted to find a new snowflake backend bridge i bet we'd have some options. but also there's the "if it ain't broke" reason for keeping it) 16:25:59 i mean there's a lot of improvements i want to make but i'm leaning towards just going for it and seeing what happens 16:26:05 i vote for 'yes put snowflake in 10.5 stable', on the 'if not now when' argument. 16:26:16 unless there's an actual answer to the 'if not now when' question :) 16:26:20 If we don't have an idea of what work would change the situation, might as well release sooner than later 16:26:23 yeah same 16:26:37 cool let's do it! 16:26:44 and then take this list of things we just brought up, and make sure we're in a position to watch for whether they go too wrong 16:27:13 i think i'll clear time for when 10.5 ships to watch real time snowflake metrics 16:27:31 cohosh: (sorry, lurking :) can the broker be scaled up relatively quickly? or is it a fixed resource? 16:27:34 we should have some more alerts set up by then too which is good 16:28:37 sysrqb: it's addressed by TLS and DNS name, so we can replace the hardware with a more powerful machine easily, but it would not be so easy if it got so big we had to distribute it across multiple machines 16:28:44 i wonder if it makes sense to wait until TB 11 so we have a little more capacity to deal with fires. 16:29:04 dcf1: great, that was the answer I was hoping for. thank 16:29:05 i'd rather not wait for that reason 16:29:05 s 16:29:36 (re capacity) 16:29:52 because i'm theoretically supposed to be funded to work on snowflake 16:29:56 ok 16:29:58 and this is in scope for that 16:30:18 and what better way to find out what's next on the list of bugs than having users find them for us :) 16:30:41 yup 16:32:43 cohosh: did that answer your question? anything more you want help brainstorming about how to monitor or how to predict? 16:33:03 with these hesitations, are we assuming a soft launch? 16:33:21 yeah i think we should ship it 16:33:34 arlolra: what do you mean by a soft launch? 16:33:46 ship it but don't yell about having done so 16:34:30 hmm, maybe that's a good idea 16:35:19 i could go either way on that one. good idea because the build-up might be more gradual, bad idea because then you don't know how the build-up is going ("did it happen yet?") 16:36:05 It will be one of the default bridge options 16:36:35 and we announce it on the release post 16:37:32 i have some doubt that announcing it on the blog will make a large difference for usage on censored networks 16:37:42 yeah 16:37:54 like, users in China will know about it and use it, or not 16:37:57 so maybe we just wait a bit to let the comms team do their thing outside of that 16:38:05 but i assume training and word of mouth will be more mportant than the blog 16:38:19 yeah okay i like the soft launch idea 16:38:25 right, i was thinking something similar: the build-up in china will happen suddenly when the right person qq's the right other person 16:39:06 (or whatever it is the cool kids in china do these days) 16:39:33 i'm optimistic about the role of snowflake in e.g. turkey too 16:40:13 yeah I'm excited about this 16:42:02 okay cool, so sysrqb i'll talk to you closer to the release date about any last patches we need 16:42:12 and figure out when we need to get those in by 16:42:51 any more discussion before we assign reviews? 16:43:47 * cohosh looks at needs help with 16:44:14 i'll do another round of snowflake!32 16:44:23 thanks dcf1 16:44:37 i wrote a shim for moat in go for bridgedb#32276 16:44:46 it takes a lot of code from goptlib 16:45:30 because it's basically the other half of it 16:45:48 but it's not high priority for a review and can wait a bit 16:46:04 ok I probably will not get to it this week 16:46:13 yeah no worries 16:46:44 arma2: if you have time to take a look at censorship-analysis#40002 and see if there's anything it's missing that'd be cool :) 16:47:10 not sure how involved you want to be in the analysis stuff 16:47:13 on my list! (now) 16:47:40 i probably have a different perspective from other people, because of how many times i've done this, so yeah if i'm around it makes sense 16:47:55 yay there's a link to writeup in a comment i tagged you in 16:49:33 that's it from me 16:49:45 anything else for today? 16:51:02 are there good instructions for how volunteers can set up a goflake? we should get them a home on community.tpo 16:51:20 they could probably be better 16:51:42 and they could be linked from snowflake.tpo too now that i'm looking 16:51:48 that's a good idea, and while we're at it making sure the logs are useful enough 16:52:11 Jacobo Nájera has been doing it independently and asking questions 16:52:15 https://lists.torproject.org/pipermail/anti-censorship-team/2021-March/000141.html 16:52:24 https://lists.torproject.org/pipermail/anti-censorship-team/2021-April/000165.html 16:52:44 yeah that has been great 16:53:04 this is where our wiki lives now with the current instructions: https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/wikis/home#how-to-run-a-snowflake-proxy 16:53:09 but this is hard to find 16:53:54 https://snowflake.torproject.org/#extension next to 'install in firefox' and 'install in chrome' could be a good spot 16:53:58 we also have a docker container 16:54:02 oh. that sounds like a nice task for the outreachies 16:54:46 emmapeel: oh nice, i like that suggestion 16:55:11 i'll create a ticket after the meeting ends and include the points we just brought up 16:55:18 i will open a ticket , but maybe gus has an idea of where to go. community.tpo sounds good tho. ill open it there 16:55:26 There's also an outdated version of the instructions (copied from Trac) that no one on this team has edit permissions for, I think 16:55:27 ok then, less work for me :D 16:55:29 https://gitlab.torproject.org/legacy/trac/-/wikis/doc/Snowflake/ 16:55:49 It would be nice to delete it or replace it with a link to the current page 16:55:55 yeah :/ the gitlab wiki situation isn't great 16:56:02 https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/19409 might be easier now that libwebrtc is not in the mix 16:56:23 i've had it on my todo list to ask gaba or anarcat who has permissions for the legacy wiki pages 16:56:41 arlolra: ah actually i don't think yet :/ 16:56:58 i started this process after we'd switched to the go version 16:57:02 i just found this https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40031 16:57:19 and since you have to package each go library dependency individually it is very difficult to do 16:57:27 and to update when we bump the version of webrtc 16:57:29 i see, ok 16:58:07 emmapeel: ah yay it already exists :) 16:58:30 i'll just add to that ticket then 16:59:43 okay i think i'll end the meeting here since we're at the hour mark 16:59:57 cohosh: yeah. i am not sure that support.tpo is the best place for these instructions tho... 17:00:21 emmapeel: yeah, community.tpo is a better pick now that it exists 17:00:25 emmapeel: that's fair, we can update the ticket title 17:00:36 and actually, snowflake.tpo could be the right place for the instructions, and then community.tpo could point to them. several options. 17:01:03 (or vice versa. just don't try to maintain two copies and it will work out :) 17:02:01 yeah let's continue discussion on the ticket 17:02:14 #endmeeting