15:59:58 #startmeeting anti-censorship meeting 15:59:58 Meeting started Thu Dec 2 15:59:58 2021 UTC. The chair is cohosh. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:59:58 Useful Commands: #action #agreed #help #info #idea #link #topic. 16:00:03 lol 16:00:15 welcome to the anti-censorship meeting 16:00:31 here is our meeting pad: https://pad.riseup.net/p/tor-anti-censorship-keep 16:00:32 hi 16:00:57 Hi~ 16:00:58 o/ 16:01:13 hello 16:01:20 feel free to update the pad with agenda items and what you've been working on 16:01:24 o/ 16:03:47 okay, i guess the first item is yours anadahz? 16:04:06 Do we need a live discussion of https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/merge_requests/64 ? I have been watching a bit and it seems it necessitates some design changes? 16:04:26 I posted the PT 3.0 announcement in the pad, but I heard it from anadahz. 16:04:27 dcf1: yeah that would be helpful 16:04:39 let's add it as a discussion point 16:04:44 yes I wanted to add it though thx dcf1 16:05:02 I think, as usual, they are asking for comment on the new spec 16:05:37 This time there is even a Markdown version of the proposed spec :D 16:06:11 Oh right you are, I hadn't noticed. 16:06:13 https://github.com/Pluggable-Transports/Pluggable-Transports-spec/blob/main/releases/PTSpecV3.0/Pluggable%20Transport%20Specification%20v3.0%20-%20Base%20Specification%20v3.0.md 16:06:34 so is this just a proposal for v3? 16:06:45 or comments will feed into a future version of the spec? 16:07:30 cohosh: good question You can still add comments 16:08:12 what's the juicy new stuff in there? 16:09:19 ahf: https://github.com/Pluggable-Transports/Pluggable-Transports-spec/milestone/3?closed=1 16:09:30 The new stuff seems to be language specific in process pluggable transport 16:11:15 hm, ok 16:12:37 anadahz: you want comments as issues right? 16:12:54 If it helps also the updated proposals include these here: https://github.com/Pluggable-Transports/Pluggable-Transports-spec/tree/main/proposals/Completed%20proposals 16:13:08 cohosh: Whatever is easier 16:15:05 Also if you have a suggestion for easier reviewing please send it my way. 16:15:09 oh no i think the server my irc client is on is being rebooted 16:15:17 i might drop out at some point 16:15:44 thanks anadahz 16:15:44 The thing I am not quite understand is that we only have Tor in C, and Rust, but none of language specific in process pluggable transport pt spec cover these languages. 16:16:15 cohosh: According to Meetbot you are the only one that can end this meeting as you have started it. 16:16:23 too late 16:16:38 I think the lang-specific documents are for the implementers of PTs, not implementers of the managing processes like tor or ptadapter. 16:16:45 I hope she will be back to end the meeting ;) 16:17:17 So they serve the same role as pyptlib or goptlib (which also do not implement the managing process part of the protocol) 16:17:42 but at some point will be interesting to explore how to do PTs in rust as a library and how to integrate them in arti 16:17:48 Yes, that makes senses. 16:17:50 yes 16:18:18 Let's talk about snowflake!64 16:18:25 Yes 16:18:28 (snowflake forward proxy support) 16:18:46 https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/merge_requests/64 16:18:48 shelikhoo, can you summarize? It seems there have been a number of difficulties to overcome. 16:19:16 1. We need to patch a 3rd party library 16:20:10 "GitLab is not responding (502)" d'oh shouldn't have refreshed 16:20:29 the sysadmins are rebooting some stuff right now i think 16:20:30 i think anarcat is rebooting a bunch of machines 16:20:32 Tor infra is collapsing beneath our feet 16:20:34 yeah 16:20:41 :D 16:20:46 oh ok, thanks geko :) 16:20:46 the anti-censorship meeting being censored!11 16:20:57 lol 16:21:01 okay I got !64 back now 16:21:23 2. DNS issue 16:21:35 :) 16:21:37 (whew, back) 16:22:08 2.1 We need to have a default public DNS address, the one on the local machine may not be reachable for the proxy 16:22:09 shelikhoo was just summarizing the state of snowflake!64 16:22:23 i am rebooting a bunch of hosts indeed 16:23:13 2.2 On some system like MacOS, iOS, Go's stdlib have some mindset preventing DNS over SOCKS5 proxy from working, we need to document it 16:24:34 3. Some user may wants to use proxy on broker communication, but not on peer connection, this is not support by pt spec thus removed, it is what we wanted? 16:24:59 Over 16:25:23 Is there any other topic on !64 we need to discuss 16:25:31 i guess to back up a bit, do we have some background on TOR_PT_PROXY? and how absolute should sending all traffic through the proxy specified by TOR_PT_PROXY be? 16:25:58 i know of the use case tla mentioned in the ticket 16:26:21 TOR_PT_PROXY is basically a represtentation of the torrc options HTTPProxy, HTTPSProxy, Socks4Proxy, Socks5Proxy. 16:26:48 tor goes through that list in some priority order, and the first one that's set, it gives to the pt in TOR_PT_PROXY. 16:27:17 So I think the intent is that the pt would use the proxy for all it's outgoing network traffic, just as tor would. 16:27:44 Something to understand, though, is that tor's proxy settings are not really intended to be a circumvention feature, though they can be used that way, by using e.g. Shadowsocks as a proxy 16:28:26 The intended use, as I understand it, is similar to the FascistFirewall option: it means that you are on a restricted network that has to use a proxy or certain ports in order to get any connectivity. 16:29:06 Like you're in an office, and the network connection is not censored per se, but your only access is through an HTTP proxy, you don't get an external IP addresses or even a NATed one. 16:30:17 So my impression is that the failure by a PT use to the configured proxy would be treated more as a functionality failure than a safety failure; it's not on the same level as if tor failed to use the PT for some of its traffic. 16:31:03 thanks dcf1 for that background 16:31:18 When we were cooking up TOR_PT_PROXY, I thought that we had decided that tor needed to check for positive acknowledgement from the pt, in the form of "PROXY DONE" or "PROXY-ERROR", that the pt had understood the configured TOR_PT_PROXY, but I don't think that language is actually in the spec. 16:31:35 Because TOR_PT_PROXY was a later addition, it was not something in the spec from the beginning. 16:32:13 I think, in large part, the challenges are because Snowflake doesn't have the typical easy TCP-based connection model. Snowflake uses mainly UDP. 16:32:14 link to the spec: https://gitweb.torproject.org/torspec.git/tree/pt-spec.txt#n124 16:32:46 SOCKS5 (unlike SOCKS4) has protocol-level support for proxying UDP, but it may not be commonly implemented? 16:32:49 https://datatracker.ietf.org/doc/html/rfc1928#section-7 16:32:49 yeah, i was impressed that it was as easy as it was to get WebRTC working over the proxy 16:32:59 it did require upstream library changes, but 16:33:05 i was expecting that to be uglier 16:33:19 I know cohosh tried the SOCKS5 proxy in OpenSSH and it did not work, but shadowsocks-rust worked. 16:34:41 Actually FullCone UDP support in SOCKS5 proxy is not that common 16:34:55 I am still working on its support in V2Ray 16:35:06 In response to shelikhoo's point (3), I think we are in an unanticipated situation. There may in fact be a use case for not proxying the peer connections, but there's no way for the PT protocol to express that except as perhaps a SOCKS argument or command-line argument. 16:35:40 I too am impressed with shelikhoo's results so far on this challenging task. 16:36:27 how do browser handle webrtc traffic if a proxy is configured? do they send the UDP packets over the socks proxy? 16:36:34 Normally with a TCP connection, we would effectively proxy the DNS request by giving a hostname to the proxy, rather than an already resolved IP address. 16:36:36 Actually I have been researching to get WebRTC proxy support into V2Ray for a while, so it is basicly copy and paste from my existing codes. 16:37:07 But apparently we use some APIs that internally make DNS queries and it is not trivial to proxy those, is that right? 16:37:08 No, browser do not support this 16:37:35 Chrome connect to peer directly without proxy leaking user ip address 16:37:43 mmm 16:37:48 Firefox just block webrtc connection 16:38:14 now a days it looks like webrtc is so comonly used that I will be surprised many networks actually block it's use 16:38:32 dcf1: I have already patched the library again, so it will work on most platforms 16:38:45 on MacOS/iOS there will be an issue 16:38:46 One way to control proxying of DNS queries is to set net.DefaultResolver (https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/merge_requests/64#note_2763509), but that is somewhat fragile as it's global state that affects everything (could be a problem if snowflake is used as a library) 16:39:02 * meskio goes to read the issue in snowflake 16:39:07 The macos/ios problem is new to me, what is that about? 16:39:23 https://go.dev/src/net/conf.go#L73 16:39:28 I have solved the global state issue already 16:39:30 https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/merge_requests/64#note_2763905 16:39:34 it is just go 16:39:46 stdlib have issue with macos 16:40:04 It have a fixed mindset that these query must be done via OS API 16:40:05 shelikhoo: I understand, my point is that there may be different design ways of solving these problems, I am trying to provide some context 16:40:25 and brainstorm alternatives, because it seems any solution has benefits and drawbacks 16:41:03 Setting that global state won't solve this unless we perform DNS manually 16:41:09 it's interesting that our initial reason for working on this was to get snowflake working on iOS 16:41:12 Setting that global state won't solve this 16:41:29 unless we perform DNS manually, this won't be solved 16:42:36 We can just import a DNS library and replace go's stdlib to avoid its incorrect mindset 16:43:30 what if we configure STUN servers as IP addresses in the client? 16:43:51 and the broker 16:43:54 It seems one of the difficulties of solving the problem properly is the need to vendor/path outside libraries and add new dependencies? 16:43:55 This can be used as an workaround on MacOS and iOS 16:44:16 (by default) 16:44:40 Oh, so it's not really API's internal use of DNS, it's actually us calling e.g. net.ResolveUDPAddr? 16:44:45 Like this? https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/merge_requests/64/diffs?commit_id=529a83e10ba0ffde84b8063b80867ac653cb62c9 16:45:29 Yes, pion webrtc call net.ResolveUDPAddr as well. I have patched it 16:45:34 So we can actually fix all those holes by doing our own resolution? I'm having trouble understanding the current state of things. 16:46:15 Yes, I think we can fix MacOS/iOS issue by doing our own resolution 16:47:11 ok. yes, in that case I don't understand why there would be a problem with "On some operating systems, Go standard library will ignore programmer's setting and always use System DNS resolution resulting in DNS Leak." Since we would not ever be giving the standard library names to resolve. 16:47:29 Same as with TCP-based proxying, we give the proxy a hostname, the go stdlib never knows about it. 16:48:14 Yes, so we decide to solve this by introduce a new library to do DNS ourself? 16:49:39 I am not sure how I feel about it. It's not a pleasant step to have to take. 16:50:03 I am conflicted on this issue because it's a lot of work and disruptive code changes, for something that only works with a small number of proxy clients. 16:50:24 yeah it's an edge case for sure 16:50:32 But if we do it, we should do it completely and not only halfway. 16:51:44 when the issue was originally created, it was a blocker for getting snowflake working on iOS 16:52:02 which would be really nice to have 16:52:14 but my understanding is that it is not a blocker anymore? 16:52:14 Because a proxy was small enough to fit in the iOS memory limits, but all of SNowflake was too big? 16:52:19 yup 16:52:34 so it was: app -> small proxy -> big Snowflake 16:52:48 but then they raised the memory limit so app -> big Snowflake works now, as I understand it? 16:53:02 yes, that is my understanding 16:53:34 ok, thanks. I don't know if we reached any conclusions, but it helped me understand what is going on and what the considerations are. 16:53:42 I'll try and summarize the discussion in the notes. 16:55:51 okay we can continue discussion in the ticket :) 16:56:05 thanks shelikhoo for working on that, it is a really hard problem 16:56:35 next item on the agenda is... ggus? 16:56:46 yes 16:57:02 looks like some russian ISPs are blocking tor 16:57:13 we got some users asking for help on frontdesk@ and on #tor 16:57:52 i added ooni probe results on the pad, but i will open a ticket later today after the meetings 16:58:08 the tor block started today 16:58:46 Side note, perhaps with MASQUE, UDP-capable proxies will be more common in the future. 16:59:21 cohosh: You are welcome~ dcf1: Sorry for introducing distributive changes, this is the least distributive way to get things done.... 16:59:37 I can think of.... 16:59:57 ggus: oh wow :/ 17:00:12 s/distributive/disruptive 17:01:15 shelikhoo: oh no need to apologize, we might decide to hold off on merging the changes 17:01:21 Do not worry shelikhoo, we need to at least explore the space of what's possible. 17:01:22 but the work is still really useful 17:01:48 Yes.... 17:01:59 aggregate view of tor test in russia by ASN: https://explorer.ooni.org/experimental/mat?probe_cc=RU&since=2021-11-25&until=2021-12-02&test_name=tor&axis_x=measurement_start_day&axis_y=probe_asn 17:02:36 * cohosh realizes there are a lot of ASes in Russia 17:02:56 that are measured by OONI 17:04:29 I have a mr that will not be assigned by the bot: docker-snowflake-proxy!3 17:04:50 any takers? I guess cohosh is a bit overloaded, shelikhoo do you feel like? is a Dockerfile 17:04:52 ? 17:04:59 Yes! 17:05:03 I will take this 17:05:05 oh thanks meskio if we're happy with the bot, we can probably ask ahf to expand it to more repos 17:05:23 shelikhoo: thanks 17:05:37 let me know which ones you want to expand it to and i'll add that :-) 17:05:41 ggus: this blocking looks intense, the anomalies are showing that not only dirauths blocked but also all default bridges o.O 17:07:03 like going from not blocking vanilla tor to this suddenly in one day 17:08:07 https://gitlab.torproject.org/tpo/community/support/-/issues/40050 17:08:15 ggus: do we have updates from TM btw? i saw your note that some of the blocking appeared to be lifted? 17:08:16 cohosh: yeah :/ 17:08:34 cohosh: i don't have updates about TM yet 17:08:46 but in china, they are blocking tor again 17:09:07 i didn't have time to follow up and open a metrics-timeline contribution 17:09:26 Regarding Russia the blocking may be related to the new regulations, see the RIPE 83 presentation of Alexander Isavnin: https://ripe83.ripe.net/archives/video/630 17:09:55 anadahz: can you add this to that ticket? 17:09:56 And the presentation slides: https://ripe83.ripe.net/wp-content/uploads/presentations/29-Sovereign-Internet-and-Number-Resources.pdf 17:10:04 ggus: sure 17:11:35 There's recent blocking of randomized protocols in China, I know gfw-report and shelikhoo were looking into it. https://github.com/net4people/bbs/issues/69#issuecomment-962666385 17:12:21 I have recently got access to a VPS on GFW's Hyper Attention List to reproduce look like nothing proxy blocking behaviour and successfully reproduced the block of random number like protocols experiment by GFWReport. 17:12:21 4 Types of services are set up on the server: Shadowsocks-Rust, VMess+TCP, Random Stream(just random number, not a proxy), and HTTP Server. Within 4 seconds of communication from a device in China to this server, (Shadowsocks-Rust, VMess+TCP, Random Stream) are blocked. No probe is observed on the server before the blocking decision. The services are blocked by dropping packets from China to this server's specific port. The packet from this 17:12:21 server to China seems not influenced by this block. (This is different from the previous popular method of blocking that block packet from VPS's Server to China's Client) . HTTP communication is not influenced by this blocking method. 17:12:21 Preliminary tests show HTTPS traffic is not influenced by this blocking method. 17:12:21 It is currently unknown about the criteria for an IP address's inclusion in this blocking method, and the scope of this block. 17:12:22 Packet capture data and reproduce steps is available on request. 17:12:37 a little bit of context^ 17:12:43 thanks 17:13:04 anadahz: interesting. some honest slides in there 17:13:36 ggus: I don't see any effects on the graphs from China, is this from user reports? https://people.torproject.org/~dcf/metrics-country.html?start=2021-08-01&country=cn 17:13:53 Sorry, I guess we should be wrapping up. 17:14:12 ggus: p.s. thanks for making the metrics-timeline post. 17:14:29 * cohosh is interested in this discussion as well 17:15:20 was it the drop in directly connecting users? 17:15:38 I am curious as to why obfs4 seems not yet affected in China, it should have the same characteristics as Shadowsocks or VMess. 17:16:06 i guess it depends on how targeted the randomized protocol blocking is? 17:16:06 obfs4 is usually not hosted on impacted VPS provider 17:16:13 ah 17:16:47 and even on impacted VPS provider, is not easy to encounter this kind of block 17:17:22 I have contacted gfw report to exchange information 17:18:04 awesome 17:18:38 really impressive work shelikhoo 17:18:58 thanks~ 17:18:59 For the reading group, I'm interested in doing Meteor from CCS 2021 (which already happened?), but we can talk about it another time if necessary. 17:19:43 looks interesting to me, i think i'm away from meetings for the rest of the year, but it can happen without me as well :) 17:20:18 Oh, the ACM digital library version is even not paywalled. https://dl.acm.org/doi/10.1145/3460120.3484550 17:20:27 i might multi-task the meeting next week with the s28 PI meeting 17:21:29 well, we can maybe push the discussion until after jan 1 17:21:36 just have it in the queue 17:21:43 sounds good to me 17:21:51 * cohosh same 17:22:03 a side note for obfs4: If we deploy an obfs4 proxy on a server on attention list, it is very likely for it to be blocked as well, since even random stream traffic is blocked. 17:22:36 shelikhoo: have you measured whether it's affected by the same blocking? 17:22:56 shelikhoo: what is an "attention list" ? 17:23:13 anadahz: only certain foreign IP address ranges are affected by this new blocking. 17:23:26 It's a few popular VPS and hosting services. 17:23:34 "greylist" 17:23:44 aha! 17:24:04 address that not fully blocked, have get a more strict blocking 17:24:31 I have not tried obfs4, but is probably blocked. 17:24:45 I can try this later 17:25:05 based on observed behavior 17:25:55 thanks! 17:26:33 shelikhoo: Do you deploy these tests manually or do you have a script/something that does it? 17:26:45 dcf1: sorry, i was in another meeting. some weeks ago, ooni was reporting that some dir auths and built-in obfs4 bridges were working 17:26:47 in china 17:27:01 not working, rather? 17:27:23 working 17:27:32 ok 17:27:58 that's probably the hump visible in 3rd week of november at https://people.torproject.org/~dcf/metrics-country.html?start=2021-08-01&country=cn 17:28:05 https://explorer.ooni.org/measurement/20211202T163204Z_tor_CN_4134_n1_Vy5OZEKvEyn2sFpt in some ISPs is still working 17:28:10 anadahz: There is code to assist this test, but test is deployed manually. https://github.com/xiaokangwang/entropyBlockingReproduce 17:28:43 There is no incentive to automate this since the VPS on attention list is really hard to find 17:29:03 oh wow i just realized how late the meeting has gone, and that i missed another meeting heh 17:29:11 dcf1: https://explorer.ooni.org/measurement/20211202T032836Z_tor_CN_4837_n1_zu6BUxjMSaoC6xo0 17:29:55 if it's okay with everyone, i think i'll wrap it up here 17:30:09 shelikhoo: thanks for sharing. In order to run this you need a *nix shell? 17:30:20 it was some really good discussion today 17:30:39 shelikhoo: in case we are able to find another vantage point 17:31:09 anadahz: No, this should work anywhere Go+Rust runs. 17:31:50 If it is not, I can change the code to get it working easily 17:32:05 great! 17:32:16 okay good spot to end the meeting :) 17:32:19 thanks everyone! 17:32:22 #endmeeting