15:59:15 #startmeeting tor anti-censorship meeting 15:59:15 Meeting started Thu Feb 17 15:59:15 2022 UTC. The chair is meskio. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:59:15 Useful Commands: #action #agreed #help #info #idea #link #topic. 15:59:20 hello everybody! 15:59:24 here is our meeting pad: https://pad.riseup.net/p/tor-anti-censorship-keep 15:59:35 feel free to add what you are working on and put items in the agenda 15:59:45 hi! 16:00:47 we have one point to discuss: obfs4proxy-0.0.12 causing connection failure warnings, before eventually succeeding 16:00:58 https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/40804 16:01:14 * meskio is reading the issue, but please jump in if you know about it 16:01:38 This is something new with the upgrade to obfs4proxy-0.0.12 16:01:58 Hi~ 16:02:06 0.0.12 has a change to fix a distinguisher in the elligator encoding of public keys during the obfs4 handshake 16:02:29 regrettably I still haven't investigated to understand exactly what the nature of this distinguisher is 16:03:08 but since then, it seems all obfs4 bridge users have been seeing "unable to connect to X.X.X.X:443 ("general SOCKS server failure")" in the tor log, sometimes multiple times, before getting a working connection 16:03:38 uff :( 16:03:42 the change in obfs4proxy-0.0.12 is supposed to be backward compatible (I can find documentation saying so in a second), but perhaps it is not fully 16:04:18 Will this error go away if we upgrade both client and server to the same version? 16:04:44 so it succeeds eventually? 16:04:45 that will be tricky, to get every bridge operator to upgrade 16:04:50 my guess: obfs4proxy-0.0.12 client with an older server has something like a 50% probability of handshake failure and causing "general SOCKS server failure"), so you get an expected 2 tried before a connection works (geometrically distributed) 16:05:02 ah 16:06:18 Although I haven't looked into the elligator implementation problem, I suspect it is of a similar nature: actually indistinguishable some fixed fraction of the time, distinguishable with some probably the remainder of the time 16:07:15 it's killing me that there's not a general technical understanding of what the bug actually is, and I think I or someone is going to have to actually figure it out 16:07:33 shelikhoo: I don't know if upgrading the server fixes it 16:07:37 AFAIK currently censors are not doing probabilistic attacks, so we should be fine to roll back if needed 16:07:42 but we should fix that problem 16:08:10 cohosh: it seems like it succeeds eventually; I don't know exactly what mechanism causes tor to rety the bridge 16:08:13 I have in my queue of things to do to get my hands dirty with obfs4, I could give it a try to this problem 16:08:22 https://packages.debian.org/sid/obfs4proxy 0.0.13 is now in debian 16:08:25 but I have to recognize that I had no idea what I'm getting into 16:08:43 irl: I doubt 0.0.13 fixes the problem, it just removes utls 16:08:56 BTW, we might not want to upgrade to 0.0.13 in TB 16:09:05 right, if there is more than has to be fixed there then we should figure out what it is 16:09:13 and make sure that the fix is also going into debian 16:09:28 one of the difficulties is that the academic paper that introduces elligator is notoriously hard to read and understand 16:09:28 sure, first figure out what is needed there 16:09:34 dcf1: re the elligator bug, we talked to aaron johnson of nrl, who seems to have a handle on it 16:09:39 i can introduce you if that would be useful 16:10:00 he told us that indistinguishability is reduced by a factor of 8. and two factors of 8 if both sides haven't upgraded. 16:10:19 Loup Valliant (who worked with yawning to make the 0.0.12 fix) has made blog posts that are a lot more readable 16:10:29 https://loup-vaillant.fr/tutorials/cofactor about the bug specifically I believe 16:10:38 https://loup-vaillant.fr/articles/implementing-elligator on elligator generally 16:10:57 dcf1: We need to first understand the exact nature of this connection failure bug, and try upgrading the both the server and the client then observe consequence is one of the way for us to understand this. 16:11:28 Also https://www.reddit.com/r/crypto/comments/fd9t3m/elligator_with_x25519_is_broken_what_workaround/, almost 2 years old now 16:11:30 If it does not go away with updating both the client and the server, then this issue is not related to server 16:12:01 arma2: thanks, that would be helpful. Okay, a factor of 8 is in line with my intuition. 16:12:24 (2 factors of 8 = factor of 64, to be clear) 16:12:35 It might be just a bug introduced in client, than handshake version compatibility issue 16:13:07 I agree with you shelikhoo we need to test that 16:13:12 dcf1: ok i will mail you and aaron. and cc shell and meskio 16:13:21 thanks 16:13:48 and yes, understanding the failing-to-connect bug sounds super useful. we had a user in #tor yesterday who was experiencing it (in retrospect) and we just told him to keep trying 16:13:49 I guess here we should first decide if this is a reason to roll back to a previous version in TB while we try to fix it 16:13:57 https://gitlab.com/yawning/obfs4/-/blob/77af0cba934d73c4baeb709560bcfc9a9fbc661c/internal/README.md 16:14:08 "Representatives created by this implementation will correctly be decoded by existing implementations." "As the representative to public key transform should be identical, this change is fully-backward compatible (though the non-upgraded side of the connection will still be trivially distinguishable from random)." 16:14:48 So the author says it's backward compatible; maybe that's not actually true for some mathematical reason, or there could be an implementation bug. 16:15:41 I'm thinking a little bit about the potential added distinguishability of a client making multiple repeated connections to the same bridge (not really the overriding concern when using default bridges though) 16:16:39 does the tor browser obs4proxy auto retry these connections? or if there is one bridge does it just sit there 16:16:57 (until you try a new application level request, and then it marks the bridge up and tries again, once) 16:17:17 I'm not sure if it is tor or obfs4proxy that is retrying the same bridge, but something is. 16:18:44 i'm impressed with the test suite tails has that caught this issue: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/40804#note_2778826 16:19:35 is it correct to summarize as "an upgraded client can only sometimes connect to a non-upgraded bridge"? 16:20:12 seems like we want to fill out the 2x2 matrix of upgraded/non-upgraded client/bridge 16:20:13 arma2: that sounds right -- with the caveat that we haven't tested a connection to an upgraded bridge 16:20:59 so arma2 is going to connect us with aaron johnson to help understand what is going on with the mathematical transform and whether that could be causing connection failures 16:21:17 meskio is wondering if TB should downgrade obfs4proxy until we figure it out, about that I don't really have an opinion 16:21:40 It's a few users whom the log messages bother in the issue tracker and the forum, it seems 16:22:03 I guess the visible thing for most users is taking a bit longer to connect 16:22:14 If an upgraded -> upgraded connection works, then additionally we can make plans for getting server operator to upgrade (which includes e.g. getting an upgraded version in Debian) 16:22:39 the upgraded version is already in debian 16:22:52 but haven't poked people about it, or updated the docker image 16:23:01 which debian is it? 16:23:09 sid 16:23:12 Possible idea: if a non-upgraded server is detectable from the client (or detectable some fraction of the time), we can log a message to that effect, which can lead users to choose a different bridge or get their bridge to upgrade (if it's their own privately operated bridge, ofr example) 16:23:15 we should work on getting it in bakcports 16:23:24 but I think we need to wait until is in testing 16:24:30 dcf1: yes, it is my understanding that 7/8 of the time, the client can tell, and could pass a message up the chain or something 16:25:31 great, that too is in line with my expectations. The client can do whatever distinguishability attack the censor would do. 16:26:03 that's all I had to say on this topic, I will transcribe the high points into the meeting notes 16:26:15 thanks 16:26:34 cohosh: do you remember if it was really aaron johnson who told us this? i am having trouble confirming my distant memory 16:27:01 I think it makes sense not to rollback the version for now, and see if we can fix it, or upgrade people, soonish 16:27:01 arma2: yep 16:27:23 yes you remember? and yes it was? and yes i am having trouble? :) 16:27:30 it was aaron 16:27:31 Yes, maybe understand what is happening first.... 16:27:41 awesome, thanks 16:27:46 the next topic is about the utls support we are going to have at Snowflake client for contacting broker 16:27:59 i thnk it's worth looking at our process, maybe a bit later to see how we can detect breaking changes earlier 16:28:06 should we enable it by default? 16:28:18 I can do some tests and try to reproduce it, and see if upgrading would solve the problem 16:28:19 (like how tails did it) 16:28:37 shelikhoo: what are the drawbacks of enabling it? 16:28:37 cohosh: agreed. maybe we can even team up with tails, rather than reconstructing what they have 16:29:20 the utls implemented some old browser's fingerprint, so for attackers aware of it, it will standout 16:29:39 stand out more than go TLS? 16:30:14 shelikhoo: in doing this work, did you get of sense of whether utls is well maintained? 16:30:35 i guess no if the available fingerprints are old? 16:30:47 I can't say how do they compare, but there are use of Go language other than anti-censorship 16:31:01 true 16:31:02 but utls is exclusively used for anti-censorship 16:31:59 No, the maintainers is not adding fingerprints of more recent browsers in a timely manner 16:32:44 do you have any idea how hard will be to produce a new updated fingerprint? 16:33:24 Chrome 83 is the last version in uTLS, but the main stream Chrome is at 97 16:34:09 I didn't investigate the work required for adding new fingerprints 16:34:54 I too was disappointed recently looking at the state of fingerprints in uTLS. 16:35:00 Based on common sense, adding new extensions to Client Hello isn't hard. The hard thing is supporting some extensions that Go do not support 16:35:21 shelikhoo: One suggestion is to get in touch with Psiphon; I believe they are users of uTLS and perhaps have some insight or internal tweaks. 16:36:04 dcf1: Yes, I will do that later. 16:37:41 The way TLS fingerprint issue is solve in V2 is by browser forwarding 16:38:21 it is a bring your own browser model, so users just open a URL, and the webpage will connect to both V2 and remote server 16:38:26 this is what we used to do with meek, before switching to obfs4proxy for meek in tor browser 16:38:38 then relay traffic 16:38:50 except that we used the version of firefox bundled with tor browser, not bring your own browser 16:38:57 TB is already a browser, would be nice if we could use it 16:39:14 meskio: well, it actually some with a lot of practical problems 16:39:16 ahh, so that was done before... 16:39:38 is the fingerprint of chrome the same for desktop and android? 16:39:38 one of which is that the tor browser Firefox ESR is generally older than the general population of Firefox users 16:39:58 there are so many old androids with old versions of chrome around, I don't think they can block by that fingerprint 16:40:05 shelikhoo: https://github.com/rod-hynes is the Psiphon developer I know has worked on/with uTLS 16:41:04 dcf1: Yes. I will try this contact this person if necessary 16:41:28 good, should we wait to make a decision about uTLS to see what psiphon people thinks about it? 16:41:42 shelikhoo: This is the ticket where meek-client gained the ability to use uTLS, in addition to browser forwarding: https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/meek/-/issues/29077 16:42:16 And here is where the decision was made to use obfs4proxy with uTLS and its own meek implementation, for the same of having one fewer binary to ship: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/29430 16:43:09 (Aside, if obfs4proxy-0.0.13's removal of uTLS support causes problems for the meek implementation in Tor Browser, there is always the option of going back to the mainline meek-client implementation. It still works, as far as I know.) 16:44:35 dcf1: Yes. I will have a look at these tickets to get the context. 16:44:41 hi all/4 16:44:46 That's why obfs4proxy's implementation is called "meek_lite": originally it lacked the browser camouflage feature, so it was considered lighter but more distinguishable. It became less "lite" when it gained uTLS capability. 16:44:52 Oh wow hi Serene! 16:45:15 keroserene: hello, nice to see you here 16:45:21 been a minute :) here to help if I can 16:45:23 :D 16:45:25 Hi~ keroserene ! 16:46:01 keroserene: not sure if you got the context, but we are talking about snowflake connecting to the broker using uTLS 16:46:46 I've been here since the start of today's mtg getting context, indeed, awesome progress since last... 16:46:56 :) 16:47:54 dcf1: about obfs4proxy, I think we should bring back uTLS into meek_lite 16:48:13 or I used to think it, now shelikhoo is bringing some good questions to it 16:49:02 i've asked acute to pause on uploading the obfs4proxy backport package so that we don't get a ton of users upgrading automatically to a broken thing 16:49:39 keroserene: welcome! 'indistiguishable from background while quiet' indeed :) 16:49:55 BTW ooni has seeing in the past go TLS being blocked by fingrprinting (don't recall the country) 16:50:08 I think it is Iran 16:50:42 I have received some report about this. China is also doing this to a lesser extent 16:50:59 irl: thanks, let's investigate before doing that move 16:51:35 it looks like is a topic we should continue talking next week 16:51:39 anything more about it? 16:51:44 should we move to the reading group? 16:52:31 this week we are discussing "Weaponizing Middleboxes for TCP Reflected Amplification" 16:52:39 https://censorbib.nymity.ch/#Bock2021b 16:53:15 I can try to give a summary 16:53:55 https://geneva.cs.umd.edu/weaponizing 16:54:00 the paper proposes mechanisms to use middleboxes, like censorship ones, to amplify DoS attacks 16:54:10 I'm afraid I have to split after about 5 minutes 16:54:42 they do use genetic algorithms to find out what kind of packets produce higher amplification 16:55:01 to produce attacks of up to 50,000x 16:55:29 dcf1: that is a pity, if you want we can move the conversation to next week 16:56:04 the great thing to me is that the amplification attacks use TCP, which was thought to be impossible, because an off-path attacker needs to guess sequence numbers and prevent the victim from terminating the attack with a RST 16:56:35 yep, this is amazing 16:56:43 but because middleboxes are designed to be lax about understanding protocols (because they e.g. may only see on direction of a conversation), you don't need to strictly conform to the protocol to get them to send data 16:57:20 meskio: it's okay, I read the paper and I can read the discussion backlog 16:57:25 :) 16:57:51 it was very interesting how they found so many router loops, so they could trigger the middlebox response many times with the same package 16:58:28 so like with the GFW injecting 3 RSTs for 1 SYN, you get a small amplification factor (but greater than 1.0), but with censor boxes that inject whole blockpages you can get a large factor 16:59:20 Also nice to see the cross-project use of resources, they used Quack reports from Censored Planet in the intial pass to find likely vulnerable middleboxes and the URLs that would trigger them 17:00:01 they also reuse their own genetic algorithm, used before for censorship evasion 17:01:24 something I found weird in the paper is that they are suprised to find so many middleboxes outside the 'standard' censoring countries 17:01:33 they even talk about non-censored countries like the US 17:01:37 I don't know much about the US 17:01:44 but in europe middleboxes are common 17:01:58 and countries do censor a lot of websites in internet 17:02:12 (about piracy, gambling, ...) 17:02:58 anywa, I had a lot of fun reading the paper, is cool to see the censors being trolled 17:05:00 One of the issue that is not directly related to this paper but maskio's observation is that there are censorship middle box in region with less internet censorship. Who developed these middle boxes. If these boxes are from somewhere like China, it is possible for these box to send traffic data back to China to assist censorship at home? 17:05:47 in spain one of the mayor ISPs uses fortinet middleboxes 17:05:53 Let's say a Tor bridge connect to Tor network in standard Tor protocol, and accept anti-censorship traffic 17:06:06 https://www.fortinet.com/ 17:06:44 middleboxes are everywhere 17:06:47 wich is US based 17:06:50 it is possible for adversaries to know about its traffic to Tor network, and then make restriction to its anti-censorship traffic 17:07:09 they broke the internet so much that we invented quic 17:08:12 middleboxes are often used for traffic engineering purposes so they appear anywhere you have traffic at scale 17:08:26 they also tend to be very buggy 17:08:26 but yes, there is all this paranoia if chinese hardware sends data to china, who knows, but it looks like this paranoia is also comming from intered parties in the US 17:08:46 s/intered/interested/ 17:09:33 although my interest is remove these middlebox, not replace them with US one 17:09:48 I agree 17:09:59 but I see more and more ISPs around deploying them 17:10:08 and governments asking them to do so 17:10:27 in spain most ISPs do block content by DNS, no middleboxes yet, but they are slowly catching up 17:10:48 Yes.... 17:12:39 this one is a couple of years old, but describes how every ISP in spain blocks a pro-abortion website: https://sindominio.net/sincensura/en/post/informe/ 17:12:45 (and abortion is legal...) 17:14:04 Yes... I think this meeting is about 15 minutes over the schedule... 17:14:15 yes, let's finish here 17:14:24 Yes. Thanks~ 17:14:31 #endmeeting