15:59:48 <meskio> #startmeeting tor anti-censorship meeting
15:59:48 <MeetBot> Meeting started Thu Oct 27 15:59:48 2022 UTC.  The chair is meskio. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:59:48 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:59:51 <meskio> hello everybody
15:59:57 <shelikhoo> hi~
16:00:04 <itchyonion> hello
16:00:10 <meskio> here is our meeting pad: https://pad.riseup.net/p/tor-anti-censorship-keep
16:00:17 <meskio> feel free to add what you've been working on and put items on the agenda
16:01:00 <meskio> mmm, today I'm getting reconnected to the pad several times
16:01:08 <meskio> I guess is my tor circuit
16:02:00 <meskio> I kept two items from last week, let's check if we need to talk about them or we just skip them
16:02:14 <meskio> blocking by TLS fingerprint in Iran
16:02:19 <dcf1> Regarding TLS fingerprint in Iran
16:02:59 <dcf1> my understanding is that Tor Browser has now deployed uTLS-supporting versions of Snowflake, but also that the fingerprints of Snowflake in Tor Browser were not being blocked
16:03:17 <dcf1> Orbot (which was being blocked) has released a beta release that has uTLS support and circumvents the blocking
16:03:45 <arma2> have we seen a recovery in snowflake bridge load? or would it be too early to see it?
16:03:54 <dcf1> my understanding is that the beta Orbot is not something that is easily installable (e.g. from an app store), you have to know about it and install it. I may be wrong about that.
16:04:02 <meskio> pretty cool for Orbot
16:04:14 <dcf1> arma2: there's no change I can see in bridge load.
16:04:19 <arma2> ah ha, so we would expect to see the load back only when they do a 'real' orbot release
16:04:43 <dcf1> A possible explanation is that it was only Orbot that was blocked, and that Orbot is still blocked for most users. But I could be mistaken about that.
16:04:51 <dcf1> correct, that's my guess.
16:05:19 <dcf1> or it could be that people found alternatives in the meanwhile and don't need snowflake as acutely as they did in the early days.
16:05:38 <arma2> makes sense. good work everybody on tracking it down to the go tls signatures.
16:05:51 <arma2> (i've been catching up on everything this week because i am doing a few presentations next week)
16:06:05 <arma2> (my slides will be public and other people will be able to use them for other presentations if they want)
16:06:49 <arma2> and yes, it makes sense that even if orbot puts out a new stable, the word-of-mouth that made everybody use orbot won't immediately return.
16:08:01 <meskio> anything else on this?
16:08:10 <dcf1> that's all from me
16:08:15 <meskio> shelikhoo: any news on the UDP and Iran investigation?
16:08:42 <shelikhoo> not yet... I was generating charts...
16:08:51 <shelikhoo> yet to got time to work on that..
16:09:23 <shelikhoo> the things we know right now it that it blocks unknown traffic
16:09:37 <shelikhoo> and there is a residual block of about 70 seconds
16:10:03 <meskio> ok, I guess we can move on then to the first new topic for today
16:10:10 <meskio> builtin bridges and their usage
16:10:30 <meskio> I have created this issue to discuss it: https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/102
16:10:48 <meskio> the situation is that we are having less and less builtin bridges, as some are retiring
16:11:04 <meskio> and were wondering how much it makes sense to search for new builtin bridges to replace them
16:11:20 <meskio> currently is becoming easier to get bridges from Conenct Assist
16:11:47 <meskio> and in Connect Assist we only use builtin bridges when we don't know anything about the country where Tor is blocked, I expect it to be mostly corporate firewalls
16:11:55 <meskio> giving settings bridges there should be fine
16:12:18 <meskio> so basically I'm looking for usecases for builtin bridges and if there is a reason to keep them around
16:12:32 <meskio> why will people manually configure them in Tor Browser?
16:12:34 <arma2> yep. i added one thought to the ticket, which summarizes as: built-in bridges are more reliable than community bridges, so until we have the subscription model working, they are genuinely better -- if they work.
16:13:03 <cohosh> Connect Assist relies on moat, right?
16:13:21 <cohosh> so if moat is blocked, at least there will be some defaults that *might* work?
16:13:21 <meskio> BTW, I'm not proposing to drop them directly, but to don't work on them too much and stop recommending them in circumvention settings
16:13:48 <dcf1> it's true, built-in bridges don't make sense for circumvention in any realistic threat model
16:13:50 <meskio> cohosh: do you expect to exist a place that blocks moats domain fronting and not the builtin bridges?
16:14:07 <dcf1> nevertheless, a 2017 study found that over 90% of bridge users used default bridges: https://censorbib.nymity.ch/#Matic2017a
16:14:22 <meskio> arma2: I agree, but that is why we distribute multiple bridges, but might be a reason to keep them
16:14:25 <dcf1> "90% of bridge clients use default bridges that are trivial to identify"
16:14:57 <dcf1> I've thought about why that might be; one reason could be censors that are not paying attention; another could be people using bridges who don't "need" them.
16:15:03 <shelikhoo> meskio: there are some company that have MITM for all TLS connections
16:15:09 <arma2> meskio: it makes me wonder what is the "mean time until failure" of 3 community bridges
16:15:14 <shelikhoo> and schools as well
16:15:25 <emmapeel> maybe you dont want people on your local network to know you are connecting to tor, and that is why you use efault bridges?
16:15:36 <shelikhoo> so TLS connection and domain fronting might have issue
16:15:44 <cohosh> meskio: it doesn't seem likely, but if something goes wrong with moat, there will be something there
16:15:59 <shelikhoo> while obfs4 don't
16:16:15 <arma2> cohosh: it is a good point that both moat and connect assist rely on the same mechanism
16:16:17 <dcf1> emmapeel makes a good point: the "censor" in many cases may be something not very sophisticated like a school firewall
16:16:22 <meskio> cohosh: valid point, yes
16:17:13 <meskio> but for when you want not to be noticed as using tor we could make an easier way to use settings bridges
16:17:17 <arma2> dcf1: we actually had a censor like that at the country level in the past: venezuela and CanTV. they blocked the public relays by ip address and thought they were done. though in that case, connect assist could have still helped you.
16:18:03 <meskio> yes, I guess the first question here is, do we want to stop using builtin bridges in connect assist and always use settings bridges for those cases?
16:18:34 <arma2> i think if the settings bridges had equal reliability to the built-in ones, i would say yes. but if they don't, then it's harder to choose.
16:18:52 <arma2> (and i think they don't)
16:19:43 <meskio> but builtin bridges are not free, they require some work from us to find good candidates and coordinate with them
16:20:10 <meskio> in theory we only distribute bridges that work on circumvention settings
16:20:23 <meskio> but we don't test how fast they are or how long they have being living
16:20:27 <meskio> maybe we should improve that
16:20:54 <arma2> right. long term, we want to improve reliability of bridges via the subscription model. so even if they are short-lived it is still ok.
16:21:20 <meskio> I'm not sure the subscription model is actually so important here, but maybe
16:21:21 <arma2> short term, understanding the dynamics of our community bridges seems smart. how long do they actually last? are we really giving out good ones?
16:22:16 <meskio> yes, let's investigate that more
16:22:22 <arma2> the bad case to avoid is that the user gets some bridges, uses tor, and suddenly their tor stops working. i don't know what they do in that case but i assume some of them close tor browser and leave.
16:22:42 <arma2> (i think what we hope they will do is open up the connect assist window and go through the process of getting new bridges)
16:23:07 <meskio> mmm that is a good point, we should look on what connect assist does when the bridge stops working, it should request bridges again but maybe it doesn't
16:23:20 <arma2> yeah, i think nothing notices that the bridges stopped working.
16:23:35 <arma2> this is one of the pieces of the big grand plan that i am calling the subscription model :)
16:23:59 <meskio> cool, I see a path forward that will take us some time, and if we keep loosing builtin bridges we should start thinking on recruiting more
16:24:13 <ggus> built-in bridges are nice because you can easily bypass some soft censorship like in universities and workplaces -- and i have seen that happening in our trainings in the global south --, but they don't provide anything against state censorship level like tm, ir, cn, ru
16:24:43 <ggus> the built-in bridges that we have doesn't seems to be overloaded
16:24:52 <arma2> ggus: right. they are nice because they are *so easy*. easy to use, easy to explain, easy to find in the interface.
16:25:14 <arma2> plus once you are using them, tor works more reliably than it does when you have found some random bridges.
16:26:58 <meskio> cool, anything else on this point? we can continue on the ticket and let's have a long term plan into that
16:27:18 <meskio> the next point in the agenda is about bridgeline size limit
16:27:54 <meskio> few days ago TB 11.5.5. included a bridgeline that was longer than what the pt mechanism supports
16:28:25 <meskio> there has being some proposals of PT specs 2.0 (and 3.0) that with many other things have fixed that problem
16:28:43 <meskio> we are not so much involved in the process of the new PT specs
16:28:59 <meskio> I guess we can live with this size limit for now
16:29:12 <meskio> but maybe we want to plan on it and start talking with arti about it
16:29:25 <dcf1> there are some things we can do in the short term to mitigate the length limit for snowflake
16:29:28 <meskio> do we want to push for arti to support PT spec 2.0 (or later)?
16:29:40 <meskio> or starting thinking on integrating some parts of it into our spec?
16:29:43 <dcf1> the largest part of the bridge line is the ice=stun:... list
16:29:52 <arma2> if arti will fold PTs into the arti process, maybe it won't want to use socks at all between tor and the PT
16:30:04 <dcf1> every STUN server has a "stun:" prefix and a port number; we could make those implicit if not specified.
16:30:53 <meskio> arma2: arti is implementing the socks API for now, and it is a useful API that might take some time to deprecate as we can develop PTs without involving arti on them
16:31:13 <arma2> ok
16:31:18 <meskio> I do think having PT libraries inside arti will be really handy, but I see a near future where both cohesist
16:31:40 <arma2> yep
16:31:52 <shelikhoo> but it would be great if this length issue is addressed in someway
16:31:57 <arma2> i agree we should find some better long term answer here. this problem will bite us again.
16:32:01 <shelikhoo> even if the PT is running out of the process
16:32:46 <meskio> yes, AFAIK the source of the problem is that those arguments are passed in the login field of socks5 wich has a size limit
16:33:05 <arma2> isn't the PT 2.0 spec supposed to be more friendly for mobile too?
16:33:09 <meskio> dcf1: your proposal makes sense, maybe we should open a ticket about it and see when we can fix that in parallel to other more long term solutions
16:33:29 <arma2> ...is it time for tor to officially move to the pt 2.0 spec? :)
16:33:32 <meskio> arma2: more friendly to mobile ==> specifies an API to write PT libraries
16:34:03 <meskio> I have to recognize that I haven't read slowly PT 2.0
16:34:11 <meskio> and by the way, there is already a PT 3.0
16:34:11 <cohosh> arma2: snowflake does follow the v2 spec for go libraries
16:34:52 <cohosh> the issue here is not related to the mobile friendliness issue afaik
16:35:18 <arma2> i don't know good answers here; but it is a thing that the anti-censorship team should be good at, so if we aren't, we should get it on the roadmap to somehow make the situation better
16:36:06 <cohosh> tangentially, this is another case in which some kind of tor browser + pt CI would have come in useful
16:36:29 <meskio> yep :)
16:37:20 <shelikhoo> anyway, if we wants to fix this issue now without changing C-Tor or pt spec, we could write the full bridgeline in a file, and pass the path to that file in the socks arg...
16:37:40 <shelikhoo> but this won't be something beautiful...
16:37:48 <meskio> you mean have the bridgeline include the path of this file?
16:37:56 <meskio> include -> pass the file as an argument
16:38:02 <shelikhoo> something like this
16:38:18 <shelikhoo> so we just need to change tor browser and pt
16:38:23 <arma2> don't be shy about fixing c-tor or the pt spec. if we're going to implement a hack just so we never have to touch tor... maybe we're doing it wrong
16:38:24 <shelikhoo> not C-tor or spec
16:38:29 <meskio> I think it will be nice to look into long term solutions, while we do the hacking we need to survive with what we have
16:40:04 <shelikhoo> I think our proposal about PT version is still currently awaiting approval or suggestion to get into C-Tor
16:40:52 <shelikhoo> improving C-Tor and pt spec implemented is ideal, but in reality it might take too long...
16:41:02 <dcf1> there's a very old issue about alternatives to the SOCKS model, where the idea of writing parameters to a file was also proposed
16:41:05 <dcf1> https://gitlab.torproject.org/legacy/trac/-/issues/7153
16:41:23 <dcf1> "Feeding it directly to the pluggable transport instead (as a file in the filesystem, presumably) saves two data-reformatting operations."
16:41:44 <arma2> shelikhoo: ticket # for this proposal? if there are things you need from other teams we should escalate until you get them
16:42:23 <meskio> arma2: https://gitlab.torproject.org/tpo/core/torspec/-/merge_requests/63
16:43:00 <meskio> I will open a ticket about this problem and try to collect solutions from PT v2 and past discussions
16:43:15 <meskio> and see if we can produce a proposal to change our pt-spec to solve this problem
16:43:38 <meskio> dcf1: do you want to create the ticket about the snowflake ice= args?
16:44:10 <dcf1> meskio: ok
16:44:20 <meskio> thanks
16:44:24 <meskio> anything on this?
16:44:58 <shelikhoo> nothing more from me
16:45:02 <meskio> snowflake-02?
16:45:06 <shelikhoo> yes!
16:45:32 <cohosh> :D
16:45:32 <shelikhoo> after we switch on the support for distributed snowflake support on broker
16:45:54 <dcf1> the bridge totally works for me with no trouble. i just had to specify the snowflake-02 fingerprint in the bridge line.
16:45:58 <shelikhoo> and get tor browser to include update version of snowflake (thanks cohosh)
16:46:09 <dcf1> There were a lot of end-to-end changes needed to make that possible.
16:46:29 <shelikhoo> it is now working in production environment with a custom bridge line!
16:46:32 <shelikhoo> cheers!
16:46:46 <meskio> amazing
16:46:47 <dcf1> my question is: do we want to give the snowflake-02 bridge line to testers, or are we waiting for something else to happen?
16:47:02 <shelikhoo> I think it is good for testing
16:47:29 <dcf1> thanks
16:47:34 <meskio> should we already send a mr to tor-browser-build including it? (so it will be distributed by circumvention settings builtin list) or do we want to test more before doing it?
16:47:49 <meskio> AFAIK no one actually consumes the circuvention settings builtin list...
16:48:17 <shelikhoo> it would need to have some UI change to show that secondary snowflake bridge...
16:48:23 <shelikhoo> I think...
16:48:24 <dcf1> let's keep in mind the limitation that snowflake-02 is probably not quite as capable, hardware-wise, as snowflake-01.
16:48:47 <dcf1> I don't think it's a good idea to quickly move to a model where it is handling half of all snowflake traffic
16:49:08 <shelikhoo> so we can give tester the bridge line to use it
16:49:11 <dcf1> I need to check, but I think snowflake-02 only has a 1G uplink, for example (snowflake-01 has a 10G and frequently uses more than 1G)
16:49:26 <shelikhoo> without making it default yet
16:49:38 <dcf1> enabling snowflake-02 in an alpha would be all right, I think
16:49:39 <arma2> dcf1: what is the right place for doing the load balancing? the broker? somewhere else?
16:49:53 <arma2> (load balancing = send the right amount of traffic to each snowflake-0x)
16:50:15 <dcf1> unfortunately the only possible place to do load balancing, technically, is in the client
16:50:18 <shelikhoo> we need to do that by advising client to use the desired bridge line
16:50:35 <meskio> shelikhoo: AFAIK we don't need UI changes, TB will configure both in torr
16:50:36 <dcf1> that's because the client is where the fingerprint is defined, because of the way bridge lines work
16:50:48 <meskio> but that means half clients to each of them, so not much load valancing...
16:51:17 <arma2> hm. should we put a weight on each bridge line to steer clients into using the right loads? that would be static so not best.
16:51:33 <arma2> or ultimately should we just make all of the snowflakes able to handle 1/N of the snowflake load
16:51:42 <dcf1> arma2: there's also no technical method to do that, weight different bridge lines in torrc
16:51:44 <meskio> AFAIK tor doesn't have any mechanism to weight bridges
16:51:51 <arma2> dcf1: correct, we would need to add that
16:52:22 <arma2> (though, hm, we could fake it now by giving three snowflake lines, 2 for the big snowflake and 1 for the little. probably not best but could work. :)
16:52:32 <meskio> we could put twice snowflake-01 and once snowflake-02
16:52:40 <dcf1> yes, that would probably work
16:52:42 <meskio> but this will result in three attempts to connect, isn't it?
16:52:48 <shelikhoo> or we could build this into tor browser?
16:52:58 <meskio> hehe, we are saying the same...
16:52:59 <shelikhoo> that when user choose snowflake
16:53:19 <shelikhoo> it internally choose one of the defined bridge line
16:53:21 <dcf1> correct, it will result in racing multiple rendezvouses, which is not ideal but is what we were planning to go ahead with
16:53:25 <shelikhoo> with weight considered
16:53:42 <arma2> what is our plan for the third snowflake bridge? will we have bridges with widely varying capacity?
16:53:52 <dcf1> never mind, snowflake-02 has a 10G link
16:53:54 <arma2> or is our plan to have every snowflake bridge be quite good
16:54:19 <arma2> i think the right long term plan is that every snowflake bridge should be good enough, and then we just add snowflake bridges as we need to handle more load
16:54:24 <dcf1> maybe it's okay actually. I have to apply the recent performance tuning from snowflake-01 to snowflake-02 as well.
16:54:43 <arma2> then the clients don't have to try to be extra smart. just, "pick a snowflake bridge and use it"
16:55:26 <meskio> sounds good
16:55:29 <arma2> (though, since right now tor clients use two guards, which means they use two bridges, i think if they have two snowflake bridge lines they will use both of them.)
16:56:47 <meskio> ok, so I hear this is not much of a problem (yet) and maybe we can already add the bridgeline into TB alpha
16:57:12 <dcf1> yes, I think there's no problem with putting it in an alpha now
16:57:15 <arma2> yay
16:57:23 <meskio> shelikhoo: will you create a mr in tor-browser-build or ask the applications team to do that?
16:58:24 <shelikhoo> So we wants to include the second bridge line in parallel to the primary one in tor browser?
16:58:35 <shelikhoo> and expect the client to try to connect them all
16:58:40 <shelikhoo> is that correct?
16:58:52 <arma2> i think yes
16:59:01 <cohosh> can we have it be a separate default bridge option?
16:59:13 <cohosh> this might put a lot of strain on the broker and on our proxy pool
16:59:20 <cohosh> if every client is using double resources
16:59:21 <arma2> cohosh: ah ha, so the interface lets you pick "snowflake 1" or "snowflake 2"?
16:59:26 <cohosh> yeah
16:59:39 <meskio> everybody will pick 1...
16:59:54 <dcf1> conceivably you can do something like we used to have with "meek-google", "meek-amazon", which was sort of a hack to work around the bridge configuration not providing a way to set per-bridge parameters
16:59:58 <shelikhoo> My opinion that once user choose snowflake, the tor browser should choose a bridge based on weight for user
17:00:12 <arma2> meskio: technically, tor clients will pick both, they will connect to both, fetch bridge descriptors for both, then build circuits through either. if they have a lot of circuits open, they will likely continue to use both.
17:00:36 <shelikhoo> so there is only one option on ui
17:00:44 <meskio> do I have a dejavu of this being discussed in the past? did we arrive to a proposal of a path to move it forward?
17:00:48 <arma2> so in some sense they will be producing "double" load, but also a given tor byte will go to only one of them.
17:01:15 <dcf1> so shelikhoo is proposing programming some additional intelligence into tor browser itself, and having it rewrite the torrc file dynamically
17:01:16 <shelikhoo> while bridges are automatically load balanced on client side
17:01:39 <cohosh> the tor browser team is pretty overloaded, unless this is an easy fix it's not likely to happen for a few months
17:01:44 <dcf1> some previous discussion: https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/28651#note_2784244
17:01:54 <dcf1> https://gitlab.torproject.org/tpo/core/tor/-/issues/40578
17:02:41 <arma2> i continue to think that we should just have two snowflake bridge lines, and let tor do its thing. clients will connect to both of them, but client load will be spread between them. this is because of tor's current default of using 2 guards aka 2 bridges.
17:02:42 <shelikhoo> dcf1: yes, so when user choose choose snowflake option, it choose one bridge from many bridges
17:03:02 <shelikhoo> but don't rewrite torrc after that
17:03:29 <meskio> arma2: that will double the load on the broker, but I think the broker is not much overloaded right now
17:03:46 <arma2> shelikhoo: tor already shuffles your bridge lines and uses only the top two that it picked. every client will get its own random top two. (in this case there are only two, so it's the trivial case. but that's how we handle using 20 default bridges.)
17:03:48 <cohosh> i'm more worried about unrestrictively NAT'd proxies
17:04:17 <shelikhoo> meskio: and double the load on proxy, while we only have limited amount of proxy
17:04:18 <shelikhoo> yes
17:04:19 <cohosh> and the broker was having load issues
17:04:29 <cohosh> dcf1 recently had to upgrade the machine it was on
17:04:40 <cohosh> so maybe it's good now
17:04:46 <cohosh> but doubling that load is not a trivial thing
17:05:12 <shelikhoo> arma2: yes... so it should be fine once we add more bridges and more proxies
17:05:49 <cohosh> i would two different UI options and let ConnectAssist do something smart there about choosing the bridge
17:06:18 <cohosh> at least for now
17:06:21 <arma2> when we add a third snowflake bridge will we put a third UI option?
17:06:32 <meskio> right now TB doesn't use connect assist bridgelines for snowflake, but I can hack that by telling TB that those bridges are not builtin but 'bridgedb' ones
17:06:58 <cohosh> meskio: ah i see, i had a misunderstanding of what connect assist was doing
17:07:23 <cohosh> i guess it doesn't matter as much what we do for alpha versions of tor browser
17:07:41 <cohosh> we could go with adding both bridge lines and doubling load for alpha users and reconsider when we want to move this to stable
17:07:43 <meskio> TB is ignoring the bridgeline that circuvention settings is giving if it says 'builtin', but is possible to hack it around, I've done it for TM
17:08:14 <arma2> shelikhoo: we do want to fix https://bugs.torproject.org/tpo/core/tor/40578 before we add too many more snowflake lines. but that will only really matter once we get to three or four or five.
17:08:35 <arma2> shelikhoo: that is, the goal would be that no matter how many snowflake bridge lines we have, users will produce load on two of them.
17:08:42 <arma2> (including broker load)
17:09:19 <meskio> arma2: #40578 is closed
17:09:25 <meskio> should we reopen it?
17:09:32 <arma2> well fuck
17:09:34 <meskio> or what do you mean about fixing it?
17:09:34 <arma2> yes we should
17:09:40 <cohosh> arma2: you are a lot more confident on promising network team work than they are: my understanding is they don't want to add new or different features to c-tor
17:09:45 <arma2> people keep closing tickets "because arti"
17:09:47 <cohosh> i think they closed it for this readon
17:10:09 <arma2> yes, i have been assuming i would need to build this feature myself
17:10:29 <arma2> or maybe arti is magically in place already by then. maybe.
17:10:44 <shelikhoo> yes... that's what we wants, but right now for the next week we just give custom bridge line to testers, and add the second bridgeline to Tor browser?
17:11:23 <meskio> we can plan on this for arti, in theory should be working in ~1year, it should be fine to wait that time (I hope)
17:11:40 <meskio> shelikhoo: +1
17:11:47 <cohosh> sounds good to me
17:11:50 <meskio> I think we should close the meeting here, is 15min after the hour
17:11:57 <meskio> but lets keep this problem in mind
17:11:59 <shelikhoo> okay, I will try to open a mr and seek help if necessary
17:12:08 <meskio> great, thanks
17:12:28 <cohosh> this is really awesome work by the way, it was a long time coming and it's great that it's here :D
17:12:35 <meskio> #endmeeting