15:57:43 <shelikhoo> #startmeeting tor anti-censorship meeting
15:57:43 <shelikhoo> here is our meeting pad: https://pad.riseup.net/p/tor-anti-censorship-keep
15:57:43 <shelikhoo> feel free to add what you've been working on and put items on the agenda
15:57:43 <MeetBot> Meeting started Thu Sep 21 15:57:43 2023 UTC.  The chair is shelikhoo. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:57:43 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:57:49 <shelikhoo> Hi~ Hi~
15:58:00 <onyinyang[m]> hello :)
15:58:06 <meskio> hello
15:58:06 <Sombra[m]> hey!!
15:59:58 <cohosh> hi
16:01:11 <shelikhoo> okay, let's start with the first topic:
16:01:12 <shelikhoo> deprecation of CAPTCHA moat
16:01:12 <shelikhoo> https://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/173
16:01:18 <meskio> that topic is mine
16:01:40 <meskio> we are discussing deprecating the captcha moat API in favor of the circumvention settings one
16:01:56 <meskio> CAPTCHAs don't seem to block any censors, as is pretty cheap to solve them
16:02:02 <meskio> but they are hard for some users to solve
16:02:26 <meskio> our mechanism in circumvention settings is not great, but seems to work better
16:02:46 <meskio> the mechanism there basically is distributing different bridges each day so censors need to keep trying to list the bridges
16:03:16 <meskio> anyway, I've being poking app developers that use the captcha API about the deprecation
16:03:49 <meskio> it looks like we might need to keep it running for a bit, and I might make a fake captcha API in rdsys to keep it running after turning down BridgeDB
16:04:06 <meskio> fake captcha I mean handing out a single captcha, the same one all the time
16:04:16 <meskio> and giving out bridges the same way we do in circumvention settings
16:04:18 <ggus> meskio: if settings pool is doing a good work and users are already using this alternative, maybe we could raise the bar on the captcha challenge and see if china censors will still be able to block the bridges? i'm curious how they are solving the moat captchas...
16:04:56 <meskio> ggus: solving captchas is cheap: https://anti-captcha.com/
16:05:10 <meskio> for 1 dolar you get thousands of captchas solved
16:05:16 <meskio> I don't think we can play that game
16:05:35 <Sombra[m]> meskio: also: https://www.deathbycaptcha.com/
16:05:44 <dcf1> we had trouble in the past that the pool of pregenerated captchas was small: https://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/40038
16:07:16 <meskio> yes, we could solve that by dinamically generating the captchas instead of building them offline
16:07:55 <meskio> but I think CATPCHAS are hard for users and easy for censors
16:08:18 <dcf1> no disagreement, I'm in favor of deprecating it too, it you think it's a good idea
16:08:40 <meskio> this plan is part of the work I'm doing to migrate bridgedb distributors into rdsys
16:09:06 <Sombra[m]> meskio: how many bridges we send every day? have we considered creating decoy bridges and make it hard for censors to block (choose how to block) them all?
16:09:35 <shelikhoo> I think censors can test if a bridge works in an automatic way
16:09:42 <meskio> Sombra[m]: the number depends on how many bridges we have
16:09:46 <shelikhoo> while users need to try themselves
16:09:50 <dcf1> ggus: what do you mean "raise the bar"? make the captchas harder to solve?
16:10:11 <ggus> meskio: i remember many years ago users complaining about the captcha being difficult to solve, but that was fixed (~2018)
16:10:14 <ggus> dcf1: yes
16:10:28 <ggus> there are many new fancy captchas around these days
16:10:37 <ggus> it's not about weird words
16:10:50 <cohosh> Sombra[m]: we have some metrics for this: https://metrics.torproject.org/bridgedb-transport.html
16:10:57 <dcf1> "Make CAPTCHAs easier to solve." https://github.com/NullHypothesis/gimp-captcha/commit/27623507038c7cb5ae052a38463f1bafffb4ec6f
16:11:04 <cohosh> https://metrics.torproject.org/bridgedb-distributor.html
16:11:12 <ggus> if there's a possibility to explore other captchas solutions, that would be interesting
16:12:07 <meskio> ggus: I haven't look much around, I think the libraries we could use to generate captchas ourselves are not way better than what we do right now
16:12:25 <meskio> and I'm not sure our users will be happy if we use reCAPTCHA
16:12:32 <shelikhoo> the fancy one are usually drawn with js
16:12:42 <shelikhoo> including the interactive ones
16:12:54 <meskio> we could do some experiments with captchas in the HTTPS distributor
16:13:01 <dcf1> I see ggus's point, if we are going to turn off this service anyway, there is an opportunity to do an experiment and perhaps gain some knowledge before we do.
16:13:47 <ggus> and users won't be affected because we can move them to the 'settings' pool via the api
16:14:13 <dcf1> of course, need to weigh the expected benefit of an experiment against the time/attention required to run it
16:14:51 <cohosh> sort of relevant: https://lists.torproject.org/pipermail/tor-dev/2021-July/014602.html
16:15:28 <cohosh> there were some replies on proposed experiments there
16:17:02 <meskio> doing changes in TB will require some work, but I could do some experiments from our side in the API, anyway it will take TB and others some months to face them out
16:17:43 <shelikhoo> it should not be so hard to run the experiment on https distributor
16:18:05 <shelikhoo> unless it requires modify bridgedb..
16:18:07 <meskio> yes, the https distributor might be a good target for it, as we have more control on it
16:18:26 <meskio> I will redo the https distributor as part of rdsys sometime next quarter
16:18:47 <ggus> do we have stats for 'settings'? https://metrics.torproject.org/bridgedb-distributor.html
16:19:08 <ggus> or 'moat' means 'moat and settings'?
16:19:08 <meskio> no, this is one of the things I want to solve also with that work on rdsys
16:19:24 <meskio> those stats are comming from bridgedb, we are not producing equivalents from rdsys yet
16:19:28 <meskio> but my plan is to add those
16:20:07 <meskio> I will create an issue to explore CAPTCHA experiments and try to come out with proposals for it
16:20:24 <meskio> let's see how many cycles I actually have to play with that...
16:20:36 <shelikhoo> yes, I think we have kind of reached an agreement on how to proceed. namely we remove inhouse image CAPTCHA, and then add a 3 party one to https as an experiment?
16:21:10 <shelikhoo> anything we wish to discuss on this topic?
16:21:21 <meskio> mmm, I'm not sure we want to do 3 party, as our users might not like that
16:21:28 <meskio> but I will keep that as an option
16:21:32 <meskio> that is all for me
16:21:38 <shelikhoo> yes... let's see...
16:22:10 <shelikhoo> I think most in house one will either be too difficult to implement or don't work
16:22:21 <shelikhoo> but there is only one way to find out
16:22:22 <shelikhoo> okay the next topic is:
16:22:35 <shelikhoo> snowflake domain front resolving to a mix of fastly and cloudflare since 2023-09-20
16:22:35 <shelikhoo> https://lists.torproject.org/pipermail/anti-censorship-team/2023-September/000314.html
16:22:35 <shelikhoo> https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42120
16:23:05 <shelikhoo> we have already briefly discussed this in the irc channel
16:23:07 <dcf1> the snowflake broker domain front started to resolve to a mix of fastly and cloudflare IP addresses
16:23:15 <dcf1> we don't know if there is some pattern, if it's regional, etc.
16:23:29 <dcf1> if a user gets a cloudflare IP address, snowflake won't work for them
16:24:08 <shelikhoo> we can either change the domain name
16:24:13 <shelikhoo> or pin the ip address
16:24:16 <dcf1> probably the most straightforward way to mitigate the issue is to choose a different front domain, one that still always resolves to fastly
16:24:52 <dcf1> right, or as shelikhoo says, find some way to internally pre-resolve the name (but that creates inconsistencies between the DNS layer and the TLS layer)
16:25:37 <dcf1> but snowflake is currently broken for an unknown fraction of users, ideally it would be good to find a solution quickly (but not so hasty that we make a mistake)
16:25:40 <meskio> piking a new domain front will help for circumvention settings users, as we can do the change there without a need of a TB/orbot/... release
16:26:06 <meskio> but maybe the pinning is good in the long term...
16:26:56 <dcf1> actually, of the two options, locally pinning an IP address seems more fragile
16:27:31 <shelikhoo> or we can have a resolve in stead of
16:27:32 <dcf1> the problem with changing the domain front is there are not a ton of good options
16:27:59 <shelikhoo> so the domain name will be resolved to an ip from an alternative domain name
16:28:10 <shelikhoo> but the request's SNI will remain unchanged
16:28:12 <dcf1> https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40068#note_2767823
16:29:01 <dcf1> shelikhoo: yes, that will work, but we must keep in mind it does create a detectable inconsistency: the censor sees a TLS connection without an associated DNS query immediately before
16:29:11 <cohosh> yeah the trick with chosing the domain is we don't have a good idea of which are already targeted for censorship
16:29:28 <shelikhoo> dcf1: yes... that's true
16:29:29 <cohosh> only two other options have OONI measurements, and one of those is blocked in Iran
16:29:54 <shelikhoo> there are a few possible domain names I previously collected
16:29:57 <cohosh> i should say two of the options that were listed by shelikhoo earlier, i haven't tried to update that list
16:30:14 <shelikhoo> we can have a pool of domain names/end points
16:30:19 <shelikhoo> and have a fallback system
16:30:39 <shelikhoo> it will have the issue of creating a lot of suspicious traffic
16:31:00 <shelikhoo> but would work, if the user's physical safety is not at risk\
16:31:17 <dcf1> there are two things to consider: one is what to do in the long term, which might involve more flexible SNI selection, as well as new rendezvous methods not based on TLS. the other is what we should do in the next 7 days to fix what is broken now.
16:32:10 <cohosh> are there any drawbacks to going with foursquare.com for now?
16:32:37 <cohosh> https://explorer.ooni.org/chart/mat?since=2023-08-22&until=2023-09-22&time_grain=day&axis_x=measurement_start_day&axis_y=probe_cc&test_name=web_connectivity&domain=foursquare.com
16:32:41 <dcf1> I kind of feel like that's the best option. Maybe we can get a little bit of user testing first?
16:33:09 <dcf1> One thing we should do is make a forum post, at least, so users can know what's going on and they are not the only ones affected.
16:33:41 <dcf1> That was my fault. I should have thought of that yesterday. I should not have let nina13[m] and the community team start to get failure reports without a heads-up.
16:34:29 <meskio> the forum post is a good idea
16:34:40 <cohosh> okay it sounds like we have until next week for the tor browser release, we can let them know what's up and our intended fix so they can get it tested with users and then make the MR early next week?
16:34:52 <dcf1> nina13[m], ggus: would you be able to walk a user through changing `front=cdn.sstatic.net` to `front=foursquare.com`, if you get in contact with a tech savvy user?
16:35:05 <dcf1> I'll make a forum post
16:35:11 <meskio> should we change the domain in circumvention settings to foursquare.com?
16:35:21 <dcf1> The other thing is that this affects guardian project and orbot as well
16:35:29 <cohosh> meskio: yeah i think that's a good idea
16:35:46 <meskio> I think orbot is already using circumvention settings, but not sure on which cases they do
16:36:45 <nina13[m]> dcf1: It is easier just to give a full bridge line to test - but yes, sure
16:37:21 <dcf1> meskio: hmm, if orbot is using circumvention settings, I wonder why there are still relatively so few users of the snowflake-02 bridge? does circumvention settings only have a snowflake-01 bridge line?
16:37:23 <nina13[m]> championquizzer: ^^
16:38:09 <meskio> dcf1: I don't know, there is a button now in rdsys to autodiscover your configuration that I think uses that API
16:38:19 <shelikhoo> dcf1: another possibility is that they are only using circumvention settings to get bridges, but when user select snowflake, it will always use bridge 01
16:38:20 <meskio> but I guess the normal snowflake button uses the hardcoded bridge they have
16:38:30 <meskio> and I might be using a new unreleased orbot...
16:38:32 <shelikhoo> so we need to ask them how the setting is used
16:39:05 <meskio> also TB does ignore circumvention settings for builtin bridges, orbot might do the same
16:39:16 <dcf1> Orbot 17 has both bridges, but it's not released yet, except in beta, afaik. I walways thought that was the cause of the low use of snowflake-02, that we were just waiting for Orbot to make a full release of v17. But maybe it is more complicated.
16:39:47 <meskio> mmm, I have here orbot 17, so I guess I'm using the beta...
16:39:50 <meskio> is in fdroid
16:40:39 <dcf1> Play Store (https://play.google.com/store/apps/details?id=org.torproject.android) still says "Updated on Nov 1, 2022"
16:40:43 <meskio> ok, so we are not going to fix existing orbot users with circumvention settings for now, wich means most Iran
16:40:52 <dcf1> "Version: 16.6.3-RC-1-tor.0.4.7.10"
16:41:24 <meskio> anyway I think we should do the change both in TB repo and settings API
16:41:59 <meskio> give the bridgeline to community to hand it around and post it in the forum
16:42:32 <dcf1> okay, let me make a forum post (sometime later today, after a few hours) and I'll write an email to guardian project
16:42:42 <cohosh> thanks dcf1
16:42:45 <meskio> thanks
16:42:52 <meskio> I'll do the changes in settings API
16:42:57 <shelikhoo> okay, anything more on this topic?
16:43:11 <shelikhoo> the next topic is:
16:43:12 <meskio> and I can do the TB change also if needed
16:43:21 <meskio> EOF
16:43:22 <shelikhoo> Remove Go 1.20 support? kcp library seems to crash compiler:
16:43:22 <shelikhoo> https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/jobs/370910
16:43:22 <shelikhoo> # github.com/xtaci/kcp-go/v5
16:43:22 <shelikhoo> /go/pkg/mod/github.com/xtaci/kcp-go/v5@v5.6.3/timedsched.go:19:6: internal compiler error: panic: interface conversion: interface is nil, not ir.Node
16:43:22 <shelikhoo> Please file a bug report including a short program that triggers the error.
16:43:23 <shelikhoo> https://go.dev/issue/new
16:43:25 <shelikhoo> note: module requires Go 1.21
16:43:28 <shelikhoo> https://github.com/xtaci/kcp-go/blob/v5.6.3/timedsched.go#L19
16:43:30 <shelikhoo> Go compiler issue: https://github.com/golang/go/issues/61074
16:43:32 <shelikhoo> Has to do with new compiler builtin "clear" in go1.21 https://tip.golang.org/doc/go1.21#language
16:43:34 <shelikhoo> ...which kcp-go started using 10 days ago https://github.com/xtaci/kcp-go/commit/691fc2febef3dd927dca7815b207fb4dad19b58f
16:43:39 <shelikhoo> thanks for comment from dcf1
16:43:52 <shelikhoo> (if I guessed correctly)
16:44:00 <dcf1> yes
16:44:11 <shelikhoo> this is one of the issue with tracking the most recent version of dependency
16:44:38 <shelikhoo> it seems only go 1.21 will work with most recent version of kcp library
16:44:39 * dcf1 is bummed because programming in Go used to be so much fun
16:44:46 <dcf1> now it sucks like every other language
16:44:58 <shelikhoo> should we go head and remove support for 1.20
16:45:38 <dcf1> it sounds like those are the only two options, hold back the compiler version or hold back the kcp-go version
16:45:40 <shelikhoo> (shell look at go-quic's intentional break of go tool chain versions )
16:46:03 <dcf1> altenratively, patch out the use of the new "clear" builtin in kcp-go, it's only in the fec.go part, which we do not use
16:47:23 <shelikhoo> I personally don't think there is that much value in keep supporting 1.20
16:47:45 <shelikhoo> since if someone can use 1.20, they probably can just use 1.21
16:47:54 <dcf1> https://tip.golang.org/doc/devel/release go1.20 released 2023-02-01
16:48:34 <shelikhoo> but I don't have 'firm' opinions
16:48:58 <dcf1> there's probably no good long-term option than to keep in the upgrade treadmill
16:49:25 <dcf1> unless removing go1.20 causes a lot of problems, I don't see a better option
16:49:38 <meskio> once we stoped supporting old enough version of go so debian stable and others will not work I don't see a mayor problem with moving to 1.21
16:49:41 <dcf1> we already require a version that's well past what's in debian stable, right?
16:49:47 <shelikhoo> yes...
16:50:50 <shelikhoo> okay I think it is decided that we are going to remove support for 1.20
16:50:50 <dcf1> (Go developers were jealous of Rust basically requiring all users to constantly be on the nightly release.)
16:51:00 <shelikhoo> unless someone object now
16:51:20 <shelikhoo> oh, yes, rust stable get rust....
16:52:17 <shelikhoo> okay, anything more to discuss on this topic?
16:52:35 <shelikhoo> we have an interesting link this week:
16:52:41 <shelikhoo> https://www.bamsoftware.com/papers/fep-flaws/#sec:obfs4
16:52:41 <shelikhoo> writeup of Elligator distinguishers that have affected obfs4proxy
16:52:49 <shelikhoo> anything more to discuss in this meeting?
16:53:11 <dcf1> This is my writeup of crypto bugs in fully encrypted (look-like-nothing) circumvention protocols.
16:53:29 <dcf1> It's the first proper writeup of the obfs4proxy distinguishers we dealt with in the past couple of years.
16:54:13 <meskio> nice, thank you for documenting them
16:54:14 <dcf1> nothing more from me
16:54:26 <cohosh> yeah, nice work
16:54:45 <shelikhoo> okay I will call it a meeting, thanks for your writing dcf1!
16:54:46 <shelikhoo> #endmeeting