16:00:45 #startmeeting tor anti-censorship meeting 16:00:45 Meeting started Thu Dec 9 16:00:45 2021 UTC. The chair is cohosh. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:45 Useful Commands: #action #agreed #help #info #idea #link #topic. 16:00:57 welcome! 16:01:14 here is our meeting pad: https://pad.riseup.net/p/tor-anti-censorship-keep 16:01:27 feel free to add items to the agend and update us on what you've been working on 16:01:34 * cohosh does this 16:01:36 hello 16:03:07 hi~ 16:03:41 meskio, I'm curious about your impression of BridgeDB scraping (e.g. https://gitlab.torproject.org/tpo/community/support/-/issues/40050#note_2765440 "meskio and i picked out a moat bridge yesterday ... and it works") 16:04:05 doesn't have to be now, I just think it would be interesting to hear from someone who has worked closely with BridgeDB. 16:04:30 bridgedb is a black box to me in some sense 16:04:40 but yes, I'm not sure how they do the scrapping 16:04:55 I'll expect them to do it ince in a while, so newer bridges takes some time to be blocked 16:04:59 dcf1: there's some backlog in #tor-dev about this 16:05:06 For the sake of the record, I will post some links related to censorship in Russia for the first discussion item 16:05:32 https://gitlab.torproject.org/tpo/community/support/-/issues/40050#note_2764320 <- about the general phenomenon, testing results, link to NTC where there is testing by Russian residents 16:05:55 dcf1: when i looked into scraping for the belarus enumeration attempts, i found it hard to find any info from the metrics: https://gitlab.torproject.org/tpo/anti-censorship/censorship-analysis/-/issues/40002#note_2728643 16:06:03 https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40014 <- changes in the DTLS fingerprint of snowflake-client, as that would found to be a distinguisher currently being used 16:07:10 I want to take a moment to say 16:07:34 how proud I am to work with a team like this one during moments like this 16:08:02 It was a sudden change, to be sure, but it's not a crisis, everyone is doing their part with a cool head 16:08:27 we have sufficient backup plans and mitigations readied, we were not caught sleeping 16:09:04 the strength of the team made what might have been a crisis into a manageable set of circumstances 16:09:27 good work, all 16:09:57 thanks for the words, it is intense, but is nice to be involved in such a good team :) 16:10:24 The situation in Russia is i nteresting because it's a new and changing situation, let's keep in view that even more severe censorship is already business as usual in other places 16:10:38 I am really glad to work in this team~ 16:10:50 ❤ 16:11:09 <3 16:11:22 yeah this week was a lot of really amazing work 16:12:39 should we proceed by taking stock of what we have? 16:12:46 meskio: maybe you want to talk about the telegram bot? 16:13:00 sure 16:13:36 seeing that moat is blocked in russia, and other distribution mechanisms are hard to reach, ggus thought that using telegram to distribute bridges might be a good idea 16:13:47 telegram has a lot of users in russia (and other places) 16:13:50 https://gitlab.torproject.org/meskio/telegram-bridge-bot 16:13:59 I hack together a simple bot -^ 16:14:14 https://forum.torproject.net/t/tor-blocked-in-russia-how-to-circumvent-censorship/982 16:14:14 and is reachable over @GetBridgesBot 16:14:23 you get bridges by sending the command /bridges to it 16:14:34 it looks like is pretty much in use 16:14:45 * meskio goes to look to the usage picture 16:15:13 https://gitlab.torproject.org/tpo/anti-censorship/team/uploads/8db01ff2c4523388b2632f5673b5034e/telegram_bot.png 16:15:24 ^- this is one day old, few hours after launching it 16:15:30 one thing that makes me nervous about telegram is that it's famous for having no barrier to making accounts. so i assume our attacker has many many accounts already. i wonder if there's a way to learn how old the account is, or really anything to distinguish them 16:16:12 The Telegram bot is posted at https://github.com/net4people/bbs/issues/97#issuecomment-988401477, but not at NTC yet, I'll make a note to make a post there 16:16:51 arma2: yes, I expect telegram bridges to be blocked fast, we'll see 16:17:19 I don't think we can know how old an account is, let me investigate 16:18:12 (i got that trick from the salmon people, who said they wanted to demand a facebook account created before 2018 or the like) 16:18:56 (but i fear it is just a great sentence to write in a paper, and not actually practical to do) 16:19:33 I could check how many profile photos they have, old users usually have changed their profile photo at least once (I havent myself, so...) 16:19:44 I don't see any dates in user info 16:20:21 Or we can just make dynamic address bridge happen, so blocking a bridge won't matter so long as it takes more times than changing bridge's address 16:20:27 download profile photo, check last-modified idk 16:20:28 huh. there is definitely an arms race to play there, then. ("must have n profile photos", "must have a profile photo we haven't seen before", ...) not sure we want to play it though. 16:21:48 not sure if I can see the date of modification of the profile photo 16:22:06 been really impressive in the last week or so to see how you all work together on this big thing - also together with comms/community team <3 you all rock! 16:22:44 I see the bot as an acute workaround for an unexpected failure in another system, namely BridgeDB's anti-enumeration defenses. Maybe telegram will also turn out to be viable long-term, if we can figure out how to apply anti-enumeration there. 16:23:27 There are some Telegram bots that taking care of users that are not so old, like kicking them out of the channel. Not sure if its useful in this case though but there may bot that check user's account creation. 16:23:33 the idea of serving different bridges each 24h might help here, as censors will need to keep requesting bridges 16:23:56 meskio: can you see when the accounts were created? 16:24:08 anadahz: interesting, I don't see that info in the bot API, can you point me to any bot that does that? 16:24:33 cohosh: no, this is what I get: https://core.telegram.org/bots/api#user 16:24:40 https://www.reddit.com/r/TelegramBots/comments/aw9e0m/bot_that_gives_approximate_creation_dates_for/ 16:24:59 anadahz: thanks, I'll look into it 16:25:16 what else is there to say about the current situation? cohosh, DTLS fingerprinting? 16:26:17 yeah, thanks for your help with that dcf1 16:26:33 we have some fingerprinting defences in snowflake now 16:26:45 it's tricky because the fixes are in a library dependency and not snowflake proper 16:27:08 so you have to build snowflake outside of the module system with the patched library 16:27:15 (direct link to Telegram bot: https://t.me/creationdatebot) 16:27:32 the easiest way to do it is through tor browser builds: https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40014#note_2765254 16:27:33 I wonder how long will it takes before some other difference in implementation will be used to block snowflake again 16:27:58 shelikhoo: good point, we've opened some issues in pion/dtls for other differences identified in that ticket 16:28:05 shelikhoo: yes, it's an interesting question 16:28:14 https://github.com/pion/dtls/issues/408 16:28:20 https://github.com/pion/dtls/issues/398 16:28:32 these are two other fingerprinting differences ^ 16:28:46 i suppose what we'd like it something like uTLS but for DTLS 16:29:18 i also doubt pion/dtls is willing to implement DTLSv1.0 16:29:46 valdikss have pointed out that the self-signed certificate generated is different from browser too 16:29:47 personally I don't think DTLS 1.0 matters, browsers will get to 1.2 eventually 16:30:00 we were only really looking at one browser fingerprint, maybe they are already there, I don't know 16:30:10 in terms of deployment, you can get working compiled biaries of snowflake here: https://people.torproject.org/~cohosh/ru_snowflake_fix/ 16:30:30 instructions here: https://ntc.party/t/ooni-reports-of-tor-blocking-in-certain-isps-since-2021-12-01/1477/55 16:30:47 and we have the fix merged into tor-browser-build so should be slotted for the next release: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/merge_requests/375 16:31:11 for those wanting to dig deep into snowflake distinguishers, also be sure you know about prateek's paper there: https://arxiv.org/abs/2008.03254 16:31:38 oh wow thanks for that writeup dcf!! 16:32:14 cohosh: there is a trick for replacing one go module with another, or with a local directory: https://go.dev/doc/modules/gomod-ref#replace 16:32:24 it would be interesting, in a "retrospective" kind of way, to see if prateek predicted the ones they picked or no 16:32:38 arma2: no, not in this case 16:32:39 dcf1: oh! should we merge that into snowflake? 16:32:51 i see in his recommendations, "Do not offer ‘supported_groups’ as an extension in theserver hello" 16:33:11 arma2: oh, sorry, you're right, I was thinking of something else 16:33:30 ok. and he's got two others besides that one. in case we want to think ahead to next week :) 16:33:46 (wonder if our censors read this paper too) 16:33:49 To make snowflake work for peoples influenced by this DTLS block, we might need to encourage standalone proxy operators to update software version 16:34:12 yes good point shelikhoo 16:34:22 we can use this module replacement trick and update the docker container 16:34:31 this makes that process easier than i thought 16:34:43 with respect arma2, these and other features, and a link to Prateek's paper, are currently present in our ongoing issue of Snowflake fingerprinting 16:35:33 yes, on the point of standalone proxies, we need to encourage people to upgrade, or if we need to, we can potentially exclude proxies that have not upgraded, at the broker 16:35:59 yep. i'm not saying they're surprises to us. but i am saying that what censors do in practice is often not what our roadmap predicts. 16:36:21 or make sure updated client only match with updated standalone proxy 16:36:35 (but that will be a little complex) 16:37:10 the less complexity we add in the broker matching, probably the better 16:37:25 it is nice that if the client fails to connect it will keep trying 16:37:51 shelikhoo: right, cohosh and i discussed that last night, and the direction we were heading is: try to get headless snowflakes to upgrade, and eventually stop handling the old ones, and then the broker matching algorithm can stay simple 16:38:46 Yes, so we wants to send proxy version in the broker request 16:38:58 yep. and apparently we already do. 16:39:03 Yes 16:39:15 we used this to exclude old proxies before 16:40:11 (i also started out with the idea "if the client is in russia and the snowflake is running this version then" and she talked me out of it) 16:40:40 the other state of things is that meek-azure is blocked in some places, by blocking of the front domain 16:41:05 we don't have a workaround for that, but it seems that the meek-azure block has effect in fewer networks than the larger Tor relay+bridge block 16:41:10 should we try to change meek-azure's front domain? or leave it in place and leave russia suffering for having blocked it? 16:41:30 so it's possible that some people for whom Tor is blocked as still able to circumvent using meek in the normal way 16:41:53 is it possible to use multiple front domains? 16:42:11 we can add more fronted domain, so keep tor inaccessible more and more normal website will be impacted 16:42:29 i like it 16:42:58 also the option to use a custom fronted domain? 16:43:25 you can already do that by altering the Bridge line 16:44:08 cohosh: the bridge line for meek? 16:44:37 Did anyone tested this during the blocking in Russia? 16:44:37 ah ha yes there it is in about:config 16:44:52 Bridge meek 0.0.2.0:3 url=https://meek.azureedge.net/ front=ajax.aspnetcdn.com 16:45:17 It's also the case that Tor relays+bridges are blocked on fewer networks than *.torproject.org is blocked (which appears to be much more total, using the usual "decentralized control" ISP-specific blocking) 16:45:33 https://explorer.ooni.org/search?since=2021-11-09&until=2021-12-10&failure=false&probe_cc=RU&test_name=web_connectivity&domain=www.torproject.org 16:45:56 anadahz: if you give me an alternate domain front that azure will handle, i'll test now 16:46:07 hello o/ 16:46:15 Hi~ 16:47:00 arma2: I don't have one, but I can try to setup one 16:48:05 maybe we could produce a list of fronting domain and put this in the guide 16:48:19 I am not sure, but I wonder if the Tor + meek blocking is tied to the TSPU (centralized GFW-like filtering), rather than being left to ISPs. That could explain why it has effect in fewer places. 16:49:06 https://gitlab.torproject.org/legacy/trac/-/wikis/doc/meek#microsoft-azure 16:49:10 ^ lists of domains there 16:50:16 Try *.cdn.office.net 16:50:31 https://theobsidiantower.com/assets/known-good.txt (list from 2017) 16:51:01 ok. am slowly trying to get the original meek to test. i'll have results post-meeting. :) 16:51:36 i have another category of fix we should consider: rotating fresh things (bridges, fallbackdirs, etc) into each new tor browser release 16:51:47 11.0.2 has a new obfs4 bridge, and it works currently behind the censorship 16:52:10 if we shipped a fallbackdir list with some fresh relays in it, vanilla tor might even work. apparently the relay block list hasn't been updated in many days. 16:52:33 it's not a good strategy against a later phase of the arms race, but, maybe we're not in that later phase yet. 16:52:36 wooo! 16:52:46 :D 16:54:46 anything else we should do/try in the next few days? 16:55:13 i did realize when testing the DPI fixes that the TB version of snowflake doesn't have the latest cleint features 16:55:18 including AMP cache 16:55:34 a version bump just got merged 16:55:37 yes, we can talk about that another time if necessary 16:55:58 https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/merge_requests/376 16:56:05 i think another trick we should consider in the short term is: stop giving out old moat bridges via moat. they don't work for people in russia, but the new ones do, and if we give out new ones, moat users will be able to use them. 16:56:18 it turns out that it's not relevant for the current situation, but if it had been, it would have been nice to allow people to test it by changing a configuration file, rather than downloading a new binary 16:56:44 yeah fair, i think this is slated for alpha anyway 16:56:44 we were talking about this idea in #tor-dev earlier. it could be as simple as: "if moat request is coming from ru, return this string" and then go change the string once a day, like we change the telegram answer. 16:56:52 cohosh: oh, ok :) all done then 16:57:31 arma2: I'm affraid of adding a lot of things that depend on us doing things manually 16:57:42 arma2: i really dislike these spaghetti hacks, i think we have an existing feature one sec 16:57:53 i am cool with doing it in a smarter way too :) 16:58:12 but the goal would be: users in ru get a fresh moat bridge not an old one 16:58:19 # Filename that contains blocked bridges list. Comment out to disable. 16:58:19 #COUNTRY_BLOCK_FILE = "blocked-bridges" 16:58:38 ok. so we could make a big list of all the moat bridges older than december n, and put them in the list 16:58:55 i think that'll work, not sure if it's used in production 16:59:02 but this would be a good opportunity to try it out 16:59:10 yep 16:59:28 is not used in production, I mean that config flag is not in polyanthum's config 16:59:38 I hope this feature works 16:59:44 :) 16:59:47 anyway, the general theme of the idea is: since they're not refreshing their blocks, let's take advantage of that fact for now, in all ways. 17:00:34 and the reason for the theme is: many hundreds of thousands of people in russia are judging tor this week, and whether they perceive that we're taking action will stick with them 17:01:23 I'm not sure I'll be able to do it today, but if no one does it before I could try to produce that blocking list tomorrow 17:02:20 meskio: thanks! we can create an issue to track progress and then people can pitch in if they have time 17:03:26 sure 17:03:38 I can create the issue 17:04:00 ok we're at the hour mark, anything else we should discuss for today? 17:04:16 I'm good 17:05:01 i guess also shoutout to shelikhoo for their work on helping verify and test things in russia 17:05:16 Thanks~ No problem~ 17:05:30 shelikhoo: it sounds like you are continuing this work, thank you for doing that! 17:05:34 https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/47 17:06:33 Yes.... I think I can keep tracking this task. 17:06:36 btw, since the blog post about russia, we got new +200 bridges https://metrics.torproject.org/rs.html#search/type:bridge%20running:true%20first_seen_days:0-2%20 17:06:52 in 2 days?! 17:07:14 yes 17:07:28 \o/ 17:07:33 dcf1: can you give me a concrete azure front? i am getting confused by the "*." notation 17:07:38 community power! 17:07:51 ok, I guess in the 2 days preceding the post there were 74, still it's more than I would have thought 17:07:54 https://metrics.torproject.org/rs.html#search/type:bridge%20running:true%20first_seen_days:2-4 17:08:26 nice 17:08:42 dcf1: that's right! :) 17:08:52 maybe try outlook-1.cdn.office.net ? 17:09:24 arma2: try binaries.templates.cdn.office.net too 17:09:44 ok. trying to get them working from nyc first before trying in ru :) 17:11:05 alright, i think we can end the meeting here? 17:11:07 (got it working in nyc with the original ajax.aspnetcdn.com front, but not with either of the next two suggestions yet) 17:11:17 and continue discussion in #tor-dev 17:11:38 * cohosh is again super impressed with all this good work <3 17:12:09 ajax.aspnetcdn.com is on cs22.wpc.v0cdn.net. 17:12:22 these two is on a1847.dscg2.akamai.net. 17:12:32 yeah, akamai won't domain front to azure 17:12:48 we need to run some full scale probe to refresh that fronting list 17:12:50 ok that makes sense for why those both fail 17:12:57 amazing work ac team!!! 17:13:12 ok i'm gonna end the meeting now xD 17:13:15 #endmeeting