17:07:11 #startmeeting Tor VPN s101 Low Level meeting discussion 17:07:11 Meeting started Wed Mar 16 17:07:11 2022 UTC. The chair is ahf. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:07:11 Useful Commands: #action #agreed #help #info #idea #link #topic. 17:07:22 okay, we just did a small status over on BBB first 17:07:38 but since david and i have been in tor.git C+release land and grant land, not much have happened here 17:07:53 cyberta: you wanted to talk next steps? 17:08:17 maybe update from my side (I reported already yesterday, but I dunno if you have seen it): 17:08:37 it was the inverted log list? 17:08:41 I was working on the package to UID mapping this week 17:08:48 neat 17:08:53 no the inverted list was just a side thing 17:09:08 i see 17:09:20 (i'm back, catching up with irc here) 17:09:31 we can quite reliable map packages to UIDs I have a somewhat working prototype 17:09:38 I'll push it these days 17:09:41 is the package to UID mapping possible? just so that we can query for a UID + some state (like a bool or a tribool) 17:09:48 oh, very nice 17:10:13 does it also include the usual "system packages" that i sometimes see in some views on the android platform? 17:10:20 some of the code needs to be rewritten in rust 17:10:30 is it in C? 17:10:41 no it's completely in java :) 17:10:45 ah 17:10:54 it is fine to have it in java entirely, then we can just call it from rust still 17:11:05 we can also see systempackages, however we cannot distinguish between them 17:11:07 as long as it only uses primitime input and output types i think 17:11:10 they have all the same UID 17:11:14 ok, they have the same UID 17:11:17 that is fine i think 17:11:18 which is 0 17:11:33 okay, that is very interesting 17:11:45 i guess they could be group'ed by their UID then, so uid == 0 is platform services 17:11:46 so the idea would be to read the packages in rust from the file descriptor 17:11:48 and everything else is apps? 17:12:11 from the file descriptor? how do you get these names now? 17:12:12 re: and everything else is apps? -> almost :9 17:12:36 okay, so there is some platform components that execute as something different than uid = 0 ? 17:12:39 ok, I'll try to focus on explaining that now 17:12:42 sounds good 17:13:12 some Android file manager apps in past used to be able to list everything that had Android package ID. But I am unaware of any pubic APIs as such. 17:13:26 so, we need to read the packages from the file descriptor, and get the source and destination IP as well as the source and destination ports of each package 17:13:50 the package parsing would be done in rust 17:13:55 wait, are we talking 'packets' now? 17:14:05 sorry packets, yes 17:14:19 ahhh, i got confused here. i thought you meant to read app names out from a file descriptor and got confused 17:14:30 yes, you're right, we get the int fd out of the platofrm and read and parse its content in rust 17:14:34 sure, I ment IP packets, sorry 17:14:40 no worries, i make that mistake very often too 17:15:42 from the Android world we can read /proc/net/tcp, /tcp6, /udp, /udp6, /icmp etc 17:16:02 right 17:16:24 we will parse the files for source and destination ips and ports and get the line with the UID 17:16:33 in the best case ;) 17:16:40 hm 17:16:42 that is the solution for older Androids 17:16:45 that is what we want to do on linux too 17:16:46 yeah 17:16:51 i was about to say there is a more modern api for this 17:17:05 https://developer.android.com/reference/android/net/ConnectivityManager#getConnectionOwnerUid(int,%20java.net.InetSocketAddress,%20java.net.InetSocketAddress) 17:17:06 this one 17:17:15 for the newer Android version beginning from Android Q there's a dedicated API for it 17:17:22 exactly 17:17:31 that works nicely, too 17:17:36 ok, this is very useful because it means we can build something for *desktop* linux and you have just confirmed it works on android too 17:17:40 that is very perfect 17:17:51 the parsing of /proc/net/* is super inefficient, but we can probably live with it 17:18:09 yip, until we find a better solution 17:18:22 we only need to do it when we see the first conntracked udp packet OR the tcp syn-flag-only packet 17:18:55 why? 17:19:02 so it is not very frequently we need to parse it, and we can potentially batch it up to be the entire set of packets we read out of the VPN file descriptor rather than doing it per packet 17:19:13 why? because we only need to populate the structure once the connection is made 17:19:24 unless we think a lot of apps are moving their fd's between each other - i cannot imagine that 17:19:41 chrome does it, but only internally between its subprocesses 17:19:56 so in that case it doesn't matter too much 17:20:01 yep 17:20:07 the efficiency is not a conern i'd say 17:20:27 and we know in the packet whether it's udp or tcp and whether it's v4 or v6, so we only have to parse one file 17:20:59 as I said indirectly before: the /net/proc/* solution doesn't work in all cases, it might be a bug in my implementation though 17:21:13 exactly 17:21:23 okay, so you do see some connections where you cannot find the matching data in the files there? 17:21:46 sometimes I don't get the matching UID 17:22:03 do you get a wrong UID or do you just get nothing? 17:22:15 null 17:22:23 ok 17:22:40 I need to spend some time to figure out why 17:22:52 it would be a bit interesting to figure out if there is a pattern here. i could imagine this code being race condition prone, but it would be interesting if there is a whole class of applications that don't show up there 17:22:57 for example some system services or something 17:23:40 right, needs further fiddeling 17:24:06 I don't expect races, but who knows :) 17:24:48 ya 17:24:52 ok, this is super interesting 17:25:12 parsing the /proc files will be pretty easy to do in rust. we found a crate that already supports all these files 17:25:12 the JNI needs a method to call getUID(sourceAddress, destinationAddress) 17:25:16 so parsing it is basically a one liner 17:25:20 yep 17:25:40 i have actually not found out how we handle the rust-calling-to-java path yet, the other way is easy 17:26:02 for platform services, can apps make network connections by calling into platform services, and thus have some of their activity get UID 0? 17:26:29 no, I don't think so 17:26:31 mikeperry: in theory i think they can because on unix an application can transfer ownership of a file descriptor, but i would have a hard time imagine it happening across the UID barrier 17:26:56 chrome does it, but that will be in its own UID since all the processes will belong to the same UID 17:27:40 it would violate the sandboxing model I think, if an app can hide itself behind UID 0 17:27:45 are the proprietary Google Play APIs that some apps used considered Platform Services? or do they still get the app UID? 17:28:07 but we should double check that 17:28:44 hey all. just checking why you are using the PackageManager getInstalledApplications() functionality for this? 17:28:47 you are (not) 17:29:00 mikeperry: given that we learn about the socket's existence the moment it does a connect() to the end-point on the internet, we will discover the *source* application here 17:29:13 iirc, most notably, Google Play is needed to register push notifcations with a remote notification server 17:29:18 n8fr8[m]: this is for the IP packets in the VPN engine 17:29:25 n8fr8[m]: so we try to get the UID out given a packet 17:29:31 and then we need to map it to a string name later 17:29:38 where i think getInstalledApplications() matters? 17:29:46 n8fr8[m]: we're using getPackageManager().getNameForUid() 17:29:52 got it. 17:30:05 so yes, we are using the packageManager for the last step 17:30:49 mikeperry: there are some functions an app can call that would cause another app/service/package (like Google Play/FIrebase) to generate network traffic 17:32:13 Push notifications work through Firebase, some of the Firebase services have hard dependency on Google play service, but I dont think push notifictions is one of them. I'll double check rn. 17:32:23 but as long as the origin of the app doesn't do anything fucky, then we are pretty good i think 17:32:30 Ankit: cool 17:33:52 so, maybe next steps now ? 17:34:02 yes 17:34:35 i think david and i will be returning to this reality this week ideally and will continue on the interface work 17:34:48 one thing that would be nice to get done is to have the android app being build as part of the gitlab-ci chain 17:35:00 so we can have people get APK files out for testing when we merge code 17:35:09 we did it with fenix for the tor browser team at some point 17:35:19 ok, cool 17:35:37 https://gitlab.torproject.org/tpo/applications/fenix/-/blob/tor-browser-90.0.0b6-11.0-1/.gitlab-ci.yml 17:35:41 probably a lot of this that can be reused 17:35:48 I'm not sure if I have the rights to work with docker runners in tor's CI 17:36:01 hm 17:36:03 i can give you that 17:36:22 ok, cool. Maybe also Ankit 17:36:36 Ankit: you have a user on gitlab.torproject.org yet? 17:37:02 cyberta: it looks like it already works: https://gitlab.torproject.org/cyberta/onionmasq/-/pipelines 17:37:04 I do 17:37:10 it's tied to the project, so when you clone it you get the permissions too 17:37:14 Ankit: what is your username? 17:37:15 perfecto 17:37:48 ankitgusai19 17:38:13 ah, another blocker: 17:38:24 awesome, if you fork the project in https://gitlab.torproject.org/tpo/core/onionmasq/ you should be getting access to everything there too 17:38:26 cyberta: ok? 17:38:28 we cannot cross-compile the rust code right now 17:38:34 why not? 17:38:53 there's an issue with smoltcp 17:39:00 oh really 17:39:01 I didn't spend time on it yet 17:39:11 maybe also just with the build chain version 17:39:13 i'd have thought the issue would be arti's use of libsqlite3 17:39:28 I've created a ticket for it 17:39:30 ok, i will dive into the cross compilation stuff this week 17:39:32 looking 17:39:44 (arti cross-compile well sith the "static" feature flag) 17:39:58 ahhh, that one with the sizeof 17:39:58 yes 17:40:04 i saw that one as well 17:40:05 I started adapting a build script from another software project I'm involved with 17:40:16 trinity[m]: does that pull in the bundled version of sqlite from the C file too? 17:40:23 which uses the same tech stack 17:40:31 ah, they use smoltcp too? 17:40:41 no, but rust as a core lib :) 17:40:44 ah 17:40:48 ahf: bundled sqlite and bundled openssl 17:40:55 trinity[m]: perfect, then i wont worry about that part 8) 17:41:00 trinity[m]: thank you! 17:41:45 cyberta: the issue there is pretty easy to avoid because we can compile smoltcp without the raw IP socket mode 17:41:49 trinity[m]: oh, so you made it already cross-compile? 17:42:19 yes, I've made a very basic app for ios&android that does a single http request over arti 17:42:38 https://gitlab.torproject.org/trinity-1686a/arti-mobile-example 17:42:50 aah, great, thanks for sharing 17:42:53 looks like there is a good option for some knowledge sharing there :-D 17:43:40 almost anything I have to say is aleady in arti repository in doc/{android,ios}.md 17:43:54 then I should rftm :) 17:44:11 read the fine markdown 17:44:23 ok this is perfect 17:44:25 exactly :) 17:44:28 it sounds like we all have stuff for our plates here 17:44:35 right 17:44:52 shall we call it and then we can sync as we progress over the next two weeks and sum things up there? 17:45:07 thanks again for switching to IRC / matrix today 17:45:18 yes 17:45:39 no worries, hope you enjoy your traveling! i am very jelous there haha 17:45:43 talk to you all later o/ 17:45:45 #endmeeting