17:00:23 <hellais> #startmeeting 17:00:23 <MeetBot> Meeting started Mon Jun 22 17:00:23 2015 UTC. The chair is hellais. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:23 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic. 17:00:33 <hellais> here we go 17:00:38 <hellais> so who is here? 17:00:59 <dcf1> Hi hellais. 17:01:28 * sbs is here 17:01:40 <hellais> dcf1: hey :) 17:01:53 <poly> hello 17:02:12 <hellais> vasilis will not be able to attend as he is travelling to tornot 17:02:15 <hellais> *toronto 17:02:27 <hellais> so I would say we start as usual with report backs 17:02:57 <poly> should I start? 17:03:08 <hellais> go for it! 17:03:13 <poly> I've been working some more on network-meter lately 17:03:49 <poly> I've added the ability for network-meter to select, pass arguments and execute a program 17:04:18 <poly> after discussing with hellais, it seemed the next step was to begin working on the plugin parsing code 17:04:35 <poly> for this purpose, I have created a simple testing plugin for traceroute 17:05:09 <hellais> can you share the link of it? 17:05:14 <poly> currently I've implemented the code to load the plugin 17:05:20 <poly> sure! just a second 17:05:25 <hellais> I think sbs may have some good feedback on that part 17:05:52 <poly> https://github.com/measurement-kit/network-meter/tree/simplify/plugins 17:06:15 <poly> the plugin takes form in a zip archive with the extension nmp 17:06:41 <poly> inside you can find the main.json file 17:07:01 <poly> a sample file is provided here https://github.com/measurement-kit/network-meter/blob/simplify/plugins/traceroute/main.json 17:07:13 <hellais> poly: I think we shouldn't commit to the source code repo the zip files though 17:07:17 <sbs> poly: is the .nmp file derived from main.json? 17:07:53 <poly> sbs: it's simply a zip of the traceroute directory 17:08:13 <poly> sbs: we do this so you can include executables and assets within a plugin if you wish to 17:08:29 <poly> hellais: so just keep the source code, no zip? 17:09:01 <sbs> poly: got it, thanks... yes, in this case I am also of the opinion of @hellais that the repository should only contain sources 17:09:02 <hellais> poly: yes 17:09:23 <poly> will fix 17:09:51 <sbs> poly: what is the difference between executing via full path and via global (if I understand correctly the purpose)? 17:10:31 <poly> that's just used to show that you can have multiple paths for network-meter to look for the executable at 17:10:51 <hellais> sbs: that was added so that you can tell the plugin engine to either look for the executable inside of the plugin directory or if it should expect it to be installed system wide 17:11:09 <sbs> sbs: ah, so the program tries the first one and then falls back on the next one, etc.? 17:11:42 <hellais> sbs: usually you would not specify more than one of them 17:12:04 <hellais> but I guess that could be the logic if you decide to do so 17:12:23 <poly> What I want to implement this week is after a user loads a plugin, network-meter should extract it to some local folder. Additionally, netowrk meter should allow for some method of managing the plugins (deleting, etc) 17:12:40 <sbs> understood 17:13:10 <poly> do we want network-meter to be self-container or should it extract the plugins to a folder like ~..network-meter? 17:13:32 <poly> ~/.network-meter rather 17:14:04 <hellais> I think it would be ideal for it to be self contained 17:14:14 <hellais> or rather, there should be a way to make it also be self contained 17:14:34 <hellais> one of the use cases that we were discussing with sbs was that of making a network-meter based app that ships to the app store 17:14:57 <poly> are app store apps not allowed to write to user's directories? 17:14:58 <hellais> since in that case you must ship all the binary code as part of the app and disable downloading of plugins 17:15:03 <sbs> hellais: I agree that both ways should be available, probably the default should be self contained, but advanced users may want to explore, and/or also to play a bit with it making small changes 17:15:22 <hellais> the plugins should be part of the main app 17:15:49 <sbs> poly: you can write but my understand is that you cannot write code that you can later execute 17:16:08 <poly> sbs: the way I'm thinking of doing it is having some kind of switch that disables the "plugin management" interface 17:16:29 <poly> so you create the plugins you need, install them into network-meter, flip the switch and ship the code 17:17:29 <hellais> that sounds good 17:18:06 <poly> only issue I can think of is the ability to add plugins is still there... user could enable it 17:18:15 <poly> I don't know if that violates the terms of the app store 17:18:29 <sbs> poly hellais: ideally the method to disable permanently the functionality to minimize the chances of packaging an app with plugins enabled by mistake 17:18:56 <sbs> * ideally there should be... 17:19:01 <hellais> yeah 17:19:07 <sbs> but I guess we can deal with this a bit later in the dev process 17:19:11 <hellais> I mean as long as it's not exposed from the GUI I think it's fine 17:19:36 <hellais> if it's some obscure config option that the user need to edit some text file to enale it shouldn't be a problem 17:20:17 <hellais> and the code for fetching plugins and installing them should be very well isolated from the rest 17:20:32 <hellais> so it's clear that when the app is built there is no code path that can lead to that 17:21:02 <sbs> hellais: +1, that's exactly what I meant 17:21:57 <poly> sbs: hellais: can be done 17:22:06 <sbs> poly: as additional feedback, I think it could be sensible to validate the parameters for external programs using regular expressions, I added this functionality in the prototype that I was writing 17:22:13 <sbs> see https://github.com/bassosimone/neubot-scheduler/blob/master/etc/neubot/net_tests/traceroute 17:22:59 <poly> sbs: sounds reasonable too 17:23:00 <sbs> and https://github.com/bassosimone/neubot-scheduler/blob/master/neubot_scheduler/router.py#L127 17:23:48 <sbs> poly: I also designed the code to have parameters to start with $ so that other names of argv were not expanded (I think there are cases in which you want some strings to be passed as is and other strings to be instead expanded) 17:24:20 <sbs> poly: the usage of $ or any other char is of course irrelevant, but it may have sense to mark a token as a variable to be expanded 17:25:17 <poly> sbs: can you elaborate on the last part? I don't think I really understand it 17:26:16 <sbs> poly: sure... I think there should a mechanism to identify tokens that should be expanded in the command string to distinguish them from other tokens 17:26:36 <sbs> poly: e.g., "command": "/usr/bin/foo -bar $baz $foobar" 17:26:54 <poly> I see 17:26:57 <sbs> poly: in this example /usr/bin/foo and -bar are unchanged and $baz and $foobar are expanded 17:27:06 <poly> The way I've done it is slightly different 17:27:17 <sbs> poly: I was also saying that the choice of $ is purely arbitrary and any other char could be used 17:27:43 <poly> sbs: https://github.com/measurement-kit/network-meter/blob/simplify/plugins/traceroute/main.json#L36 17:28:07 <poly> here the program will look at every paramter after 'traceroute' and match it to an argument 17:28:47 <poly> the argument is even evaluated depending on whether it is a "flag", "parameter" or "flaggedParameter" 17:28:54 <sbs> poly: I see, it's like a bison parser that expands tokens and all tokens in the command line could be expanded (some to the empty string) 17:29:33 <poly> sbs: yup. So if a user unticks ipv6 in the interface, ipv6 would be replaced with "" 17:30:03 <poly> if the user choosed 30 as max_ttl in the interface, "max_ttl" is replaced with "-m 30" 17:30:10 <poly> etc. 17:30:17 <poly> Does this seem like a reasonable design? 17:30:52 <hellais> I guess the "$" prefix can improve readability 17:31:03 <hellais> and allow you to more explicitly defined strings inside of the command line 17:31:19 <hellais> so if you always want to run traceroute with the --verbose option, for example, 17:31:32 <hellais> you can say "command": "traceroute --version $host $port $bar" 17:31:40 <hellais> I can see some value in doing that 17:31:43 <poly> hellais: makes sense, didn't think of that use case 17:31:49 <poly> will add it 17:32:04 <hellais> yeah I also didn't before sbs pointed it out 17:32:15 <hellais> so should we move onto the next report back? 17:32:28 <sbs> cool, here's my report! 17:32:33 <hellais> sbs: hit it 17:33:07 <sbs> I've been working on the iOS application using measurement-kit to implement OONI tests 17:33:12 <sbs> most interesting achievements: 17:34:13 <sbs> 1) closed a ticket reporting that it was not possible to run a test multiple times (https://github.com/measurement-kit/measurement-kit-app-ios/issues/7) 17:35:13 <sbs> 2) I modified the measurement-kit-build-ios repository to produce iOS framework directories so that integration of measurement-kit is more smooth and in line with iOS/MacOS good practices (https://github.com/measurement-kit/measurement-kit-app-ios/pull/13) 17:35:59 <sbs> 3) I rewrote the code running logic to submit jobs through measurement-kit async interface (https://github.com/measurement-kit/measurement-kit-app-ios/pull/14) that allows to close two other tickets: 17:36:28 <sbs> 3.1) "Fix crash when running DNS injection" https://github.com/measurement-kit/measurement-kit-app-ios/issues/4 17:36:46 <sbs> 3.2) "Tests should be run using the async interface" https://github.com/measurement-kit/measurement-kit-app-ios/issues/9 17:37:20 <sbs> in the process I learned a lot about iOS coding and did some interesting experiments and fixed a bug that was leading the async interface of measurement kit to crash 17:37:43 <hellais> good stuff 17:37:48 <sbs> now I need someone to review the pull request mentioned above, test it and merge it, so that I can move forward with the iOS app 17:37:57 <sbs> </report> 17:38:01 <sbs> hellais: thanks :) 17:38:08 <hellais> ok I shall go 17:38:25 <hellais> so with vasilis beginning of last week we had a call with the HLS clinic 17:38:42 <hellais> we talked about the memo they sent us and defined some next steps top get an informed coonsent procedure 17:38:45 <hellais> fopr 17:38:57 <hellais> lepidopter and ooniprobe 17:39:18 <hellais> I also did a bit of work on preparing the OTF proposal due end of month 17:39:47 <hellais> in terms of development I mainly worked on ooni-sysadmin 17:40:06 <hellais> to cleanup a bit the deployment scripts to support multiple distros 17:40:16 <hellais> in particular adding support for deploying probes on AWS: https://github.com/TheTorProject/ooni-sysadmin/commit/d53fd9c342e5ac753978c53078908d2013e774bd 17:40:34 <hellais> this will allow us to potentially run control probes from all the vantage points offerred by amazon 17:41:18 <hellais> in other news AirVPN offerred to sponsor the costs we are incurring with AWS which is very good 17:41:44 <hellais> EOF 17:42:05 <hellais> dcf1: is there something you wanted to talk about during this gathering? 17:42:26 <dcf1> No, nothing in particular. 17:42:33 <dcf1> Thanks for your detailed responses on ooni-dev. 17:42:45 <dcf1> I'm writing a patch to add meek domains to Citizen Lab test-lists. 17:42:47 <dcf1> That is all. 17:42:55 <hellais> ah great 17:44:41 <hellais> dcf1: you may be interested in expanding the existing bridges.csv: https://github.com/citizenlab/test-lists/blob/master/lib/lists/services/tor/bridges.py 17:45:16 <hellais> dcf1: where do you get the list of domains from? 17:45:18 <dcf1> hellais: except we need to test URLs, not IPs. 17:45:46 <dcf1> I'm making a list of the front domains we currently use (www.google.com, a0.awsstatic.com, ajax.aspnetcdn.com), 17:45:57 <hellais> dcf1: but it already does output meek URLs: https://github.com/citizenlab/test-lists/blob/master/lists/services/tor/bridges.csv#L25 17:46:05 <dcf1> plus the actual domains behind them (e.g. meek-reflect.appspot.com), 17:46:17 <dcf1> plus some alternatives we may want to switch to. 17:46:42 <dcf1> hellais: but the URLs are only in the bridge specification, they are not separately parsed out in the CSV. 17:46:51 <dcf1> I suspected they are not being tested for that reason. 17:47:00 <dcf1> It seems the flashproxy and meek lines do not belong in that file. 17:47:13 <dcf1> Or else someone will be testing nonsense 0.0.1.0 and 0.0.2.0 IPs. 17:47:21 <dcf1> Maybe I don't understand how it works. 17:47:52 <dcf1> I think for the domain-fronting tests, we want a data format more like the country-specific URL lists, i.e., with a "url" column. 17:48:16 <dcf1> I am almost done with the patch; I will Cc you and we can figure out the right format. 17:48:33 <hellais> currently ooni contains logic to parse the address entry and start tor with tit 17:49:08 <hellais> dcf1: ok great, then I will give you more feedback when that is done 17:49:09 <dcf1> hellais: ah, I see. That works for testing the existing bridge lines, then. 17:49:15 <hellais> correct 17:49:21 <dcf1> But I also want to test some URLs that are not currently in use. 17:49:47 <hellais> yes, then it's probably best to put it in a dedicated list 17:49:54 <dcf1> For example add a CloudFlare URL, which you couldn't use with Tor because it's not set up yet, but that we might want to use in the future and know if it is blocked. 17:50:45 <hellais> that makes sense 17:52:33 <hellais> I guess you all already said your next steps so I should say what I will do next 17:53:19 <hellais> tomorrow I leave to go to Toronto for the citizenlab summer institute so I will probably not get much development done 17:53:32 <hellais> my top priority though is working on the OTF proposal and finishing that 17:53:57 <hellais> is there anything else somebody wanted to say/talk about? 17:59:06 <hellais> <audio src="crickets.wav"></audio> 17:59:45 <sbs> hellais: lol 18:00:10 <dcf1> <audio src=crickets.wav loop></audio> 18:00:11 <sbs> hellais: I have nothing to add, except of course that I can help with the proposal, ping me when needed 18:00:56 <hellais> ok great 18:01:01 <hellais> thanks for attending the meeting 18:01:04 <hellais> #endmeeting