17:00:23 <hellais> #startmeeting
17:00:23 <MeetBot> Meeting started Mon Jun 22 17:00:23 2015 UTC.  The chair is hellais. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:23 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
17:00:33 <hellais> here we go
17:00:38 <hellais> so who is here?
17:00:59 <dcf1> Hi hellais.
17:01:28 * sbs is here
17:01:40 <hellais> dcf1: hey :)
17:01:53 <poly> hello
17:02:12 <hellais> vasilis will not be able to attend as he is travelling to tornot
17:02:15 <hellais> *toronto
17:02:27 <hellais> so I would say we start as usual with report backs
17:02:57 <poly> should I start?
17:03:08 <hellais> go for it!
17:03:13 <poly> I've been working some more on network-meter lately
17:03:49 <poly> I've added the ability for network-meter to select, pass arguments and execute a program
17:04:18 <poly> after discussing with hellais, it seemed the next step was to begin working on the plugin parsing code
17:04:35 <poly> for this purpose, I have created a simple testing plugin for traceroute
17:05:09 <hellais> can you share the link of it?
17:05:14 <poly> currently I've implemented the code to load the plugin
17:05:20 <poly> sure! just a second
17:05:25 <hellais> I think sbs may have some good feedback on that part
17:05:52 <poly> https://github.com/measurement-kit/network-meter/tree/simplify/plugins
17:06:15 <poly> the plugin takes form in a zip archive with the extension nmp
17:06:41 <poly> inside you can find the main.json file
17:07:01 <poly> a sample file is provided here https://github.com/measurement-kit/network-meter/blob/simplify/plugins/traceroute/main.json
17:07:13 <hellais> poly: I think we shouldn't commit to the source code repo the zip files though
17:07:17 <sbs> poly: is the .nmp file derived from main.json?
17:07:53 <poly> sbs: it's simply a zip of the traceroute directory
17:08:13 <poly> sbs: we do this so you can include executables and assets within a plugin if you wish to
17:08:29 <poly> hellais: so just keep the source code, no zip?
17:09:01 <sbs> poly: got it, thanks... yes, in this case I am also of the opinion of @hellais that the repository should only contain sources
17:09:02 <hellais> poly: yes
17:09:23 <poly> will fix
17:09:51 <sbs> poly: what is the difference between executing via full path and via global (if I understand correctly the purpose)?
17:10:31 <poly> that's just used to show that you can have multiple paths for network-meter to look for the executable at
17:10:51 <hellais> sbs: that was added so that you can tell the plugin engine to either look for the executable inside of the plugin directory or if it should expect it to be installed system wide
17:11:09 <sbs> sbs: ah, so the program tries the first one and then falls back on the next one, etc.?
17:11:42 <hellais> sbs: usually you would not specify more than one of them
17:12:04 <hellais> but I guess that could be the logic if you decide to do so
17:12:23 <poly> What I want to implement this week is after a user loads a plugin, network-meter should extract it to some local folder. Additionally, netowrk meter should allow for some method of managing the plugins (deleting, etc)
17:12:40 <sbs> understood
17:13:10 <poly> do we want network-meter to be self-container or should it extract the plugins to a folder like ~..network-meter?
17:13:32 <poly> ~/.network-meter rather
17:14:04 <hellais> I think it would be ideal for it to be self contained
17:14:14 <hellais> or rather, there should be a way to make it also be self contained
17:14:34 <hellais> one of the use cases that we were discussing with sbs was that of making a network-meter based app that ships to the app store
17:14:57 <poly> are app store apps not allowed to write to user's directories?
17:14:58 <hellais> since in that case you must ship all the binary code as part of the app and disable downloading of plugins
17:15:03 <sbs> hellais: I agree that both ways should be available, probably the default should be self contained, but advanced users may want to explore, and/or also to play a bit with it making small changes
17:15:22 <hellais> the plugins should be part of the main app
17:15:49 <sbs> poly: you can write but my understand is that you cannot write code that you can later execute
17:16:08 <poly> sbs: the way I'm thinking of doing it is having some kind of switch that disables the "plugin management" interface
17:16:29 <poly> so you create the plugins you need, install them into network-meter, flip the switch and ship the code
17:17:29 <hellais> that sounds good
17:18:06 <poly> only issue I can think of is the ability to add plugins is still there... user could enable it
17:18:15 <poly> I don't know if that violates the terms of the app store
17:18:29 <sbs> poly hellais: ideally the method to disable permanently the functionality to minimize the chances of packaging an app with plugins enabled by mistake
17:18:56 <sbs> * ideally there should be...
17:19:01 <hellais> yeah
17:19:07 <sbs> but I guess we can deal with this a bit later in the dev process
17:19:11 <hellais> I mean as long as it's not exposed from the GUI I think it's fine
17:19:36 <hellais> if it's some obscure config option that the user need to edit some text file to enale it shouldn't be a problem
17:20:17 <hellais> and the code for fetching plugins and installing them should be very well isolated from the rest
17:20:32 <hellais> so it's clear that when the app is built there is no code path that can lead to that
17:21:02 <sbs> hellais: +1, that's exactly what I meant
17:21:57 <poly> sbs: hellais: can be done
17:22:06 <sbs> poly: as additional feedback, I think it could be sensible to validate the parameters for external programs using regular expressions, I added this functionality in the prototype that I was writing
17:22:13 <sbs> see https://github.com/bassosimone/neubot-scheduler/blob/master/etc/neubot/net_tests/traceroute
17:22:59 <poly> sbs: sounds reasonable too
17:23:00 <sbs> and https://github.com/bassosimone/neubot-scheduler/blob/master/neubot_scheduler/router.py#L127
17:23:48 <sbs> poly: I also designed the code to have parameters to start with $ so that other names of argv were not expanded (I think there are cases in which you want some strings to be passed as is and other strings to be instead expanded)
17:24:20 <sbs> poly: the usage of $ or any other char is of course irrelevant, but it may have sense to mark a token as a variable to be expanded
17:25:17 <poly> sbs: can you elaborate on the last part? I don't think I really understand it
17:26:16 <sbs> poly: sure... I think there should a mechanism to identify tokens that should be expanded in the command string to distinguish them from other tokens
17:26:36 <sbs> poly: e.g., "command": "/usr/bin/foo -bar $baz $foobar"
17:26:54 <poly> I see
17:26:57 <sbs> poly: in this example /usr/bin/foo and -bar are unchanged and $baz and $foobar are expanded
17:27:06 <poly> The way I've done it is slightly different
17:27:17 <sbs> poly: I was also saying that the choice of $ is purely arbitrary and any other char could be used
17:27:43 <poly> sbs: https://github.com/measurement-kit/network-meter/blob/simplify/plugins/traceroute/main.json#L36
17:28:07 <poly> here the program will look at every paramter after 'traceroute' and match it to an argument
17:28:47 <poly> the argument is even evaluated depending on whether it is a "flag", "parameter" or "flaggedParameter"
17:28:54 <sbs> poly: I see, it's like a bison parser that expands tokens and all tokens in the command line could be expanded (some to the empty string)
17:29:33 <poly> sbs: yup. So if a user unticks ipv6 in the interface, ipv6 would be replaced with ""
17:30:03 <poly> if the user choosed 30 as max_ttl in the interface, "max_ttl" is replaced with "-m 30"
17:30:10 <poly> etc.
17:30:17 <poly> Does this seem like a reasonable design?
17:30:52 <hellais> I guess the "$" prefix can improve readability
17:31:03 <hellais> and allow you to more explicitly defined strings inside of the command line
17:31:19 <hellais> so if you always want to run traceroute with the --verbose option, for example,
17:31:32 <hellais> you can say "command": "traceroute --version $host $port $bar"
17:31:40 <hellais> I can see some value in doing that
17:31:43 <poly> hellais:  makes sense, didn't think of that use case
17:31:49 <poly> will add it
17:32:04 <hellais> yeah I also didn't before sbs pointed it out
17:32:15 <hellais> so should we move onto the next report back?
17:32:28 <sbs> cool, here's my report!
17:32:33 <hellais> sbs: hit it
17:33:07 <sbs> I've been working on the iOS application using measurement-kit to implement OONI tests
17:33:12 <sbs> most interesting achievements:
17:34:13 <sbs> 1) closed a ticket reporting that it was not possible to run a test multiple times (https://github.com/measurement-kit/measurement-kit-app-ios/issues/7)
17:35:13 <sbs> 2) I modified the measurement-kit-build-ios repository to produce iOS framework directories so that integration of measurement-kit is more smooth and in line with iOS/MacOS good practices (https://github.com/measurement-kit/measurement-kit-app-ios/pull/13)
17:35:59 <sbs> 3) I rewrote the code running logic to submit jobs through measurement-kit async interface (https://github.com/measurement-kit/measurement-kit-app-ios/pull/14) that allows to close two other tickets:
17:36:28 <sbs> 3.1) "Fix crash when running DNS injection" https://github.com/measurement-kit/measurement-kit-app-ios/issues/4
17:36:46 <sbs> 3.2) "Tests should be run using the async interface" https://github.com/measurement-kit/measurement-kit-app-ios/issues/9
17:37:20 <sbs> in the process I learned a lot about iOS coding and did some interesting experiments and fixed a bug that was leading the async interface of measurement kit to crash
17:37:43 <hellais> good stuff
17:37:48 <sbs> now I need someone to review the pull request mentioned above, test it and merge it, so that I can move forward with the iOS app
17:37:57 <sbs> </report>
17:38:01 <sbs> hellais: thanks :)
17:38:08 <hellais> ok I shall go
17:38:25 <hellais> so with vasilis beginning of last week we had a call with the HLS clinic
17:38:42 <hellais> we talked about the memo they sent us and defined some next steps top get an  informed coonsent procedure
17:38:45 <hellais> fopr
17:38:57 <hellais> lepidopter and ooniprobe
17:39:18 <hellais> I also did a bit of work on preparing the OTF proposal due end of month
17:39:47 <hellais> in terms of development I mainly worked on ooni-sysadmin
17:40:06 <hellais> to cleanup a bit the deployment scripts to support multiple distros
17:40:16 <hellais> in particular adding support for deploying probes on AWS: https://github.com/TheTorProject/ooni-sysadmin/commit/d53fd9c342e5ac753978c53078908d2013e774bd
17:40:34 <hellais> this will allow us to potentially run control probes from all the vantage points offerred by amazon
17:41:18 <hellais> in other news AirVPN offerred to sponsor the costs we are incurring with AWS which is very good
17:41:44 <hellais> EOF
17:42:05 <hellais> dcf1: is there something you wanted to talk about during this gathering?
17:42:26 <dcf1> No, nothing in particular.
17:42:33 <dcf1> Thanks for your detailed responses on ooni-dev.
17:42:45 <dcf1> I'm writing a patch to add meek domains to Citizen Lab test-lists.
17:42:47 <dcf1> That is all.
17:42:55 <hellais> ah great
17:44:41 <hellais> dcf1: you may be interested in expanding the existing bridges.csv: https://github.com/citizenlab/test-lists/blob/master/lib/lists/services/tor/bridges.py
17:45:16 <hellais> dcf1: where do you get the list of domains from?
17:45:18 <dcf1> hellais: except we need to test URLs, not IPs.
17:45:46 <dcf1> I'm making a list of the front domains we currently use (www.google.com, a0.awsstatic.com, ajax.aspnetcdn.com),
17:45:57 <hellais> dcf1: but it already does output meek URLs: https://github.com/citizenlab/test-lists/blob/master/lists/services/tor/bridges.csv#L25
17:46:05 <dcf1> plus the actual domains behind them (e.g. meek-reflect.appspot.com),
17:46:17 <dcf1> plus some alternatives we may want to switch to.
17:46:42 <dcf1> hellais: but the URLs are only in the bridge specification, they are not separately parsed out in the CSV.
17:46:51 <dcf1> I suspected they are not being tested for that reason.
17:47:00 <dcf1> It seems the flashproxy and meek lines do not belong in that file.
17:47:13 <dcf1> Or else someone will be testing nonsense 0.0.1.0 and 0.0.2.0 IPs.
17:47:21 <dcf1> Maybe I don't understand how it works.
17:47:52 <dcf1> I think for the domain-fronting tests, we want a data format more like the country-specific URL lists, i.e., with a "url" column.
17:48:16 <dcf1> I am almost done with the patch; I will Cc you and we can figure out the right format.
17:48:33 <hellais> currently ooni contains logic to parse the address entry and start tor with tit
17:49:08 <hellais> dcf1: ok great, then I will give you more feedback when that is done
17:49:09 <dcf1> hellais: ah, I see. That works for testing the existing bridge lines, then.
17:49:15 <hellais> correct
17:49:21 <dcf1> But I also want to test some URLs that are not currently in use.
17:49:47 <hellais> yes, then it's probably best to put it in a dedicated list
17:49:54 <dcf1> For example add a CloudFlare URL, which you couldn't use with Tor because it's not set up yet, but that we might want to use in the future and know if it is blocked.
17:50:45 <hellais> that makes sense
17:52:33 <hellais> I guess you all already said your next steps so I should say what I will do next
17:53:19 <hellais> tomorrow I leave to go to Toronto for the citizenlab summer institute so I will probably not get much development done
17:53:32 <hellais> my top priority though is working on the OTF proposal and finishing that
17:53:57 <hellais> is there anything else somebody wanted to say/talk about?
17:59:06 <hellais> <audio src="crickets.wav"></audio>
17:59:45 <sbs> hellais: lol
18:00:10 <dcf1> <audio src=crickets.wav loop></audio>
18:00:11 <sbs> hellais: I have nothing to add, except of course that I can help with the proposal, ping me when needed
18:00:56 <hellais> ok great
18:01:01 <hellais> thanks for attending the meeting
18:01:04 <hellais> #endmeeting