16:59:52 #startmeeting 16:59:52 Meeting started Mon Apr 4 16:59:52 2016 UTC. The chair is hellais. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:59:52 Useful Commands: #action #agreed #help #info #idea #link #topic. 17:00:11 * willscott waves 17:00:22 so who is here? 17:00:25 here 17:00:31 hello 17:00:38 hey all 17:00:51 hello 17:01:46 excellent, so what have you been up to last week? 17:03:05 hellpo 17:03:20 nothing ooni-related by me 17:03:41 * sbs here! 17:04:02 Alright, so I've been working with hellais on the ooni API, which we've decided to rename "HTTP Control Protocol". I spent a little while fleshing out a fuller specification, which you can find here https://github.com/0xPoly/ooni-probe/blob/HTTP_Interface/docs/source/control_interface.rst 17:04:25 I worked on fixing bug in the json_reports branch, started to sketch out the web API to support the GUI of OONI, began adding JSON support to MK and fixed some bugs in ooniprobe. 17:04:49 We discussed implementation strategies and I believe hellais has started experimenting with some code 17:06:05 wrote a blog for OONI which will be released at a later stage EOF. 17:06:18 I was traveling for part of the week, so was not very effective; still I managed to push some hotfixes and preparatory changes for MK 0.2 and I also completed the review of the JSON pull request for ooni (i.e. #456) which was merged just few minutes ago. EOF 17:07:29 oh wow, the json reports thing is merged -- nice job 17:08:13 I haven't done anything on OONI since launch, as I was in Poland. I'm curious which issues are most important for me to tackle hellais 17:08:58 Hi 17:09:43 nuke: hi! 17:11:14 I was at Rightscon working on how to collaborate better with censorship circumvention tools developers; helping us to test and create tests as well as submitting measurements to the pipeline. Many groups and people asked on how they can contribute to OONI and especially how they can contribute to the URL lists (https://github.com/citizenlab/test-lists/pull/35). I gave a short demo of OONI explorer 17:11:16 people and had very good feedback --people like it! EOF 17:12:23 anadahz: that's great to hear! 17:12:46 anadahz: very good! 17:15:18 anadahz: any particular takeaways from rightscon worthwhile sharing? 17:16:37 hellais: wrt to OONI explorer people have been asking to get a list or dump of all reports per country 17:17:26 anadahz: this is something different than what is available at measurements.ooni.torproject.org? 17:18:56 hellais: yes these measurements^are per date so they were asking how they can get all of them in a per country way 17:20:20 Alkasir has started to collect some reports which they will happily submit to OONI pipeline 17:21:49 anadahz: hum I see, well they can list all the reports for a given country from the explorer API and then use the filenames to download the reports. 17:22:07 anadahz: I don't think we want to be duplicating the data to support every possible way of clustering them though 17:22:56 hellais: perhaps we could make this on-demand via the API? 17:23:13 [13measurement-kit] 15bassosimone created 06bassosimone-patch-1 (+1 new commit): 02https://git.io/vVEeq 17:23:13 13measurement-kit/06bassosimone-patch-1 144cc16e4 15Simone Basso: README.md: mention Neubot's MPEG DASH test 17:23:34 anadahz: what do you mean by on-demand? 17:23:35 [13measurement-kit] 15bassosimone opened pull request #443: README.md: mention Neubot's MPEG DASH test (06master...06bassosimone-patch-1) 02https://git.io/vVEeG 17:23:45 per-country and per-test seem like desirable slices 17:24:21 could be an automatic symlink process to give directory views or just a file with the relevant test urls that make it easy to grab those subsets 17:25:12 hellais: like the API should provide a json dump upon a selection of reports 17:25:50 hellais: sort of an export functionality 17:28:34 Alkasir (https://alkasir.com/) has started to collect more blocking measurements reports which they will happily submit to OONI pipeline 17:28:35 hum, yeah I guess we could have something like that 17:30:09 anadahz: yes I remember exchanging some emails with them about that, though the last time I spoke to them, it didn't seem like they would have time to implement submission to the OONI pipeline, but we would have to write our own integration with their DB 17:30:16 anadahz: has that changed? 17:31:21 hellais: yes they have not yet sorted this out due to some dev issues 17:33:04 poly: the HTTP Control Protocol is based on this: https://ooni.readthedocs.org/en/latest/architecture.html#ooniprobe-api ? 17:33:21 correct 17:33:50 it's updated to account for decks, which were not a part of OONI when that section was originally written 17:34:35 moved to to a seperate file to make it easier to find 17:35:44 poly: The Authentication issue in: https://github.com/0xPoly/ooni-probe/blob/HTTP_Interface/docs/source/control_interface.rst#id16 could be solved by making this service available only to localhost 17:37:50 In other news I got quite many questions on how one can install ooniprobe. We should really work on the packaging and testing of ooniprobe. 17:38:10 anadahz: I discussed this with hellais. For GUI purposes, serving to 127.0.0.1 only is the easiest solution. But we are also considering using the same protocol for controlling ooniprobe on a RPi, so authentication might come in handy in the future 17:39:14 Perhaps it's even possible to run ooniprobe in Windows now: https://blogs.windows.com/buildingapps/2016/03/30/run-bash-on-ubuntu-on-windows/ 17:40:48 anadahz: while possible, not simple to package or make usable for windows users 17:41:39 anadahz: there are many roadblocks to getting ooniprobe running on windows: * Making all scapy based stuff a soft depedency and make it run without them (there is no way of automating the installation in winpcap required to use scapy) * There are quite a few places where we assume the filesystem to follow posix compatible * Having a GUI (without a GUI I don't think it makes sense to even attempt targetting w 17:41:46 indows users) 17:42:11 poly: like users controlling the RasPi in their LAN? 17:42:52 anadahz: sure, or possibly for remote control over the internet is you set it up that way 17:43:06 hellais: willscott indeed but it's a good start 17:43:42 poly: you can share a localhost service over Tor HS 17:44:41 anadahz: good point, that would make things much easier 17:45:30 still even if it's hosted as a tor hidden service it should have authentication 17:45:55 hellais: in previous discussions i've been more optimistic about MK being a solution for windows, since c code is relatively portable across platforms 17:47:41 hellais: yes we could cookie based authentication 17:48:00 willscott: yeah that is also another option, though MK is still distant from feature parity with ooniprobe and even when the library is complete we would need to put some GUI on top it nonetheless 17:48:04 either way, I don't think we should worry too much about authentication at this stage 17:49:11 yup 17:52:17 so I guess we should proceed into the next steps 17:52:23 [13measurement-kit] 15bassosimone closed pull request #443: README.md: mention Neubot's MPEG DASH test (06master...06bassosimone-patch-1) 02https://git.io/vVEeG 17:52:24 [13measurement-kit] 15bassosimone pushed 1 new commit to 06master: 02https://git.io/vVEIR 17:52:24 13measurement-kit/06master 144a1e12c 15Simone Basso: README.md: mention Neubot's MPEG DASH test 17:54:42 bnvk: regarding things to be done I think this would be a very useful addition to the explorer: https://github.com/TheTorProject/ooni-explorer/issues/75 17:54:55 [13measurement-kit] 15bassosimone 04force-pushed 06feature/ndt from 143ee87c6 to 14d370a35: 02https://git.io/vVR0b 17:54:55 13measurement-kit/06feature/ndt 14d370a35 15Simone Basso: Start working to NDT client implementation 17:57:57 my next steps are: make changes to the ooni-pipeline to handle 10s of thousands of small reports (this lead to the reports from the past 3 days not being spooled properly), work on the JSON support in MK and the web connectivity test in ooniprobe 17:58:27 hellais: cool, thanks for focusing that for me! 17:59:09 I will create a PR with the spec and hopefully start working on implementing it 17:59:35 still also need to decide on which web framework (if any) to use for the GUI - suggestions are welcome 18:00:30 I will work to enable submitting OONI reports using HTTPS and will also work on the NDT client - EOF 18:01:13 next steps: improve ooniprobe packaging and installation, continuing on documenting the OONI explorer EOF 18:01:22 Sbs: I like that ;) 18:03:11 [13ooni-probe] 150xPoly opened pull request #473: OONI Control Interface (06master...06HTTP_Interface) 02https://git.io/vVEqT 18:03:53 nuke: :) 18:05:38 ok well if there is nothing else, that's all folks 18:05:41 thanks for attending 18:05:48 #endmeeting