17:02:37 #startmeeting 17:02:37 Meeting started Mon Jun 8 17:02:37 2015 UTC. The chair is hellais. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:02:37 Useful Commands: #action #agreed #help #info #idea #link #topic. 17:02:42 hey! 17:02:52 who is in da house? 17:03:18 * sbs is here 17:07:45 I guess also aagbsn 17:07:58 so let's start with some report backs 17:08:20 sbs: do you want to update us a bit on what has happenned on libight in the past month 17:08:32 hellais: yes 17:08:42 (I have not yet written the monthly ooni status report so ever bit of information is gold for me) 17:08:47 actually, it did not happen a lot in terms of commits and/or pull requests 17:09:36 we continued to discuss about the possible contract with mlab to develop an NDT plugin for libight 17:10:08 I've also started to look at how to interface C++ and Java using JNI 17:10:53 I also did some research regarding best practices on how to structure C++ code to make sure libight is good quality 17:11:10 specifically I looked a thinking in C++ volume {1,2} 17:11:34 ah thos eare good books 17:11:48 they are the first programming books I ever owned 17:12:03 yes, volume 2 in particular was very interesting to read 17:12:12 hellais: (yes, I am here) 17:12:38 sbs: can you tell us a bit about the updates on the libight-ios front? 17:12:53 specifically the work nuke did on the app 17:13:08 I cannot tell you much for now 17:13:16 it is in the queue for me to deeply look at it 17:13:24 speak of the devil ;) 17:13:27 but have not done yet 17:13:29 hellais: lol 17:13:55 nuke: we were just talking about libight-ios 17:14:10 nuke: can you give us a little update on the development of the app 17:14:29 (i.e. what's done, what needs review, etc.) 17:14:45 hi! 17:16:04 well, we automatized the building of libight libraries in the xcode app 17:16:19 now we need to test the app, and fix every crash 17:17:43 ok cool\ 17:18:00 is there some things in particular that would need swift review? 17:18:18 I will be in turin with simone at the end of this week and we will probably be working on libight 17:27:00 well I guess in the meantime I will proceed with my status update 17:27:20 last week was mostly dedicated to the data processing pipeline 17:28:05 at this point the data pipeline is fully operational with the only caveaut being the fact that I trigged it manually as opposed to running it from a cronjob 17:29:03 the reason for doing so is that I am not yet fully convinced of the scripts I wrote to fire up amazon ec2 instances and I want to avoid wasting tons of $$ on instances that are not terminated properly :P 17:29:19 hellais: lol 17:29:42 you can see all the imported reports from a confortable UI: http://api.ooni.io/ 17:30:02 download the reports you are interested in (that are transparently proxied via s3) 17:30:26 and identify reports that present anomalies (that can be symptom of censorship) 17:30:43 we currently just have the most basic heuristic for only the http_requests test 17:31:07 but I made it easy to support other tests and more complex heuristics 17:31:30 the magic is here: https://github.com/TheTorProject/ooni-pipeline-ng/blob/master/pipeline/batch/spark_apps.py#L98 17:32:41 entries is a pyspark dataframe that contains all the reports collected for that specific type of test: https://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.DataFrame 17:33:16 on this dataframe you can run any SparkSQL query and it will be run distributed on the spark cluster 17:33:37 the current heuristic takes just some minutes to run on the whole dataset 17:33:59 if it starts taking too much we add another node and we are happy 17:34:27 the code for the frontend and API is here: https://github.com/TheTorProject/ooni-api 17:34:43 it's based on angular.js and this very neat node.js framework called loopback 17:35:18 the very cool thing about loopback is that it includes a native angular.js code generator to create the client side code based on the server side code 17:35:56 the current architecture of the pipeline can be seen here: https://raw.githubusercontent.com/TheTorProject/ooni-pipeline-ng/master/docs/ooni-pipeline-ng-architecture.png 17:36:23 with vasilis we have been also setting up the dedicated server at his university 17:37:00 lot's of profanity was generated in dealing with propriertary server remote admin tools 17:37:28 eventually we managed to get the main server to recognize all the 4 CPUs 17:37:33 but the RAM is not yet being detected 17:38:24 we also found a good deal on 2 Dell PowerEdge 1950 III and they should arrive this week 17:38:57 one of them has 2x Quad code CPUs and the other has 2x Dual core 17:39:45 16GB and 4GB of RAM respectively upgradable to 32GB 17:40:57 any questions? 17:42:54 cool stuff, loopback looks awesome! 17:43:20 yeah it's developped by the person that made express.js 17:43:23 hellais: (you passed me a blog post via chat last weekend that I eventually lost, was this blog post about loopback?) 17:45:23 sbs: not sure, what was it about? 17:46:04 a way to autogenerate something (an API?) from a standard specifying it 17:46:42 ah yes then it's probably about the loopback swagger code generator 17:46:50 that is what I used to bootstrap the azpp 17:46:52 *app 17:47:29 basically you specify the API using swagger (that is a YAML/JSON based specification language) and it will generate the boilerplate code and comments 17:48:20 hellais: cool! 17:48:45 aagbsn: do you have something you wanted to talk about? 17:49:37 I met with some people re: EU funding opportunities, but I dont think it's ready to comment extensively here on yet 17:50:21 basically there are some CFP applying to analysis of "Big Data" that OONI might be a good fit 17:51:41 however the deadline is late Aug, we'd need to be an EU entity and partner with other orgs including industry - one thing that comes to mind is our chat with the bbc who wanted to know when/where they are blocked around the world 17:53:52 uhm late august is quite soon indeed 17:54:05 yes :) but not impossible 17:54:19 also EU proposal are **very** compliated things 17:54:28 I think sbs can perhaps tell you more about them 17:54:37 that is true, the person I spoke with offered to help with questions and review 17:55:01 (and has successfully received/managed eu grants) 17:56:43 so, though we lack experience here, we might have good support in applying -- I think it's worth trying 17:57:23 oh yes indeed if you think you have some serious amount of time to dedicate to this, then by all means go for it 17:58:55 that isn't so clear as I am proceeding with contracting with OTF to work on TorFlow, starting soon 17:59:44 from what I heard, it is a few hours/day of work to organize/write the proposal with the other groups 17:59:59 for ~2 months 18:00:02 speaking of EU proposals, there are many kind of those, however in my experience (limited to only a few kind of EU research proposals) one need to write about 70-80 pages and is generally quite an effort 18:00:19 yes - though they stressed the cut-n-paste strategy 18:00:38 the first 15-20 pages that outline the project and its impact are the most important, supposedly 18:00:39 if given the pointer to this funding scheme and call, I can perhaps be more specific and/or ask to someone here who know better than me 18:01:24 ironically I haven't reviewed all the cfp yet - eu is blocking tor :( 18:01:36 but i can link you to all of the related ones for ooni+tor 18:01:45 aaggns: yeah, cut and paste helps, however, again in the cases that I have seen, calls have quite a precise scope for which you need to cast the proposal 18:02:20 aagbsn: and this is the effort-y part 18:02:33 https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/topics/1113-fct-01-2015.html 18:02:36 https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/topics/1116-fct-04-2015.html 18:02:39 https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/topics/1117-fct-06-2015.html 18:02:42 https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/topics/1052-ds-03-2015.html 18:02:45 https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/topics/1053-ds-04-2015.html 18:02:48 https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/topics/1054-ds-05-2015.html 18:02:51 https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/topics/9071-ds-07-2015.html 18:03:52 there -might- be another cfp in the same topics this january (every 6mo or so), so it might not be a bad idea to take a stab at this and try again in january if we fail 18:04:45 * sbs skimming through these links 18:06:51 uhm, it looks like there is lot to read here 18:06:58 I guess I'll return you a deferred 18:07:00 :) 18:07:01 ;) 18:07:06 ok, thanks for taking a look :) 18:07:14 I can also mail the .htmls to you 18:07:26 I guess I can find your email in the ooni list 18:08:03 or hellais can give it to me using another channel (easier) 18:08:19 that works - it's the same as in my github: aagbsn@extc.org 18:09:32 supposedly ec wants to harmonize their application process with us institutions - make it simpler and less bureaucratic 18:10:37 "harmonize", that's very PRC of them 18:11:12 harmonize is the word I used 18:11:27 I just meant that they want to make the process similar 18:11:27 then of you :P 18:11:46 ack 18:12:17 yes, that's what they told when I went to the infodays in Paris, yet the burden was quite high in the last round of proposals 18:12:30 ah, you went to the infodays? 18:12:41 anyone interesting there? 18:16:28 aagbsn: it was an infoday for a now-expired call to which as Nexa Center we submitted a proposal back in April 18:16:40 aagbsn: that is, it was not in the same round of the calls you sent me 18:17:01 ah, you haven't heard back yet then? 18:17:02 should we quickly discuss next steps and wrap this up? (we are already a bit overtime) 18:17:17 aagbsn: it was in Paris at end of February, there was interesting people and some of them are part of this proposal 18:17:42 aagbsn: no, it takes about six months to hear back from the EU after a proposal 18:17:46 hellais: agreed 18:18:18 I will continue the work on the data pipeline and start writing documentation for it as well as a tor blog post about it 18:18:42 the last part of this week I will be in Turin with simone for the nexa board meeting 18:19:03 sbs: BTW will anna be there too? 18:19:17 because we really need to fix the dates of the hackathon ASAP 18:19:56 hellais: yes 18:20:09 hellais: Anna will be there 18:20:27 (gotta roll out shortly) 18:22:29 ok excellent 18:22:47 anybody else want to share some next steps? 18:27:39 as you said, when we meet in Turin, we'll work on libight 18:27:52 I'll try to commit some time to review the iOS app, also 18:30:06 ok great 18:30:24 well if there is nothing more then that's it folks 18:31:08 #endmeeting