17:02:37 <hellais> #startmeeting 17:02:37 <MeetBot> Meeting started Mon Jun 8 17:02:37 2015 UTC. The chair is hellais. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:02:37 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic. 17:02:42 <hellais> hey! 17:02:52 <hellais> who is in da house? 17:03:18 * sbs is here 17:07:45 <hellais> I guess also aagbsn 17:07:58 <hellais> so let's start with some report backs 17:08:20 <hellais> sbs: do you want to update us a bit on what has happenned on libight in the past month 17:08:32 <sbs> hellais: yes 17:08:42 <hellais> (I have not yet written the monthly ooni status report so ever bit of information is gold for me) 17:08:47 <sbs> actually, it did not happen a lot in terms of commits and/or pull requests 17:09:36 <sbs> we continued to discuss about the possible contract with mlab to develop an NDT plugin for libight 17:10:08 <sbs> I've also started to look at how to interface C++ and Java using JNI 17:10:53 <sbs> I also did some research regarding best practices on how to structure C++ code to make sure libight is good quality 17:11:10 <sbs> specifically I looked a thinking in C++ volume {1,2} 17:11:34 <hellais> ah thos eare good books 17:11:48 <hellais> they are the first programming books I ever owned 17:12:03 <sbs> yes, volume 2 in particular was very interesting to read 17:12:12 <aagbsn> hellais: (yes, I am here) 17:12:38 <hellais> sbs: can you tell us a bit about the updates on the libight-ios front? 17:12:53 <hellais> specifically the work nuke did on the app 17:13:08 <sbs> I cannot tell you much for now 17:13:16 <sbs> it is in the queue for me to deeply look at it 17:13:24 <hellais> speak of the devil ;) 17:13:27 <sbs> but have not done yet 17:13:29 <sbs> hellais: lol 17:13:55 <hellais> nuke: we were just talking about libight-ios 17:14:10 <hellais> nuke: can you give us a little update on the development of the app 17:14:29 <hellais> (i.e. what's done, what needs review, etc.) 17:14:45 <nuke> hi! 17:16:04 <nuke> well, we automatized the building of libight libraries in the xcode app 17:16:19 <nuke> now we need to test the app, and fix every crash 17:17:43 <hellais> ok cool\ 17:18:00 <hellais> is there some things in particular that would need swift review? 17:18:18 <hellais> I will be in turin with simone at the end of this week and we will probably be working on libight 17:27:00 <hellais> well I guess in the meantime I will proceed with my status update 17:27:20 <hellais> last week was mostly dedicated to the data processing pipeline 17:28:05 <hellais> at this point the data pipeline is fully operational with the only caveaut being the fact that I trigged it manually as opposed to running it from a cronjob 17:29:03 <hellais> the reason for doing so is that I am not yet fully convinced of the scripts I wrote to fire up amazon ec2 instances and I want to avoid wasting tons of $$ on instances that are not terminated properly :P 17:29:19 <sbs> hellais: lol 17:29:42 <hellais> you can see all the imported reports from a confortable UI: http://api.ooni.io/ 17:30:02 <hellais> download the reports you are interested in (that are transparently proxied via s3) 17:30:26 <hellais> and identify reports that present anomalies (that can be symptom of censorship) 17:30:43 <hellais> we currently just have the most basic heuristic for only the http_requests test 17:31:07 <hellais> but I made it easy to support other tests and more complex heuristics 17:31:30 <hellais> the magic is here: https://github.com/TheTorProject/ooni-pipeline-ng/blob/master/pipeline/batch/spark_apps.py#L98 17:32:41 <hellais> entries is a pyspark dataframe that contains all the reports collected for that specific type of test: https://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.DataFrame 17:33:16 <hellais> on this dataframe you can run any SparkSQL query and it will be run distributed on the spark cluster 17:33:37 <hellais> the current heuristic takes just some minutes to run on the whole dataset 17:33:59 <hellais> if it starts taking too much we add another node and we are happy 17:34:27 <hellais> the code for the frontend and API is here: https://github.com/TheTorProject/ooni-api 17:34:43 <hellais> it's based on angular.js and this very neat node.js framework called loopback 17:35:18 <hellais> the very cool thing about loopback is that it includes a native angular.js code generator to create the client side code based on the server side code 17:35:56 <hellais> the current architecture of the pipeline can be seen here: https://raw.githubusercontent.com/TheTorProject/ooni-pipeline-ng/master/docs/ooni-pipeline-ng-architecture.png 17:36:23 <hellais> with vasilis we have been also setting up the dedicated server at his university 17:37:00 <hellais> lot's of profanity was generated in dealing with propriertary server remote admin tools 17:37:28 <hellais> eventually we managed to get the main server to recognize all the 4 CPUs 17:37:33 <hellais> but the RAM is not yet being detected 17:38:24 <hellais> we also found a good deal on 2 Dell PowerEdge 1950 III and they should arrive this week 17:38:57 <hellais> one of them has 2x Quad code CPUs and the other has 2x Dual core 17:39:45 <hellais> 16GB and 4GB of RAM respectively upgradable to 32GB 17:40:57 <hellais> any questions? 17:42:54 <sbs> cool stuff, loopback looks awesome! 17:43:20 <hellais> yeah it's developped by the person that made express.js 17:43:23 <sbs> hellais: (you passed me a blog post via chat last weekend that I eventually lost, was this blog post about loopback?) 17:45:23 <hellais> sbs: not sure, what was it about? 17:46:04 <sbs> a way to autogenerate something (an API?) from a standard specifying it 17:46:42 <hellais> ah yes then it's probably about the loopback swagger code generator 17:46:50 <hellais> that is what I used to bootstrap the azpp 17:46:52 <hellais> *app 17:47:29 <hellais> basically you specify the API using swagger (that is a YAML/JSON based specification language) and it will generate the boilerplate code and comments 17:48:20 <sbs> hellais: cool! 17:48:45 <hellais> aagbsn: do you have something you wanted to talk about? 17:49:37 <aagbsn> I met with some people re: EU funding opportunities, but I dont think it's ready to comment extensively here on yet 17:50:21 <aagbsn> basically there are some CFP applying to analysis of "Big Data" that OONI might be a good fit 17:51:41 <aagbsn> however the deadline is late Aug, we'd need to be an EU entity and partner with other orgs including industry - one thing that comes to mind is our chat with the bbc who wanted to know when/where they are blocked around the world 17:53:52 <hellais> uhm late august is quite soon indeed 17:54:05 <aagbsn> yes :) but not impossible 17:54:19 <hellais> also EU proposal are **very** compliated things 17:54:28 <hellais> I think sbs can perhaps tell you more about them 17:54:37 <aagbsn> that is true, the person I spoke with offered to help with questions and review 17:55:01 <aagbsn> (and has successfully received/managed eu grants) 17:56:43 <aagbsn> so, though we lack experience here, we might have good support in applying -- I think it's worth trying 17:57:23 <hellais> oh yes indeed if you think you have some serious amount of time to dedicate to this, then by all means go for it 17:58:55 <aagbsn> that isn't so clear as I am proceeding with contracting with OTF to work on TorFlow, starting soon 17:59:44 <aagbsn> from what I heard, it is a few hours/day of work to organize/write the proposal with the other groups 17:59:59 <aagbsn> for ~2 months 18:00:02 <sbs> speaking of EU proposals, there are many kind of those, however in my experience (limited to only a few kind of EU research proposals) one need to write about 70-80 pages and is generally quite an effort 18:00:19 <aagbsn> yes - though they stressed the cut-n-paste strategy 18:00:38 <aagbsn> the first 15-20 pages that outline the project and its impact are the most important, supposedly 18:00:39 <sbs> if given the pointer to this funding scheme and call, I can perhaps be more specific and/or ask to someone here who know better than me 18:01:24 <aagbsn> ironically I haven't reviewed all the cfp yet - eu is blocking tor :( 18:01:36 <aagbsn> but i can link you to all of the related ones for ooni+tor 18:01:45 <sbs> aaggns: yeah, cut and paste helps, however, again in the cases that I have seen, calls have quite a precise scope for which you need to cast the proposal 18:02:20 <sbs> aagbsn: and this is the effort-y part 18:02:33 <aagbsn> https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/topics/1113-fct-01-2015.html 18:02:36 <aagbsn> https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/topics/1116-fct-04-2015.html 18:02:39 <aagbsn> https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/topics/1117-fct-06-2015.html 18:02:42 <aagbsn> https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/topics/1052-ds-03-2015.html 18:02:45 <aagbsn> https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/topics/1053-ds-04-2015.html 18:02:48 <aagbsn> https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/topics/1054-ds-05-2015.html 18:02:51 <aagbsn> https://ec.europa.eu/research/participants/portal/desktop/en/opportunities/h2020/topics/9071-ds-07-2015.html 18:03:52 <aagbsn> there -might- be another cfp in the same topics this january (every 6mo or so), so it might not be a bad idea to take a stab at this and try again in january if we fail 18:04:45 * sbs skimming through these links 18:06:51 <sbs> uhm, it looks like there is lot to read here 18:06:58 <sbs> I guess I'll return you a deferred 18:07:00 <sbs> :) 18:07:01 <aagbsn> ;) 18:07:06 <aagbsn> ok, thanks for taking a look :) 18:07:14 <sbs> I can also mail the .htmls to you 18:07:26 <sbs> I guess I can find your email in the ooni list 18:08:03 <sbs> or hellais can give it to me using another channel (easier) 18:08:19 <aagbsn> that works - it's the same as in my github: aagbsn@extc.org 18:09:32 <aagbsn> supposedly ec wants to harmonize their application process with us institutions - make it simpler and less bureaucratic 18:10:37 <hellais> "harmonize", that's very PRC of them 18:11:12 <aagbsn> harmonize is the word I used 18:11:27 <aagbsn> I just meant that they want to make the process similar 18:11:27 <hellais> then of you :P 18:11:46 <hellais> ack 18:12:17 <sbs> yes, that's what they told when I went to the infodays in Paris, yet the burden was quite high in the last round of proposals 18:12:30 <aagbsn> ah, you went to the infodays? 18:12:41 <aagbsn> anyone interesting there? 18:16:28 <sbs> aagbsn: it was an infoday for a now-expired call to which as Nexa Center we submitted a proposal back in April 18:16:40 <sbs> aagbsn: that is, it was not in the same round of the calls you sent me 18:17:01 <aagbsn> ah, you haven't heard back yet then? 18:17:02 <hellais> should we quickly discuss next steps and wrap this up? (we are already a bit overtime) 18:17:17 <sbs> aagbsn: it was in Paris at end of February, there was interesting people and some of them are part of this proposal 18:17:42 <sbs> aagbsn: no, it takes about six months to hear back from the EU after a proposal 18:17:46 <sbs> hellais: agreed 18:18:18 <hellais> I will continue the work on the data pipeline and start writing documentation for it as well as a tor blog post about it 18:18:42 <hellais> the last part of this week I will be in Turin with simone for the nexa board meeting 18:19:03 <hellais> sbs: BTW will anna be there too? 18:19:17 <hellais> because we really need to fix the dates of the hackathon ASAP 18:19:56 <sbs> hellais: yes 18:20:09 <sbs> hellais: Anna will be there 18:20:27 <aagbsn> (gotta roll out shortly) 18:22:29 <hellais> ok excellent 18:22:47 <hellais> anybody else want to share some next steps? 18:27:39 <sbs> as you said, when we meet in Turin, we'll work on libight 18:27:52 <sbs> I'll try to commit some time to review the iOS app, also 18:30:06 <hellais> ok great 18:30:24 <hellais> well if there is nothing more then that's it folks 18:31:08 <hellais> #endmeeting