16:59:14 #startmeeting 16:59:14 Meeting started Mon Mar 14 16:59:14 2016 UTC. The chair is hellais. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:59:14 Useful Commands: #action #agreed #help #info #idea #link #topic. 16:59:22 greetings! 16:59:36 so who is here? 16:59:43 hi! 16:59:53 * sbs-mobile is here 17:00:43 hi 17:00:48 hey 17:00:52 hey all 17:01:56 excellent, so what have you been up to last week? 17:02:16 hi 17:03:06 * karsten hasn't done anything ooni-related but would like to listen in what people here are doing, because ooni and metrics are sharing a vegas team. if that's okay for everyone here. 17:03:45 alright, I'll go for it 17:04:36 so for a while I've been working with sbs on Network Meter, which is was supposed to be a general GUI for network testing suites 17:05:23 the idea was to have such a tool that possessing it wouldn't be incriminating (it would integrate with ooni-probe, but other diagnostic tests too) 17:05:58 however, last week we have noticed an issue, which I'd like to ask for your guidance on 17:06:14 What I did: * Sketched out highlights page and pulled some relevant highlights with agrabeli * Added heuristics for detecting blockpages in India * Implemented a simple vanilla Tor connectivity test and made some fixes to the lantern and psiphon tests * Reviewed various pull requests on ooni-explorer * Updated the ooni-backend and ooni-probe JSON data formats based on the code review done by sbs-mobile and a 17:06:21 dded to the data pipeline a trivial statement to not normalise data in the new data format 17:07:12 poly: what issue? 17:07:28 it appears that because we used the Electron framework, the application's size is huge. After a bit of searching, we found that the minimum size for a windows installer would be 60 MB or so for the GUI alone 17:08:07 It appears that the developers of the Electron framework have no intention on fixing this issue anytime soon (https://github.com/atom/electron/issues/2003) 17:08:30 (the reason Electron is so large is that it incorporates major parts of Chrome) 17:09:21 and the worry is that this will hinder distribution? 17:09:38 This isn't acceptable really, particularly because some of OONI's users are in countries with little bandwidth 17:10:13 hellais also mentioned the possibility of refocusing the project, in particular as an interface for OONI only 17:10:26 for reference, network meter can be found here: https://github.com/measurement-kit/network-meter 17:12:37 my understanding is that electron apps aren't too far off from being packaged with cordova as mobile GUIs 17:13:03 willscott: what is cordova? 17:13:41 takes a html / JS code base and packages it into a native android or ios app 17:13:56 (here) 17:14:25 AFAIK, Cordova is for mobile phones only and is a bit different than Electron 17:15:23 namely, on mobile phones you don't need to ship a chromeless instance of Chromium, like Electron- which is the bulk of that 60 MB 17:16:30 and conversely, you can't run Electron apps on smartphones 17:16:49 but they are basically doing the same thing 17:16:59 still users on desktop which was the main target for network-meter need to download electron 17:18:16 not sure if there's a great other option - i guess the app could start a local web server and trigger an installed browser, but that seems like a wonky ui 17:19:09 hellais suggested the possibility of a re-implementation with something like QT or GTK 17:19:44 the advantage there is of course a much smaller file size 17:19:59 what about embedding a browser in a window using Qt? how much space does that take? 17:20:19 in various experiments I've done in the past using Webkit Gtk or Webkit Qt, you still end up with a larger package to download due to shipping the webkit binary in the packaging- which is about 10 - 20 MB 17:20:50 bnvk: is that 10-20 MB in addition to the core qt libs that are ~15 MB? 17:21:48 I meant 10-20MB smaller than the 60 MB size of an Electron build 17:22:05 I am thinking that there are two main paths: * Go for a native cross platform solution such as QT or GTK and implement it in that * Go for the approach suggested by willscott of launching the browser. In thinking about this there is perhaps also a middle ground approach where QT is used only for implementing a task bar type GUI that has small amounts of information in it and mainly handles starting and stopp 17:22:12 ing of the ooni daemon, but for the full UI it starts the browser 17:22:26 this may be a dumb question, but are there existing network meter tools that you could extend via some plugin mechanism? 17:22:59 (oh. to be honest in that case, I don't see the use case where the user can't download a 50 MB app but not a 60MB app. thanks for the info bnvk) 17:23:02 maybe somebody else has already solved this issue. 17:23:12 yah, just using the default browser would get things nice and small- but is a less than ideal user experience 17:23:48 sounds like an area to keep exploring 17:23:51 we did that exact Qt thing with Mailpile and had an icon on the desktop (to the run the app) and taskbar icon and such 17:24:27 user testing showed people got confused that there was two windows (terminal) and a browser in order to use the app 17:24:52 but there's the trade off if you want the interface in web tech: file size vs. UX 17:25:50 I think the idea behind NM was to make it as easy as possible for beginner users to run ooni-probe, that the gui would provide them with nice visualization so that they'd understand what ooni was doing 17:26:18 do you think a hybrid browser/qt implementation would be to confusing for those users? or just fine? 17:26:20 how much is more complex to develop also charts etc in Qt compared to using web or electron? 17:26:44 sbs-mobile: I looked that up, it seems there are libraries for that 17:27:08 it really depends on who your users are 17:27:25 Didn't seem too difficult, but it wouldn't be "plug and play" like network-meter was 17:27:56 in other words, if we go down the pure Qt route, we loose extensibility 17:28:41 (background: one idea behind NM was to make it extendable so that other developers would create an interface for their tools through it, increasing usage and making it more deniable to use) 17:29:50 hellais: is there anyone else working on the earlier-mentioned ooni gui at the moment? 17:29:57 what is the size below which network-meter would be acceptables? 10 MiB? 17:31:09 sbs: are we counting a copy of ooni-probe bundled with it? 17:31:34 poly: currently there is nobody actively working on it, but I have intention to start working on this part once I have done a bit of refactoring of ooni-probe. My goal is to get the daemon part working again and clean up the director/nettest API to make it easier to programatically run measurements. 17:32:08 poly: yes 17:32:32 this article has some interesting insight into how github has managed doing their github desktop client with re-usable HTML/JS components: http://githubengineering.com/cross-platform-ui-in-github-desktop/ 17:33:02 basically they use a WebView on OSX and this thing for windows: https://github.com/cefsharp/CefSharp 17:33:51 though it seems like Cef is going to add another 40 MB 17:34:05 unless you rip out all the stuff that is not needed from it 17:34:47 what is the downside of using windows native browser, however, is something they do not say 17:34:48 what fraction of users will be windows users? 80%? more? 17:35:35 karsten: the target audience is average people with no particular technical understanding of ooni, just an interest in running censorship tests/network measurements 17:35:49 so I would say 80-90% 17:36:06 karsten: yeah that estimate seems accurate 17:36:15 would it be okay to start with something that's windows-only? 17:36:29 I have no objections 17:36:54 (another thing to consider: automatic updates? manual updates? my understanding was that electron helped in this area which is not exactly super simple to get right... so let's also remember about this piece of requirement) 17:36:59 (not that I have any suggestions what to use, but maybe this gets easier if you ignore platform independence for a moment.) 17:37:02 (except that I develop on linux, but could always fire up a VM :P) 17:37:44 sbs, to be honest I feel there are a number of design decisions that we need to think through and write out, that is one of them 17:38:29 one of the big decisions is whether we are going to tailor this to integrate with ooni-probe, or we want to keep going the "network test framework" route 17:39:40 I like the idea of the framework but given limited resources starting with ooni probe only could probably be more wise 17:40:12 hellias: is there any particular language that would work well with the nettest API? 17:40:23 I'd check how complex is to use the .NET's web browser for our use case 17:40:52 yes I agree with sbs, also having a concrete use case will probably also help us understand how to potentially abstract it more in the future 17:41:28 sbs, I haven't worked with .Net before, would the integrated browser be based on IE? 17:41:39 I think it could make sense to have most of the GUI logic in js/css/html so we can, e.g., also use it for lepidopter or serve it directly from ooni-probe (even though this is not good from the point of view of UX) 17:42:21 poly: my limited understanding is that it will reuse libraries and functionalities already available on modern Windows 17:42:53 poly: if i understand correctly you have done some considerable progress on the development how far are you for implementing it? 17:44:02 maybe it makes more sense to have the fully-fledged version big file size and a lightweight one for people with limited bandwidth 17:44:10 network meter currently allows you to install various new tools ("plugins"), displays an interface to run new tests (that's configured by the plugin developer), can run a bunch of tests in parellel 17:44:37 what it can't do: read the results of the tests from disk (stdout only), parse and visualize the results 17:45:33 I will go with my updates for the week: I sucked on what I supposed to achieve this last week, I made some progress, but not enough to close an issue! 17:45:52 so, hellais I assume you still want to stick to schedule of release tomorrow, correct? 17:45:55 poly: this seems promising also: http://doc.qt.io/qt-4.8/qwebview.html 17:46:42 hellais: bnvk says that in his experience this doesn't offer a significant file size reduction compared to Electron 17:47:29 anadahz: the issue is I cannot think of an easy way to provide a lightweight version, as the current version relies on Electron's 60 MB download 17:47:49 bnvk: I would ideally like to have everything ready by tomorrow, but since I am going to be in Berlin on this Friday I figured we could do the release in person on the 18th so we can do a bit of final polishing and push the big red button together. 17:48:02 that would be much more ideal for me 17:48:20 hellais poly: maybe wxWidgets (who has a tradition of wrapping the platform rather than providing uniform implementation of things) could help? http://docs.wxwidgets.org/trunk/group__group__class__webview.html 17:48:35 anadahz: the issue isn't with the features we implemented, our html/js code is likely under 1 MB in size, it's with the framework we used 17:49:40 sbs: I don't have experience with wxWidgets, but this seems promising 17:50:26 other status reports before we run out of time? 17:50:28 Resuming reports back: * Did some admin tasks on a bunch of servers * Provisioned a virtualization server and network on a trusted location * Setup a public web server hosting daily updated sanitized ooni reports (waiting for the firewall management settings to take effect -human intervention- temporalily available here: http://measurements.ooni.io:8080/) * Re organized OONI's trac; there were a 17:50:29 bunch of implemented, obsolete and unanswered tickets 17:50:38 EOF 17:50:41 hellais: I shall plan to work / hang a bit with you on Friday then 17:52:03 my status report: have been merging and reviewing PRs for MK 0.2; taking care of OONI's PRs for the yaml -> json transition; did call with Nuke and hellais on the roadmap for ooni-mobile; discussed the future of network-meter with poly -- EOF 17:52:50 poly: I just tried building an app with qwebview on OSX and the built .app is 11MB (including twisted and python) 17:53:20 hellais, that's good :) thanks 17:53:25 poly: not sure if the size will change on windows though 17:53:40 I'll look it up 17:54:46 karsten: can you share more info wrt vegas team? 17:54:58 ah, sure! 17:55:25 so, tor started organizing into vegas teams last june or so. 17:55:46 network team, applications team, metrics team, community team, soon a UX team. 17:56:13 and ooni was assigned to the metrics team in that vegas structure. 17:56:36 though it turns out we're not working as much together, as in shared meetings etc. 17:56:51 which I think is fine, because we're both working well in our teams. 17:57:24 but one idea of vegas teams was to have some channel to get feedback from people to the tor org. 17:58:12 we recently (last friday) resumed having meetings of vegas team people, and we thought it would be important not to miss any feedback/complaints/problems from ooni. 17:58:48 that's why I'm joining this meeting today, and I'd like to join future ooni meetings as well, to learn if there's anything you want tor org or the other vegas teams to know. 17:59:05 and if you want to join metrics team meetings, you're welcome to do that, too. 17:59:46 does that make (some) sense? 18:00:06 ok, gotta sign off now, hellais i'll check in with you tomorrow on various progresses on my side 18:00:06 karsten: yes! when do metrics team meet? (and on which irc channel?) 18:00:24 every other thursday at 14:00 utc in #tor-dev. 18:00:31 let me give you the link to the team page.. 18:00:39 karsten: thx for the update! started losely to follow up metrics-team mailing list. 18:00:48 https://trac.torproject.org/projects/tor/wiki/org/teams/MetricsTeam 18:01:18 oh, and from a technical POV, hellais and I discussed combining efforts on visualizing measurement results. 18:01:32 and we discussed adding tor-related ooni measurement results to metrics. 18:01:50 but I think we're past the 60 minutes, so happy to explore more next week. 18:02:55 so, thanks for letting me listen here. it was interesting to see how much more you go into the details of discussing a problem than we do in metrics team meetings. 18:03:29 karsten: ah, speaking of stuff related to metrics people this is the ticket for exporting the data we discussed about in VLC: https://github.com/TheTorProject/ooni-pipeline/issues/13 18:03:52 hellais: great! 18:04:15 ok folks I guess we are more or less done here. Let's just close with the next steps 18:04:17 from a quick read, sounds about right! 18:04:56 karsten: thanks for the link! 18:05:16 sbs: sure! 18:05:49 I plan to finalise the work needed to be done on the ooni-explorer, work on doing some more extensive testing of the JSON/data format 0.2 ooni-probe, ooni-backend, ooni-pipeline branches and then begin working on a big refactor of ooni-probe to pave the way for having this splendid GUI implemented 18:07:12 I will test out qtwebview on a Windows system, and if everything works out alright, I'll create a document with the plans for future NM and how we will port it to qt's webkit implementation. sbs is welcome to help me, if he wants to :) 18:08:33 thanks to everyone for the help, and sorry for taking up most of the meeting on one point 18:08:38 * karsten needs to run. will try to make it to this meeting next week. bye, everyone! 18:09:23 Will do MK related development aimed at closing opened pull requests, will be pleased to help poly, will finish testing yaml->json PR of ooni - EOF 18:11:02 From my side: continue working: on the explorer release, sysadmin work on servers and probes, organizing tickets, documentation, ooni packages for many OSes 18:12:10 btw there is a gentoo package request: https://bugs.gentoo.org/show_bug.cgi?id=577210 18:12:23 Ah I forgot to mention one other things that I would like to have done ideally by the end of this week, finalising the job posting for the core OONI developer and submitting it for review by tor people 18:13:08 anadahz: do you expect explorer release this week? 18:14:09 willscott: i thought this was the plan according to hellais 18:14:24 great. just checking :) 18:15:05 willscott: :) 18:16:10 we should find a way to include ooniprobe packages in Debian Jessie backports: https://lists.debian.org/debian-backports/2015/10/msg00030.html 18:16:23 irl: ^ 18:17:35 ok folks that's about the time we had for this. Thanks for attending, until next time. ciao. 18:17:36 ooniprobe has a big user base from debian and currently there are only outdated packages in sid and stretch repositories 18:17:39 #endmeeting