16:00:00 <anadahz> #startmeeting
16:00:00 <MeetBot> Meeting started Mon May 15 16:00:00 2017 UTC.  The chair is anadahz. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:00 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
16:00:21 <anadahz> hello
16:00:44 <anadahz> Who's around?
16:00:51 * darkk is
16:00:56 * sbs is
16:02:10 * lucastx is
16:02:16 <anadahz> (Agenda pad: https://pad.riseup.net/p/ooni-irc-pad)
16:02:55 <anadahz> in case you would like to add an agenda topic
16:04:43 <anadahz> So I'm going to start.
16:04:53 <anadahz> #topic Best strategy on reporting false positive/negatives reports
16:06:13 <slacktopus> <nuke> is
16:06:44 <anadahz> There are a number of reports from users and developers in different mediums (irc, mailing lists, in person, issues and emails at contact, ...) about false positive/negative in OONI reports.
16:07:35 <anadahz> Many of them are probe specific (ooniprobe, oonirpobe-android, lepidopter) and some are related to pipeline.
16:08:18 * SuperQ is
16:09:23 <anadahz> I guess it will make much sense if we have a repository or a central place where users can log, report and provide feedback in potential false positive/negative measurements.
16:11:41 <willscott> it seems like a lot of the issue is translating from user reports of 'it claims interference but i don't think there is' to what the underlying issue is
16:12:13 <willscott> in the email thread recently, it seemed like the actual issue was with outdated domains that probably should get reported to the test-lists repo
16:12:59 <willscott> but other issues might be due to e.g. hitting a cloudflare page rather than the expected domain, which is arguably something that the pipeline should consider before automatically flagging as interference
16:13:14 <darkk> I think that the entrypoint "right now" is: 1. tagged bugreport to https://github.com/TheTorProject/ooni-probe/ (another repo?) with all the required details, 2. mail to contact@ if user is not comfortable with public bug report.
16:13:58 <willscott> it could be worth a feature request to have a button in the apps for 'flag this conclusion as potentially inaccurate'
16:14:17 <slacktopus> Action: hellais just landed in Rome
16:14:21 <darkk> willscott: ooniprobe does not have blockpages fingerprint DB and users also complain about bad decisions from the ooniprobe itself
16:15:49 <anadahz> It will be nice if we could choose a default strategy of how a user should report anything related with the reports.
16:16:23 <willscott> sure. if there's going to be a repo for users to submit this sort of issue, there ideally is some plan for resolving these things :) maybe that's an over-ride list for sites ooniprobe shouldn't flag just on http-diffs?
16:16:34 <slacktopus> <hellais> I think currently the best thing is either bug report or email as they are already doing
16:17:07 <anadahz> (re: https://lists.torproject.org/pipermail/ooni-dev/2017-May/000517.html)
16:17:49 <slacktopus> <hellais> But it should probably be clear that there is a limit to the extent that A this is useful and B that this is fixable
16:18:04 <slacktopus> <hellais> The long term fix would be provided the pipeline analysis to users
16:18:11 <anadahz> Perhaps we could add this information on ooni's website and direct to a repo for the user to do a bug-report.
16:18:20 <slacktopus> <hellais> But that still has a latency of the analysis of the pipeline
16:19:30 <anadahz> a related issue in ooni-explorer, https://github.com/TheTorProject/ooni-explorer/issues/70
16:20:15 <anadahz> Shall we then use ooni-probe for the bug reports?
16:22:30 <slacktopus> <hellais> Either ooniprobe or oonipipeline
16:23:47 <slacktopus> <hellais> Ideally the user would report it on the component that it's relevant to, so ooniprobe for desktop, ooniprobe-android/iOS when that is the one that makes sense
16:24:06 <slacktopus> <hellais> But I think we can also handle migrating issues to the relevant repo
16:26:41 <anadahz> Sounds like a plan, I'm going to add a label, does 'faulty report' sounds OK for a label?
16:27:21 <darkk> fakenews
16:28:05 <darkk> 'faulty report' sounds OK, but I (personally) prefer 'fakenews' that is more informal :)
16:28:56 <darkk> on the other hand I can't imagine user actually using that tag, so, yeah, sorry for spending 2 minutes of meeting time
16:29:53 <anadahz> Nowadays in many countries/areas people can be sued/sentenced for reporting or not reporting fakenews :)
16:31:25 <slacktopus> <hellais> Users cannot attach labels unless they have write access to repo
16:31:47 <slacktopus> <hellais> I would rather change the issue template to take this sort of report into account
16:34:58 <anadahz> OK shall we proceed with the next topic?
16:37:24 <anadahz> #topic Allow public results from ORG blocked implementations from different countries
16:40:09 <anadahz> There a number of blocked implementations in a number of countries already (https://wp.censorship.exposed/en/) that use an older ooniprobe implementation to test user contributed URLs.
16:41:32 <anadahz> Unfortunately the reports are not being submitted to OONI and have a great coverage per country in UK and Brazil.
16:42:36 <anadahz> I'm not sure about Tunisia and Kenya but in Great Britain and Brazil most mainstream ISPs are covered.
16:43:55 <anadahz> I would like to find out what is neeeded to make the blocked implementations to use the newest ooniprobe version and push the reports to the OONI canonical collectors.
16:45:33 <anadahz> Is the current pipeline ready to accept reports from blocked?
16:46:24 <slacktopus> <hellais> Are they still using http_requests instead of web_connectivity?
16:46:49 <slacktopus> <hellais> Also the issue IRC was that they were creating one report per URL that made the pipeline sad
16:47:09 <slacktopus> <hellais> Also if they are running http_requests we may not care that much to have those measurements
16:50:08 <anadahz> Is one report per URL still an issue for the current pipeline?
16:50:09 <darkk> one report per URL should not make the pipeline sad when luigid pipeline is brought down (== "airflow pipeline is considered stable")
16:52:39 <slacktopus> <hellais> Yeah it depends on definition "current"
16:53:35 <slacktopus> <hellais> But before we proceed with ingesting the data some quality checks should be run on it
16:53:39 <anadahz> They use an old version of ooniprobe so I don't think that there the web_connectivity test exists there.
16:54:45 <anadahz> Also the blocked project has collected some seriours reports since many years ago it will be a pity not to have these reports in explorer.
16:55:03 <anadahz> hellais: Why do you think that the http_requests reports are not at all useful?
16:55:52 <slacktopus> <hellais> Because we have no baseline comparison next to it. I don't think they are completely useless, but for sure less useful than they could be
16:56:47 <anadahz> Let me rephrase it: not at all useful to have them in OONI data repository.
16:58:02 <anadahz> (these reports iclude cases such as: http://ccc.de/en/updates/2014/ccc-censored-in-uk)
16:58:38 <anadahz> and many more older reports when ooniprobe also had no support for the web_connectivity test
17:08:03 <anadahz> Perhaps we can continue the discussion of the blocked reports at a later point.
17:08:34 <anadahz> Anyone has something to add/discuss, otherwise we can conclude this meeting.
17:12:54 <anadahz> Thanks everyone for attending!
17:12:59 <anadahz> #endmeeting