14:00:10 <hellais> #startmeeting OONI Community gathering 2017-09-29
14:00:10 <MeetBot> Meeting started Fri Sep 29 14:00:10 2017 UTC.  The chair is hellais. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:10 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:00:32 <slacktopus> <agrabeli> Hello everyone! Welcome to the September OONI community meeting :slightly_smiling_face:
14:00:46 <slacktopus> <agrabeli> I'm Maria from OONI.
14:01:01 <slacktopus> <agrabeli> Please introduce yourselves so that we can know who's here with us today :slightly_smiling_face:
14:01:37 <slacktopus> <hellais> Hello, Arturo from OONI here :slightly_smiling_face:
14:02:45 <slacktopus> <hellais> @channel the OONI Community Gathering is starting ;)
14:03:17 <slacktopus> <agrabeli> (or just say hi)
14:03:24 <darkk> seems, we've got slack blocked in all the interesting jurisdictions :-)
14:03:45 <slacktopus> <hanna> Hi, this is Hanna
14:03:52 <slacktopus> <agrabeli> @hanna hello :slightly_smiling_face:
14:04:37 <slacktopus> <agrabeli> OK, let's get started (and feel free to introduce yourselves asynchronously)
14:04:56 <slacktopus> <agrabeli> Please add topics that you would like to discuss in this gathering here: https://pad.riseup.net/p/ooni-community-meeting
14:05:32 <slacktopus> <agrabeli> First I'd like to mention that OONI made a release this week that hopefully you will find useful: OONI Run: https://run.ooni.io/
14:05:59 <slacktopus> <agrabeli> This is an OONI Probe mobile app feature that allows you to choose which sites you want to test, and share them with your ooniprobe friends to run them
14:06:17 <slacktopus> <agrabeli> It also enables you to add a widget to your site to track its blocking around the world
14:06:25 <slacktopus> <agrabeli> Does anyone have questions related to OONI Run?
14:07:06 <slacktopus> <hanna> Yes, I wanted to ask if sites tested on OONI Run will be incorporated in testing lists
14:07:50 <slacktopus> <hellais> Not automatically, it's something we are considering doing, but we don't have yet very good ideas on how to balance usefulness of newly added URLs, while keeping the lists also compact
14:08:11 <slacktopus> <hellais> FWIW we are considering doing the same thing with measurements that are run from OONI Probe that are user selected
14:08:39 <slacktopus> <hanna> If so, do you think it is worth to add stools for description in addition to URL?
14:08:41 <slacktopus> <hellais> That said all measurements that are run via OONI Run do end up being part of the OONI measurement dataset, so it is possible for us (and anybody)
14:09:02 <slacktopus> <hellais> To "mine" the measurement data to get new URLs
14:09:27 <slacktopus> <hellais> @hanna possibly, that is a good idea. We do have the notes section, which is used for something of this sort
14:10:13 <slacktopus> <hellais> That said I think that if we want to make the test-lists thing something that sollicits more feedback also from a non technical audience, we would have to have some form of web-ish frontend for managing them
14:10:36 <slacktopus> <hellais> As editing csv files via git is not the most user friendly thing
14:11:11 <slacktopus> <hanna> Agree
14:11:57 <slacktopus> <hanna> There are at least two Google forms I know of that serve that purpose, one set by us to collect from Palestine, and one by 7iber in Jordan
14:12:21 <slacktopus> <hellais> Google forms is nice, though I think it's lacking a lot of features to make it useful for this sort of task
14:12:35 <slacktopus> <hellais> For example you can only define single entry answers in google forms
14:12:45 <slacktopus> <hanna> Right
14:12:52 <slacktopus> <hellais> So if a user wanted to submit, say, 10 URL, they would have to submit 10 separate forms
14:13:18 <slacktopus> <hellais> A while back I wrote a very simple PoC for doing this: https://github.com/hellais/tazo
14:13:57 <slacktopus> <hellais> But it was using git as a backend and it's kind of tricky to handle conflict resolution and merging when using git. Maybe dropping the git requirement entirely is a better way to go and just do periodic dumps to git if we want to keep it consumable in that way
14:15:07 <slacktopus> <hellais> I think doing this well actually is more work than you would initially be lead to believe and it hasn't made it's way up our big list of priorities yet
14:16:02 <slacktopus> <agrabeli> But certainly something very important/useful to create at some point
14:16:57 <slacktopus> <agrabeli> Are there more questions related to OONI Run? For those who have already used it, what do you think can be improved?
14:19:10 <slacktopus> <hanna> Thank you
14:19:39 <slacktopus> <agrabeli> @hanna thank _you_ , your suggestions are spot on
14:20:32 <slacktopus> <agrabeli> Further feedback for us for OONI Run can be sent here (or privately to contact@openobservatory.org) any time -- we welcome your feedback!
14:20:58 <slacktopus> <agrabeli> @hellais I believe you have an exciting/exclusive announcement to make for the community members? :slightly_smiling_face:
14:21:31 <slacktopus> <hellais> Yeah, so we are going to be rolling out the new and improved OONI API to measurements today
14:21:43 <slacktopus> <hellais> It's already live, actually, here: https://api.ooni.io
14:22:12 <slacktopus> <hellais> With this you can search and filter through measurements via the `/api/v1/measurements` API endpoints
14:22:47 <slacktopus> <hellais> One of the most useful things, is probably the fact that you can also search specifically for measurements that contain presence of network anomalies
14:23:01 <slacktopus> <hellais> And also see the ones flagged with confirmed cases of censorship (i.e. we saw a blockpage)
14:23:16 <slacktopus> <hellais> Example:
14:24:41 <slacktopus> <hellais> https://api.ooni.io/api/v1/measurements?test_name=web_connectivity&anomaly=true
14:26:33 <slacktopus> <hellais> We are still fixing some last minute bugs, so don't rely on it for evidence until we publish the announcement
14:27:21 <slacktopus> <agrabeli> So in general, the new OONI API can be used by researchers and data scientists to analyze the data
14:27:58 <slacktopus> <agrabeli> It's based on the stable release of our data processing pipeline, which has been re-engineered over the last year, to detect censorship events around the world faster and more accurately
14:28:27 <slacktopus> <agrabeli> The API is dynamically updated every day with new measurements collected by OONI Probe users around the world.
14:29:31 <slacktopus> <agrabeli> Our hope is that a data analyst community will analyze the data and support rapid response to censorship events around the world
14:30:50 <slacktopus> <agrabeli> Any questions?
14:32:17 <darkk> just small addition: it was possible for data analyst community to analyze the data before that, OONI data was open, but the API allows to select subset of "interesting" measurements instead of downloading whole several-terabytes dataset
14:33:23 <slacktopus> <hellais> Precisely
14:34:48 <slacktopus> <hanna> Just a question, why the number of tests on API is different than on the explorer, Bahrain for instance, 249 on the API, 2533 on the explorer
14:35:18 <slacktopus> <hellais> @hanna are you querying the API via `/api/v1` or are you talking about what you see from the web interface?
14:35:35 <slacktopus> <hellais> (as in `/api/v1/measurements`)
14:35:49 <slacktopus> <hanna> There web interface, but I think I mixed measurements with files
14:36:11 <slacktopus> <hellais> Yes, that count there is related to the report _files_, which will contain many _measurements_
14:36:26 <slacktopus> <hellais> What we count on explorer are unique measurements (as those are the ones we expose in the explorer interface)
14:36:51 <slacktopus> <hanna> :+1:
14:37:09 <slacktopus> <hellais> You should get the same count if you do: https://api.ooni.io/api/v1/measurements?probe_cc=BH
14:41:15 <slacktopus> <agrabeli> In other news, we published a new research report in collaboration with ASL19, ARTICLE 19, and Small Media  yesterday, examining internet censorship in Iran: https://ooni.torproject.org/post/iran-internet-censorship/
14:41:48 <slacktopus> <agrabeli> Any feedback/questions you may have for the research report, OONI Run, or the API are welcome
14:41:56 <slacktopus> <hellais> @hanna to make you understand how powerful this can be, with a simple 1 line you can extract all the URLs tested in Bahrain since 2017-09-01 that presented anomalies, like this:
14:42:02 <slacktopus> <hellais> ``` curl 'https://api.ooni.io/api/v1/measurements?probe_cc=BH&anomaly=true&since=2017-09-01' | jq -r '.results[] | [.input,.measurement_url]' ```
14:42:32 <slacktopus> <hanna> Wonderful
14:43:23 <slacktopus> <hellais> We will at some point update the docs with some neat one-liners you can run to get the measurements you need, but input on what sorts of questions you would like to ask the API would be useful
14:47:33 <slacktopus> <hellais> Do we have more items to discuss or talk about?
14:54:49 <slacktopus> <hellais> I guess one thing I would flag, is that we have plans of making desktop apps for OONI Probe supporting Windows and macOS. If you have some things you would like us to be sure to include in them, you should let us know!
14:59:02 <slacktopus> <hellais> @agrabeli think we can wrap today's gathering up?
14:59:11 <slacktopus> <agrabeli> yep
14:59:28 <slacktopus> <agrabeli> Thanks @hanna for joining us today! :slightly_smiling_face:
14:59:44 <slacktopus> <agrabeli> Feel free to ping us here anytime, with any questions/feedback/ideas you may have
15:00:06 <slacktopus> <agrabeli> I'll be circulating the next month's community meeting on mailing lists, per usual
15:00:23 <slacktopus> <agrabeli> if anyone has preferred dates/time slots for the next meeting, please let me know!
15:00:47 <slacktopus> <hanna> :slightly_smiling_face:
15:01:33 <slacktopus> <agrabeli> Thanks all, that's a wrap!
15:01:37 <hellais> #endmeeting