16:01:45 <hellais> #startmeeting OONI gathering 2016-11-28
16:01:45 <MeetBot> Meeting started Mon Nov 28 16:01:45 2016 UTC.  The chair is hellais. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:45 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
16:02:53 <hellais> here we go
16:02:56 <hellais> who is here?
16:03:29 <landers2> here
16:03:31 * sbs is here (both here as here and here as in slack)
16:03:44 * darkk is
16:04:13 <hellais> ok so I have 1 item
16:04:25 <hellais> #topic Discuss what should fit in 2.1.
16:04:31 <hellais> 0
16:05:10 <hellais> so at the of this week I would like to cut a new ooniprobe release 2.1.0 that should include the IM tests together with the tests for testing reachability of tor bridges via tcp_connect
16:05:39 <hellais> if there is something specific that you believe should land inside of 2.1.0 that is not amongst the currently "ready for review" pull requests (that should all be merged by that time)
16:05:47 <hellais> it would be a good idea to bring it up now
16:07:08 <hellais> these are the PR that should make it into 2.1.0: https://github.com/TheTorProject/ooni-probe/pulls?q=is%3Aopen+is%3Apr+label%3A%22ready+for+review%22
16:09:46 <hellais> the question I guess is, are there some issues that you think are so important that a patch for them should be written by 2.1.0 speak now or forever hold your peace
16:10:09 <darkk> I have nothing on my table to add :)
16:11:15 <hellais> ok well if nobody else has anything then we can move on :P
16:11:17 <darkk> I'm going to review these PRs in the morning, then I suggest merge them to master on Tue-Wen and have a week-or-so grace period of "TC" testing on our local PIs and PCs.
16:11:20 <darkk> s/TC/RC/
16:12:02 <darkk> just in case we notice something interesting
16:12:11 <hellais> yes that sounds good
16:12:18 <hellais> thanks
16:12:42 <hellais> ok then I guess we can go to the second item
16:12:49 <hellais> #topic Announce that we have a slack channel
16:13:07 <anadahz> ~.
16:13:11 <hellais> there is a slack channel that bridges everything written in here (and inside the newly created #ooni-entropy)
16:13:18 <hellais> to slack and vice-versa
16:13:52 <darkk> anadahz: what do you mean saying `~.` ? Are you terminating your SSH connection as soon as you hear `Slack`? :)
16:13:59 <hellais> if you are a slack user or prefer to use that for reading OONI related stuff request access here: https://openobservatory.slack.com/
16:14:02 <anadahz> hahahah
16:14:07 <anadahz> not yet :)
16:14:49 <darkk> known bug: uploaded file (pic, schema, screenshot, whatever) does not propagate to IRC
16:15:42 <hellais> see: https://github.com/ekmartin/slack-irc/issues/22
16:15:47 <anadahz> re: IM tests we should re-test them!
16:17:07 <anadahz> hellais: Perhaps it makes more to automate a bit more ooniprobe testing in order to catch any low hanging bugs.
16:18:29 <hellais> anadahz: what sort of testing are you talking about?
16:21:56 <anadahz> hellais: something similar to https://liw.fi/cmdtest/
16:22:33 <landers2> which bugs would we need that for instead of unit testing
16:23:51 <hellais> hum, I am not sure it would be that practical to use something like that for ooniprobe, especially since now it's cli usage is not as relevant
16:24:09 <hellais> I think in general we should have more unittests that test for more things.
16:24:32 <hellais> I would see some value in also having some form of functionality testing that is not covered by the unittests
16:25:08 <hellais> for example currently we don't have anything that is testing the actual nettests, but only the core of ooniprobe is under unittesting
16:25:40 <sbs> hellais: I guess this should be possible using the current test framework, right?
16:25:43 <hellais> something very nice would be to have tests that ensure that the output of a given test (and by output I mean the actual JSON file) meats the expectation
16:26:10 <sbs> hellais: +1
16:26:13 <hellais> though with complex functionality testing you can always go down the rabbit hole of then having to test your tests
16:26:43 <hellais> sbs: yeah I don't think we need some other testing framework for doing functionality testing
16:26:55 <sbs> hellais: perhaps you can at least validate that the output JSON is consistent with a certain expectation (i.e. certain keys exists and their value matches a regex)
16:28:55 <hellais> sbs: yeah that could be an approach.
16:32:11 <nuke> Hi all
16:32:56 <hellais> I guess step 0 for moving this forward would be to actually document what it is that we believe is currently not tested well enough with our current unittests and then discuss there possible ways to test these things
16:38:01 <darkk> do we have anything else to discuss?
16:38:56 <hellais> I don't have anything else
16:39:09 <hellais> does anybody else have something they would like to discuss?
16:40:03 <nuke> yes
16:40:17 <hellais> filed: https://github.com/TheTorProject/ooni-probe/issues/688
16:40:43 <nuke> I released another APK for ooniprobe-android : https://dl.dropboxusercontent.com/u/21497576/ooniprobe-2.apk
16:41:11 <nuke> I invite everyone to test it and send me crash logs or anomalies
16:41:25 <nuke> This version has almost every feature of the iOS version
16:47:18 <hellais> nuke: that's great!
16:47:35 <hellais> maybe we can then also tag a release candidate on github and upload the apk to the tag?
16:48:02 <nuke> yeah definitely, will help the testing process
16:50:28 <hellais> excellent.
16:52:26 <hellais> something that comes to mind is that we should probably also be annotating the reports from android and ios with a specific platform string
16:52:40 <nuke> for iOS is not possible to do the same as the .ipa file is signed together with the array of uuid of ios devices where it could be installed
16:53:07 <hellais> and also with the version of the actual app (as opposed to what we do now where we write the measurement-kit version)
16:53:22 <nuke> hellais: I guess mk already does something like that, ask sbs
16:53:26 <hellais> nuke: is the app configured to report to the testing or the production collector?
16:53:58 <hellais> nuke: currently measurement kit write software_name: "measurement_kit" and "software_version": "$VERSION_OF_MK"
16:54:24 <nuke> no, still on the testing collector
16:55:29 <hellais> what would be ideal is if it said "software_name": "ooniprobe-android" and "software_version": "$VERSION_OF_THE_APP"
16:55:29 <hellais> nuke: ok, that's good
16:57:34 <hellais> is there anything else to discuss?
17:01:55 <hellais> ok in that case, thanks for attending!
17:01:59 <hellais> #endmeeting