17:01:07 <hellais_> #startmeeting
17:01:07 <MeetBot> Meeting started Mon Sep 14 17:01:07 2015 UTC.  The chair is hellais_. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:01:07 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
17:01:36 <hellais> so
17:01:39 <hellais> let's start this thing
17:01:41 <hellais> who is here?
17:01:50 * sbs is here
17:01:59 <juga> hello
17:02:02 * anadahz here
17:04:18 <hellais> brennan and joe are also here in the flesh
17:04:20 <landers> hallo
17:04:23 <hellais> there we go
17:04:30 <hellais> so let's begin with some report backs?
17:04:36 <hellais> who wants to go?
17:05:48 <juga> ok, i'll start
17:05:51 <juga> i looked examples and lantern test, tried psiphon test copying lantern,  modified process.py, created a more generic socksproxy test, created other psiphon test based on the modified process.py and socksproxy test.
17:05:58 <juga> EOF
17:06:07 <hellais> best report back ever
17:06:35 <hellais> I will state my business then
17:07:12 <hellais> I mainly did a lot of fluff around organising the ADINA hackathon including reaching out to various journalists to get them to write a piece or get us some media presence
17:07:24 <hellais> Started figuring out the accomodation and travel stuff
17:07:50 <hellais> Reached out to people that should be coming to it and reminded them of the event and helped them coordinate their coming there
17:08:13 <hellais> I also had a bunch of meeting with various people we are either working with now or may work with in the future
17:09:00 <hellais> on the dev front I started implementing the Tor test #17045, but didn't do much progress on that
17:09:57 <hellais> I also discussed the UX work to be done by brennan with him, but he will tell you more when he joins in some minutes
17:10:00 <hellais> EOF
17:10:10 <hellais> (he is reading from my screen and is fixing his irssi)
17:10:23 <poly> hello
17:10:30 <poly> quick report back:
17:11:02 <poly> sbs merged in the final branch for all the argument parsing features, working on bugfixes now
17:11:08 <poly> EOF
17:11:13 <landers> working on the monitoring with munin + ansible. got it on the roadmap as will-be-done by next friday.
17:11:16 <landers> EOF
17:12:03 <hellais> bnvk: excellent!
17:12:12 <sbs> Worked on improving MeasurementKit (thanks to hellais and andahz for the review), merged network-meter as poly said, some progress on finalizing the adina15 site
17:12:13 <bnvk> aloha
17:12:26 <sbs> EOF
17:12:52 <anadahz> ok so my turn
17:13:44 <anadahz> This week I have worked in bringing up the 0.1 release of the lepidopter RasPi image
17:14:32 <anadahz> I have finally made a final implementation that can reproduce functional images
17:15:05 <anadahz> given the fact that the underlying hardware support any arm* architecture
17:15:48 <hellais> hodgepodge: hey, we were just in the weekly gathering at the report back stage. I you have some pipeline report backing to be done feel free to do so
17:16:01 <hellais> (vasilis though is still typing so wait for his EOF)
17:16:15 <hellais> (vasilis == anadahz)
17:16:36 <anadahz> The lepidopter implementation change to a type of "modular" design: adding a FHS to the image.
17:17:40 <anadahz> Apart from this I tested the release in almost all RasPi platforms and all that have network and were sold to the market.
17:17:47 <hellais> anadahz: FHS == F? Hidden Service?
17:18:18 <anadahz> hellais: Filesystem Hierarchy Standard
17:19:00 <hellais> anadahz: why do we need that?
17:19:34 <anadahz> hellais: there are numerous of scripts, cronjobs, and files that are required
17:19:37 <hellais> ah I see what you mean
17:19:42 <anadahz> and this is the most clean way
17:19:49 <hellais> in the sense that you can take a directory structure with some files and they will be put on the image?
17:19:57 <anadahz> hellais: exactly
17:20:04 <hellais> anadahz: gotcha
17:20:57 <anadahz> I have worked on the help section of the side providing installation instructions for Linux, Mac*, Win* OSes
17:21:27 <anadahz> However I'm still lacking a place on where to host this image.
17:22:51 <anadahz> Apart from the lepidopter things I have reached out to various groups and people for the ADINA event and reviewed some PRs
17:23:19 <anadahz> there are more things that I forget but we should consider this EOF atm.
17:23:34 <hellais> anadahz: when you have it ready and it's on a place where I can scp it from I will put on a torproject server
17:23:49 <anadahz> hellais: it's ready
17:24:15 <hellais> anadahz: epic! Let's discuss this out of band
17:25:11 <bnvk> i'll go next. my name is Brennan. I'm a designer / UX person who does a bit of code things as well
17:26:02 <bnvk> i'm being brought on to help spec out a REST API and a web based data visualization interface for OONI data
17:26:49 <bnvk> we're currently calling this "OONI Explorer" and will be making design interfaces and things that look pretty to eyes and make sense to less technical peoples brains over the next couple months
17:27:16 <bnvk> nice to meet all you. EOF
17:27:39 <hellais> most excellent
17:27:47 <hellais> is there somebody still wanting to report back?
17:29:02 <hellais> I guess not
17:29:05 <hodgepodge> Sure. My name is Tyler, I'm currently working a little bit on security-related stuff, and will be helping make some of the metrics aggregated by OONI more accessible to developers, and non-technical personnel over the course of the next few weeks.
17:29:08 <hellais> oh nice
17:29:13 <hodgepodge> Awh yeah.
17:29:41 <hodgepodge> Over the course of the past week, I've been reviewing the OONI architecture, specifically facets surrounding ooni-pipeline, and ooni-pipeline-ng.
17:30:22 <hodgepodge> I'm currently waiting for additional information on how to connect to the backend services used by oonib, specifically how to work with the EMR cluster used to store reports.
17:31:23 <hodgepodge> If someone could point me in the right direction with regards to how current iterations of the pipeline work at a high-level, it would be greatly appreciated. I've set-up an OONI probe in the U.S. and am in the process of setting up an instance of the ooni-pipeline on a VPS to help myself get more familiar with the codebase.
17:32:04 <hodgepodge> I'll be presenting my findings with a team at Hack The North in Waterloo, Ontario, Canada with @hugg, and hopefully I'll be able to help out with ADINA15 from here.
17:32:41 <hodgepodge> If that is an option, I'd be working on data API improvements, which is strongly related to what I'll be working on at HtN.
17:33:43 <hodgepodge> Apart from that, I have no additional ooni-related updates at this time. If it's useful, the instance of ooni-probe I am running is going to be in an environment running an i2p relay.
17:33:46 <hodgepodge> EOF.
17:34:05 <hellais> good
17:34:38 <hellais> I have some feedback on that, but first does anybody else have some things they would like to talk about or think we should discuss together?
17:38:13 <hellais> well in that case I wanted to comment on a couple of things hodgepodge said that may be of interest to other people too
17:38:53 <hellais> one is that we should probably quite soon explicitly add the ability to tag vantage points that goes beyond just the ASN
17:39:21 <hellais> hodgepodge mentioned that it would be running on a machine that is a i2p relay, that may mean that the probe could receive different treatment and it needs to be taken into account I believe
17:39:40 <hellais> ooni-probe supports a feature called "annotations" that allows you add to a report a key-value type data structure
17:40:03 <hellais> and it was actually created exactly for this reason. To be able to say "this probe is running in a university" or "this probe is at office of company X" etc.
17:40:29 <hellais> but from inspecting the reports nobody really seems to use it all that much, which makes sort of sense since it's not very well documented
17:41:52 <hellais> hodgepodge: also as bnvk pointed out we will be working together on designing the API to the data and given your interest in that part of the OONI stack you may be interested in interacting more with us about that in the near future.
17:42:13 <hellais> hodgepodge: can you remind me what information you are waiting on to connect to the backend services?
17:42:32 <hellais> actually nevermind
17:42:46 <hellais> it was the AWS API key with list cap?
17:44:26 <hodgepodge> While we're still here, and it isn't too late in your time zone, are there any additional steps I need to take to provision an ooni-pipeline environment apart from executing what is in ./bin (i.e. run Ansible, etc.)?
17:44:46 <bnvk> hellais: perhaps it beneficial to spec out ID codes for the annotations feature you mention, as seems like "filtering by University proboes" would be useful in dataviz side of things
17:45:20 <hodgepodge> I wonder if that can be inferred by ASN, since many universities are likely to run an ASN, if I'm not mistaken?
17:46:01 <hellais> hodgepodge: by ooni-pipeline you mean the ng one?
17:46:21 <hodgepodge> The problem with a tagging approach would be that users would have to opt-in to letting us know that they are running a probe at a university. You could provide a mandatory menu similar to what OpenSSL does when you're generating a new cert?
17:46:43 <hellais> bnvk: very good point
17:48:51 <hellais> hodgepodge: regarding the privacy considerations, what you say is very true, however what it comes down to is if there is consent. The user should obviously have to opt-in for this, but if they are aware of the fact that this will end up being public we should not make it hard for them to do so
17:49:37 <hellais> this would probably end up being part of the informed consent procedure we are developping with the harvard law school
17:50:05 <hodgepodge> That's wonderful. Glad to hear that you're working with them on this.
17:50:14 <hellais> hodgepodge: what pipeline are you using?
17:50:18 <hellais> you should be using the -ng one
17:50:25 <hellais> the one without the -ng is abandonware
17:50:35 <hellais> and I should probably rename it to -old or something
17:51:17 <hodgepodge> I haven't started with either pipeline yet, but I'll definitely use ooni-pipeline-ng going forward.
17:53:00 <hodgepodge> What is the best way to access the bridge reachability reports? I can filter through the reports using api.ooni.nu/api/reports, and filter by type? At the hackathon, I am going to be transforming the reports into a relational form with @hugg to allow for more interactive queries to be performed.
17:53:03 <hellais> hodgepodge: ok, I would advise you to not focus too much on the old one and just inspect the code to see what the logic is
17:53:30 <hellais> if you have some questions on some of the logic ask me and I will provide you with an answer as soon as possible
17:53:47 <hellais> but it's probably not worth your trouble to figure out how to deploy the old pipline since it was quite a pain
17:56:26 <hodgepodge> (hopefully this isn't getting off-topic for the dev. discussions)
17:56:53 <hellais> hodgepodge: in theory this should do the trick: http://api.ooni.io/api/reports/list?by=test_name
17:57:00 <hellais> but I believe I just crashed the api for some reason
17:57:40 <hellais> that should however give you in the keys the names of the tests and in the values the IDs and some other metadata about the report
17:58:21 <hellais> hodgepodge: these are very fine questions, but time is tyrann so I would say we further this discussion out of badn
17:58:24 <hellais> *band
17:58:54 <hodgepodge> Awesome, from there I think that I can use S3 to fetch the reports, and cache them locally.
17:59:00 <hellais> hodgepodge: can you also send me a PGP signed email to remind me about the AWS token (I couldn't find an email from you about that in my inbox :( )
17:59:29 <hellais> so let's do a quick and compressed, juga style next steps
17:59:48 <hellais> no need to request to speak, just type in one line, copy-paste - enter!
17:59:48 <hodgepodge> I agree, this would best be reserved for an OoB discussion. I'll send you an e-mail later today, as I don't have access to a machine with GPG right now. You should get it in the morning.
18:00:06 <hodgepodge> API fuzzing. Clearly.
18:00:12 <hodgepodge> ;)
18:01:05 <hodgepodge> Once the next iteration is underway, I can help with that.
18:03:04 <hellais> I plan to work on finishing the Tor test for CCT. Doing more organisation and communication for the ADINA event. Helping out the people that are getting started working with OONI and pointing them in a direction.
18:04:12 <juga> now not juga style:
18:04:17 <juga> Todo:
18:04:17 <juga> * test seems to don't get any data from psiphon from stdout
18:04:17 <juga> * modify process.py in a different way
18:04:17 <juga> * figure out how the ssh process before test finish (right now modifying process.py, it kills the python process, but not the ssh process)
18:04:20 <juga> * solve the hardcoded relative paths in psi_client.py from the test. (Alternative: fork psiphon python client and ask upstream to merge?)
18:04:23 <juga> Further todo:
18:04:25 <juga> * talk with anadahz about the script to install psiphon
18:04:27 <juga> * if ssh already running, http request will success, but using other ssh process
18:04:30 <juga> * check if ssh to all psiphon servers is blocked
18:04:33 <juga> * Look other possible tasks in https://pad.riseup.net/p/s34qp95tNvcv, create tickets
18:05:23 <hodgepodge> We should add an i2p reachability test as well. I might be able to work with some of the developers in the associated community on that.
18:05:51 <hellais> sbs: ?
18:05:52 <hellais> anadahz: ?
18:05:54 <hellais> landers: ?
18:05:55 <hellais> bnvk: ?
18:07:04 <anadahz> finish ADINA preparation tasks
18:07:16 <anadahz> completer lepidopter release
18:07:35 <anadahz> reachout to all the people that would like to use the image
18:07:46 <landers> continue on the munin EOF
18:07:52 <bnvk> Todo:
18:08:41 <bnvk> * Start specing out REST API endpoints and data strutures with hellais
18:09:09 <bnvk> * Start on wireframes of OONI Explorer
18:09:12 <bnvk> EOF
18:09:57 <sbs> I will attempt to finish the login of ADINA as soon as possible and then help with ADINA and possibly do something on the MK front. EOF
18:10:58 <hellais> most excellent
18:11:05 <hellais> thank you all for attending the gathering
18:11:15 <hellais> appologies for some minutes of over time
18:11:35 <hellais> but I think it makes sense given the number of people and importance of some matters
18:11:42 <hellais> have fun!
18:11:44 <hellais> #endmeeting
18:11:56 <hellais> #endmeeting
18:11:59 <hellais> oh noes
18:12:00 <hellais> #help
18:12:14 <hellais> the meetbot is not botting
18:12:25 <hellais> will ping the devs
18:12:31 <sbs> hellais: you need to change your username
18:12:40 <sbs> hellais: you were hellais_ when you started it
18:13:29 <hellais_> #endmeeting