17:00:47 <hellais> #startmeeting
17:00:47 <MeetBot> Meeting started Mon Apr 27 17:00:47 2015 UTC.  The chair is hellais. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:47 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
17:00:55 <hellais> ok so here we go
17:01:34 <hellais> so who is here?
17:01:39 <aagbsn> here
17:01:45 <poly> here
17:02:02 <poly> #help
17:02:35 <anadahz> here
17:02:53 <aagbsn> so for this meeting arturo and I proposed we update and delegate tickets related to our open data hackfest in rome
17:03:06 <aagbsn> I created tickets based on some points from arturo: https://trac.torproject.org/projects/tor/query?status=!closed&keywords=~ooni-odhf
17:03:25 <aagbsn> this is likely not a complete set of tasks
17:04:42 <aagbsn> a few tasks may block on deploying the new ooni website, but I'm not sure what the status/timeline of that task is
17:04:45 <hellais> yes true
17:05:04 <hellais> I think most of those tasks have a predictable amount of time they will require except the pipeline migration stuff
17:05:40 <hellais> but I would say that before we digg into that perhaps it's useful for us all to do a round of what has happenned in the last couple of week?
17:08:35 <hellais> I will interpret such silence as "please do begin with yourself" :P
17:08:52 <aagbsn> I can go. I have not done much OONI related tasks in the past week other than to reach out to some big data contacts and discuss with arturo our plans for the hackfest and data architecture
17:09:01 <hellais> ah ok
17:09:35 <hellais> aagbsn: anything else?
17:10:06 <aagbsn> possibly but nothing off the top of my head
17:10:14 <hellais> on my side what went on from the 14th-today is:
17:10:46 <hellais> * We had a call with some people from MLAB to discuss having libight be a library for all MLAB tests and completed the prototype of the iOS app with nuke
17:11:15 <hellais> * Reviewed the pull requests from vasilis and you on the censorship tool tests and specs
17:11:35 <hellais> * Had to update the bouncer ooni-sysadmin script to work with the production mlab-ns.
17:11:52 <hellais> * Then a lot of work was done on organising this OONI Hackfest
17:14:16 <anadahz> from my side I have finished the lepidopter image script
17:14:32 <hellais> https://twitter.com/OpenObservatory/status/592738610778120192
17:14:43 <hellais> this is what some designers have come up for the banner of it
17:15:16 <anadahz> hellais: oh sorry I thought you 've finished (long pause ;))
17:15:36 <hellais> yeah sorry I thought it would take me less to upload that image
17:15:51 <hellais> anyways we have some possible dates for the hackathon:
17:16:20 <hellais> 60% 5-6%, 40% 19-20, 10% 26-27 June
17:16:27 <hellais> err 5-6
17:16:42 <aagbsn> the other work we did this past week was related to the OTF proposal, reviewing our budget estimates and looping in isabela and tom for feedback. I believe art has been reaching out to otf for early feedback, as well
17:16:45 <hellais> the probability is there because we have so far only confirmed the first of juen and last of june
17:17:03 <aagbsn> hellais: that image rocks!
17:17:05 <hellais> aagbsn: yes true
17:17:08 <aagbsn> love it
17:17:15 <hellais> aagbsn: :)
17:17:29 <hellais> they also made a mockup of the announcement page
17:17:48 <hellais> I think we can manage to launch it by end of week
17:17:51 <aagbsn> (though I dont know why the octopus needs a scuba mask, maybe the water is quite polluted ;) )
17:18:26 <hellais> aagbsn: it's more of a special optic to see censorship better
17:18:27 <aagbsn> it would be quite good to lock in our dates for the hackathon so people can plan around it; it's a bit bleh to block out all of june
17:18:51 <hellais> yeah well when we make the announcement the dates shall be fixed :P
17:18:55 <hellais> so end of week we will know for sure
17:18:57 <aagbsn> but as hellais points out, we need a good estimate for how long these tasks will take
17:19:09 <hellais> I am waiting on the parliament to call me back and tell me what dates are good
17:19:11 <aagbsn> and who can commit to working on them, so we can complete everything before jun
17:19:53 <anadahz> is ADINA15 the official name of the event?
17:20:14 <hellais> anadahz: do you like it?
17:20:23 <hellais> it doesn't have to be, but I like how it sounds
17:20:30 <hellais> it's quite pronounceable in any language
17:20:55 <hellais> ADNA was another contender, but ADNA is actually a thing
17:21:02 <anadahz> is it just a random or implies a meaning as well?
17:21:15 <aagbsn> the elephant is that our data architecture must be available for the event
17:21:15 <hellais> A Dive into Net Aonomalies
17:21:22 <hellais> aagbsn: yes correct
17:21:31 <hellais> let's return to the main points
17:21:32 <aagbsn> and be something that we are going to stick with for a while
17:21:55 <anadahz> hellais: i like it ;)
17:22:02 <hellais> so another passtime of mine in the past week has been to explore the big data subculture
17:22:32 <hellais> it's a pretty intersting field and there are some solutions to our data storage and querying needs
17:22:40 <hellais> much better than mongodb as a matter of fact
17:23:24 <hellais> though I fear this to be a big rabbit hole so I would advise if we first go through the tickets you created
17:23:34 <hellais> and perhaps start assigning some names to the easy tasks
17:23:51 <hellais> #link ooni hackathon tickets: https://trac.torproject.org/projects/tor/query?status=!closed&keywords=~ooni-odhf
17:24:23 <hellais> so this I think is easy: https://trac.torproject.org/projects/tor/ticket/15833
17:24:32 <hellais> as a matter of fact it already is happening here: ooni.io
17:24:49 <hellais> I just have to update my script to push it also to the torproject server
17:25:52 <hellais> this is fun code to write perhaps: https://trac.torproject.org/projects/tor/ticket/15829
17:26:13 <aagbsn> I sort of question whether that task is a good use of time
17:26:40 <aagbsn> can we just ask particpants to send an email to {the list, a point of contact} ?
17:28:44 <hellais> I added the google doc where the announcement page resides: https://trac.torproject.org/projects/tor/ticket/15828#comment:1
17:28:50 <hellais> it needs more work, but the structure is there
17:29:29 <hellais> aagbsn: it's that it requires manual labor to fill in the project descriptions with $number_of_members_in_team
17:29:40 <hellais> to be honest it's a quite trivial thing to do
17:30:01 <hellais> otherwise if somebody volunteers to daily spooling the emails and updating the website we can do it the manual way
17:30:18 <hellais> there probably is some service that already does this
17:30:21 <aagbsn> did we intend on adding all applicants to the page? or all accepted applicants?
17:30:29 <hellais> all applicants
17:30:34 <hellais> we don't do selection of applicants
17:30:41 <hellais> you just have to preregister and then you can show up
17:30:56 <hellais> if you can't afford to come to Rome, there is a stipend program
17:30:59 <aagbsn> oh, I was under the impression we had finite travel stipend funding :)
17:31:02 <hellais> that require you to be approved
17:31:05 <aagbsn> but I see what you mean
17:31:15 <hellais> well the travel stipend is not for general attendance
17:31:19 <aagbsn> so perhaps, if we expect quite a number of attendees, that this is indeed a useful task
17:31:24 <hellais> it's a special thing for people in need of travel support
17:31:38 <hellais> we have a limit of 100 people
17:32:09 <aagbsn> is that a hard limit?
17:32:20 <aagbsn> if we do FCFS we might expect that some people will not make it
17:32:21 <hellais> but still I would expect we get around 100 registrations, probably not all will show up though so we should factor that into the algorithm
17:32:58 <aagbsn> yes +1
17:33:03 <aagbsn> tricky.
17:33:12 <hellais> there are a few missing things there
17:33:13 <aagbsn> (are we an airline now or something?)
17:33:17 <hellais> in the ticket list I mean
17:33:29 <hellais> for example that we neewd to make a list of OONI proposed teams
17:33:37 <hellais> that can work on some project we consider interesting
17:33:58 <hellais> this is crucial because it will set the themes of the sort of projects people will be proposing
17:34:50 <aagbsn> this is a good point; though I think we should be careful to have fairly vague categories as we ideally wont have thought of all the ideas people are proposing :)
17:35:16 <aagbsn> such as, "interactive visualization"
17:35:23 <hellais> I don't plan on having categories
17:35:34 <aagbsn> hm, what is your thought?
17:35:38 <hellais> I think people should be able to propose anything
17:35:50 <hellais> even say, make a painting that is inspired by internet freedom
17:36:05 <aagbsn> agree, though we may want some suggestive ideas
17:36:22 <hellais> well the ideas are to be very concrete and detailed
17:37:17 <hellais> and I think that if we summarise well what is in #13731 we have already a good starting point
17:39:10 <hellais> #15838
17:39:50 <hellais> we also need to come up with the rules for the hackathon and come up with what prizes we want to give out
17:40:28 <hellais> I was also thinking that since we don't know how much bandwidth we will have there it may be good to bring a computer with the OONI data in various formats and people can plug in their own hard drive and get the dump of ooni data
17:40:46 <aagbsn> yes, we should have some local copy of the infrastructure
17:41:06 <aagbsn> so many events i have been to with internet or wifi that falls over...
17:41:08 <hellais> if we want to be super pro we can buy some cheap USB drives (they must have 100s of GB) and prepare them in advance with a copy of the ooni data
17:41:13 <hellais> and we can have people pass them around
17:41:28 <aagbsn> ... ha, lets just put this on a http server on the network :)
17:41:42 <hellais> well if the wifi goe's down it's crap
17:41:48 <aagbsn> we can just do this with a cheap gbit ethernet switch
17:41:53 <hellais> also if everybody is downloading via wifi it's super slow
17:41:59 <hellais> USB 3.0 is very fast
17:42:11 <hellais> much faster than any consumer networking card
17:42:16 <hellais> even GBIT
17:42:30 <aagbsn> most flashdisks are shit or expensive. show me a usb drive that can push > 60MB/s :)
17:42:31 <hellais> (and who even has ethernet these days)
17:43:02 <hellais> aagbsn: I am not talking about flash disks I am talking about hard disk
17:43:06 <hellais> 2.5" USB 3.0
17:43:23 <hellais> the WD 1 TB drive costs about 70 EUR
17:43:30 <aagbsn> ok
17:43:46 <aagbsn> we can do both, I assume
17:44:01 <hellais> we could even do something crazy like emulating a USB drive from a computer
17:44:07 <aagbsn> I should have a spare switch or we can acquire one
17:44:18 <hellais> so you actually read the data off the hard drive of the computer
17:44:27 <aagbsn> we should also find out what kind of network infrastructure is available to us
17:44:43 <aagbsn> will we have wired ethernet uplink, can we deploy our own AP, etc
17:44:47 <hellais> yes, when I have the confirmed dates I will go and scout the venue
17:44:53 <aagbsn> ok right on
17:44:58 <hellais> I haven't actually seen it yet, but people have told me good things of it
17:45:10 <aagbsn> I can probably borrow some ubiquiti networks AP
17:45:14 <aagbsn> I also have a few myself
17:45:24 <aagbsn> (3)
17:45:30 <hellais> http://dati.camera.it/it/hackathon/
17:45:34 <hellais> they did this thing in the past
17:47:26 <hellais> I also have various wifi gear
17:47:36 <hellais> a routerboard with 2 wifi cards
17:47:49 <hellais> 2-3 APs of misc types
17:48:06 <hellais> 1 ubquity picostation
17:48:21 <hellais> or nano don't recall
17:48:22 <aagbsn> i have a few nanostation loco m2
17:48:26 <aagbsn> 2.4 band
17:48:49 <hellais> anyways we shall discuss this once we have an idea of the venue
17:48:57 <aagbsn> but no worries for now, we can update the relevant ticket
17:49:02 <hellais> so these tickets?
17:49:08 <hellais> who wants them :P
17:49:10 <hellais> ?
17:49:35 <aagbsn> (did we lose everyone or is anyone still around :) )?
17:50:49 <anadahz> we could maybe go a day before the event and setup the infrastructure
17:51:01 <aagbsn> there is some serialization to these tasks. I could potentially look at #15829 though I might not be the ideal person as I am not familiar with the new site design yet
17:51:19 <aagbsn> #15831 I can take
17:51:35 <hellais> anadahz: yes, with aagbsn we were discussing even perhaps 2-3 days before
17:51:46 <aagbsn> arturo or another local italian might be best for #15835
17:51:48 <hellais> anadahz: so arriving on tuesday or wednesday
17:52:23 <hellais> aagbsn: yeah I will take care of that. I think had already prepared some it it somewhere
17:52:32 <aagbsn> I can also look at #15838
17:52:41 <anadahz> I can take #15835
17:54:01 <aagbsn> ok I have taken those 2 tickets. what is your username on trac anadahz ?
17:54:16 <anadahz> aagbsn: yes
17:54:21 <hellais> anadahz: I think it's ideal I do #15835 :P
17:54:32 * irl arrives late
17:54:35 <anadahz> hellais: OK
17:54:39 <aagbsn> I tend to agree since I suspect a lot of this travel stuff will be in italian
17:54:46 <hellais> unless you are very familiar with the roman transport system
17:54:47 <hellais> ?
17:54:48 <hellais> :P
17:54:52 <anadahz> so what's about #15834
17:55:04 <aagbsn> so I am not sure what that means, exactly :)
17:55:06 <anadahz> hellais: not really ;)
17:55:21 <hellais> #15834 is actually probably already done
17:55:38 <hellais> we can just use the machine on which ooni-app resides and implement it as part of ooni-app?
17:55:39 <anadahz> aagbsn: 'dynamic aspect'
17:55:54 <hellais> or find a service type solution for it
17:56:46 <aagbsn> I'd like whatever we roll out to be the thing we deploy longer term
17:57:01 <aagbsn> so that is the biggest unknown in terms of time, with regard to our schedule
17:57:21 <aagbsn> or am I misunderstanding what this ticket is about?
17:57:40 <irl> that ticket could do with some details on it :/
17:57:53 <aagbsn> yes, they are just placeholders
17:58:02 <hellais> aagbsn: what is "the biggest unknown in terms of time"?
17:58:18 <aagbsn> we have about a month if we select the earliest dates for the event
17:58:56 <aagbsn> I think that is doable if we pick something simple and extensible
17:59:39 <hellais> well I was thinking that this would be a simple bit of javascript that does a query to some REST API and it either POST a proposal or GETs the existing ones
17:59:54 <hellais> we could easily implement this as part of the ooni-app api
17:59:56 <aagbsn> ah, yes, I was misunderstanding this ticket
18:00:01 <hellais> it's sort of ghetto to put it there
18:00:03 <hellais> but whatever
18:00:15 <hellais> it will take 1 hour to implement and be deployed with 1 command
18:01:05 <aagbsn> so, #15830 is the big item, I think
18:01:34 <hellais> aagbsn: yes that is the biggest one
18:02:00 <aagbsn> which is actually a few subtasks. I think arturo and I can collaborate on slicing that up
18:02:11 <hellais> (I looked at the hugo documentation and it looks like we can do this without even using javascript with it's "Dynamic Content" feature)
18:02:38 <hellais> yes #15830 is quite a task
18:02:45 <hellais> but some progress is being made
18:03:21 <hellais> we also exploring the possibility of getting some big cloud big data provider to sponsor us
18:03:40 <hellais> I was supposed to hear back from one of them today, but so far nothing. I will call them tomorrow at this point.
18:04:17 <hellais> I am currently experimenting with the rackspace cloud and I am putting all the reports into hadoop
18:04:18 <aagbsn> does someone want to own #15828 and #15832 ?
18:04:53 <aagbsn> hellais: did greenhost offer any resources?
18:05:21 <aagbsn> (or OTF?)
18:05:32 <aagbsn> i vaguely recall they were collaborating
18:06:16 <hellais> aagbsn: they don't offer big data solutions
18:06:27 <hellais> at least they don't advertise them and I don't think they have anybody expert in that field
18:06:36 <hellais> we want a solution that is as managed as possible I think
18:06:55 <aagbsn> well, 'big data solutions' is a couple machines in some racks, right ;)?
18:07:04 <hellais> the nice thing of all these cloud type providers is that you can go to a graphical dashboard and click the resize bar to scale
18:07:13 <hellais> and you don't have to edit any code or anything
18:07:34 <aagbsn> i would hope that hadoop or w/e isn't so cumbersome to use that we cant add a new node easily
18:09:05 <hellais> well there is a little bit of things to tweak there
18:09:13 <hellais> and it's much better to have a manager for it
18:09:20 <hellais> there are also free software solutions for that
18:09:32 <hellais> but really, do we want to be doing maintainance on this stuff?
18:09:49 <aagbsn> i have reached out to a few friends who may be able to help us with this task
18:09:59 <aagbsn> who do this sort of thing professionally
18:10:06 <aagbsn> so lets get feedbck from them and see what we think
18:10:11 <hellais> ok
18:10:27 <hellais> well I will continue doing work on this and seeing if I can get the pipeline running in hadoop
18:10:46 <irl> if the maintainence wasn't massive and we had the boxes, i can supply free rack space, electricity and and gig/gig internet if finding a place is a problem
18:10:48 <aagbsn> I will help with this task, I am quite interested in this aspect as well
18:11:25 <aagbsn> it would be ideal if a provider or donor could supply this infrastructure
18:12:57 <hellais> I am against a solution where we have our own machines to maintain
18:13:14 <hellais> especially if that means that I have to do all the sysadmin work to maintain it
18:13:27 <hellais> that is why I think the cloud solution is advantageous
18:13:48 <aagbsn> I am open to that solution so long as the underlying components are open source
18:14:02 <hellais> they arew
18:14:02 <aagbsn> and the cost estimates are reasonable for the resources provided
18:14:06 * irl agrees with aagbsn
18:14:28 <hellais> I don't know about the later
18:14:36 <aagbsn> free is good, but we should be able to migrate if a donor isn't able to continue
18:14:43 <hellais> but that's why I am looking into this sponsorship opportunities
18:14:58 <hellais> well the technologies that would be used are:
18:15:14 <irl> i'd be concerned if one sponsor pulling out would cause the system to vanish
18:15:23 <hellais> hadoop (HDFS), HBase and/or elasticsearch, luigi, all of which are free software
18:16:02 <hellais> irl: well it's important to always have an offline backup and design it so that it's simple to restore from the offline backup
18:16:19 <irl> yep
18:16:32 <aagbsn> should we defer this discussion and proceed with any other items?
18:16:40 <hellais> anyways if you consider the amount of stuff that these cloud platforms automate for you, the cost is really low compared to bare metal
18:16:47 <hellais> aagbsn: yeah sure
18:17:23 <aagbsn> if anyone feels like working on the above tasks, please comment or add yourself to the ticket
18:18:28 <aagbsn> perhaps we can schedule a meeting later this week to collaborate on #15828
18:18:33 <hellais> poly: are you still here?
18:18:48 <aagbsn> (the hackfest announcement text)
18:18:59 <hellais> poly: would you like to say something about what you have been hacking on these days?
18:19:19 <hellais> aagbsn: yes, let's do something ad-hoc perhaps with voice support on the hackathon stuff tomorrow
18:19:35 <aagbsn> tomorrow is ok, wednesday might be better for me
18:20:05 <hellais> aagbsn: tomorrow would be ideal, since we don't have much time
18:20:12 <hellais> or later today even
18:20:23 <hellais> Lunar^, irl: any idea what "ooniprobe 1.2.2-1 MIGRATED to testing" means?
18:20:36 <irl> jessie was just released (yesterday)
18:20:39 <hellais> it's from the "Debian testing watch"
18:20:47 <anadahz> hellais: both days are good for me
18:20:49 <irl> which means that uploads are now migrating from unstable into the new testing distribution
18:20:54 <irl> freeze is over
18:20:57 <hellais> ah nice
18:21:00 <irl> i have questions for you regarding this hellais
18:21:03 <hellais> so we should get that package cracking
18:21:04 <irl> when we get to that
18:21:15 * poly is here
18:21:20 <irl> poly can go first
18:21:28 * anadahz afk
18:21:34 <poly> thanks
18:21:46 <aagbsn> hellais: ok we can chat a bit more tonight and lets plan on collaborating tomorrow, let's say 1600 UTC?
18:22:14 <irl> (i can also be available for discussing hackfest related things if that would be useful)
18:22:22 <poly> I've been looking at implementing a censorship analyzer for tor per https://trac.torproject.org/projects/tor/wiki/doc/OONI/censorshipwiki/TorCensorshipAnalyzer
18:22:27 <hellais> aagbsn: can we do it earlier? Like 14-15 CEST?
18:22:43 <aagbsn> isn't that the same?
18:22:48 <aagbsn> I was thinking 1400 CEST
18:23:21 <hellais> aagbsn: 16:00 UTC is 18:00 CEST.
18:23:21 <poly> I'm a bit new to the OONI thing, but so far I've implemented a parser for the directory authority list
18:23:31 <aagbsn> hellais: oops tz fail. yes.
18:24:02 <hellais> poly: do you have the link to the screenshots? I think that explains what you are doing quite well.
18:24:29 <poly> right here:
18:24:31 <poly> https://trac.torproject.org/projects/tor/attachment/wiki/doc/OONI/censorshipwiki/TorCensorshipAnalyzer/1.png
18:24:36 <poly> https://trac.torproject.org/projects/tor/attachment/wiki/doc/OONI/censorshipwiki/TorCensorshipAnalyzer/2.png
18:24:43 <poly> https://trac.torproject.org/projects/tor/attachment/wiki/doc/OONI/censorshipwiki/TorCensorshipAnalyzer/3.png
18:24:46 <poly> https://trac.torproject.org/projects/tor/attachment/wiki/doc/OONI/censorshipwiki/TorCensorshipAnalyzer/4.png
18:24:55 <hellais> excellent
18:25:06 <poly> (all inkscape mockups)
18:25:29 <irl> poly: awesome!
18:26:12 <poly> currently working on the first part, which is diagnosing the connection to the tor directory authorities
18:27:02 <poly> hellais also mentioned that I could contribute to this (https://github.com/TheTorProject/ooni-probe/blob/master/docs/source/tutorial.rst) while implementing some of my tests since they are quite similiar
18:27:37 <hellais> yes and also see if the writing_tests document makes sense to you and if not how to improve it
18:28:45 <poly> I just wanted to say thanks to everyone (and hellais in particular) for being so nice and helpful towards a noobie
18:29:10 <irl> hellais is indeed lovely
18:29:42 <hellais> poly: :)
18:30:30 <hellais> poly: also if you think it's ok to do so, I would send an email to ooni-dev about that interesting filtering equipment discovery
18:30:42 <hellais> maybe somebody has seen it before
18:31:33 <hellais> also if you could pastebin the ooni report for ooniprobe blocking/http_requests -u http://www.torproject.org/
18:31:45 <anadahz> hellais: poly in which country?
18:31:53 <poly> anadahz: UAE
18:32:19 <poly> hellais: think it's safe to submit to ooni-dev@?
18:32:55 <hellais> poly: that's a very fine question indeed and I don't think I have the definitive answer
18:32:55 <anadahz> poly: did you collect the reports by yourself?
18:33:11 <hellais> I don't know if how aggressive they are towards these sorts of things
18:33:29 <poly> anadahz: yup. I just noticed my ISP changed filtering equipment
18:33:30 <hellais> probably the risk is very low, but I don't really know
18:33:41 <poly> they're using a local made solution instead of the bluecoat stuff
18:34:02 <poly> hellais: I don't think they're agressive about it, no
18:34:13 <poly> but I'll do my research first
18:35:08 <hellais> ok, some lawyers are looking into the legal situation of running ooniprobe in UAE and I can give you their memo when they are done
18:35:31 <poly> do you have my email?
18:35:48 <hellais> I don't think so. Send it to me on jabber.
18:36:14 <hellais> #help
18:36:33 <poly> done
18:36:59 <hellais> cool
18:37:06 <hellais> so it looks like we are quite overtime
18:37:07 <aagbsn> re: https://trac.torproject.org/projects/tor/attachment/wiki/doc/OONI/censorshipwiki/TorCensorshipAnalyzer/3.png I think it would be good to b ea bit more explicit about what is going on at each level of the slider
18:37:25 <hellais> irl: what was your thing?
18:37:47 <irl> so, i was wondering about translations and whether you'd made progress there.
18:38:06 <irl> i was thinking i might do the first set via debian and do updates via external people if that would speed it up.
18:38:09 <hellais> irl: no I didn't manage to do any progress there
18:38:12 <irl> turnaround will be 12 days via debian.
18:38:22 <hellais> ah that is pretty good
18:38:36 <irl> it is. those teams need more credit.
18:38:39 <hellais> well if I am blocking you I can't guarantee that I will manage to tend to it before 12 days
18:38:48 <irl> ok, i'll get that done via debian then.
18:39:01 <irl> the other thing left to do is make sure failed uploads are reuploaded as part of the cron job.
18:39:08 <irl> then i believe it is ready for upload.
18:39:39 <hellais> irl: I think in this first iteration we can even just avoid putting that there
18:40:06 <hellais> I think that logic can be improved a bit on the ooni side and then have it be just 1 command that you run periodically
18:40:17 <hellais> and it handles cleanup and such
18:40:24 <irl> hmm. ok.
18:40:27 <irl> this makes sense.
18:40:29 <irl> oh
18:40:36 <irl> does ooni detect whether or not there's a network connection?
18:40:43 <irl> there was a bug, but i didn't check it was closed.
18:41:06 <irl> or maybe there wasn't a bug, it's not in my history
18:41:18 <hellais> irl: yes, it does and now the oonireport and oonideckgen don't hang when it's missing
18:41:26 <irl> awesome!
18:41:40 <hellais> I confirm that this bug did exist
18:41:46 <aagbsn> btw re: debian packages, are we able to depend on geoip-database-extra (the AS databases are packaged now, thanks to Patrick Matth�i
18:41:46 <irl> ok, so that's basically done then.
18:42:14 <irl> aagbsn: yes, this is something that could be done.
18:42:24 <irl> well, now you've told me it's packaged i have to do that, debian policy.
18:42:36 <hellais> https://github.com/TheTorProject/ooni-probe/commit/9c2802dc85d14e9a4e03b61bbf61abfd0e88c25f
18:42:39 <hellais> here it is
18:43:01 <aagbsn> irl: oh, interesting. it is in wheezy-backports and testing/unstable
18:43:30 <irl> but not in jessie backports because that only came into existence yesterday
18:43:35 <aagbsn> I think that means it is in jessie too?
18:43:39 <aagbsn> < goes to look
18:44:08 <aagbsn> yes!
18:44:15 <aagbsn> https://packages.debian.org/jessie/geoip-database-extra
18:44:44 <irl> awesome
18:44:54 <aagbsn> oh hm, now i am confused because that page says "Packages providing..." "geoip-database-contrib"
18:45:06 <aagbsn> the -contrib package contains the downloader script, I think
18:45:15 <irl> hmm
18:45:20 <irl> is the database non-free?
18:45:24 <aagbsn> https://packages.debian.org/jessie/all/geoip-database-extra/filelist though it appears to be here
18:45:43 <aagbsn> irl: no, I wrote software to take csv formatted data and package it in the libgeoip compatible format
18:45:52 <aagbsn> in the same manner that is done for the country-code databases, more or less
18:46:52 <aagbsn> (I also extended blockfinder to produce compatible csv data from open data sources) https://github.com/ioerror/blockfinder
18:46:53 <irl> it looks like you can install the package or install the downloader to get more frequent updates which also provides the package
18:47:05 <irl> so dependencies work
18:47:23 <irl> although i still haven't got my head round metapackages that are also packages yet
18:47:28 <aagbsn> we currently do something similar via ooniresources, but maxmind doesn't version these things and it is a binary format
18:48:19 <irl> https://sources.debian.net/src/geoip-database-contrib/1.8/debian/control/
18:48:27 <irl> yep, they are conflicting and do the same thing
18:48:29 <hellais> I would for the moment keep the ooniresources download strategy
18:48:43 <hellais> then perhaps in the future we can use geoip-database-extra
18:48:50 <irl> hellais: where does that try to put the downloaded files?
18:48:59 <hellais> but changing something so crucial like this can lead to badness
18:49:34 <hellais> irl: I forget where Lunar^ set it to download it, but it's set inside of ooniprobe.conf and then ooniprobe knows where to get it locally from
18:49:35 <aagbsn> however, ooniresources is suggested to be run as root, which is A Terrible Idea, IMO
18:50:09 <aagbsn> nor does it get updated unless the user chooses to update it
18:50:28 <hellais> well you need to be root to install a package so I don't see how this is a problem
18:50:39 <aagbsn> whereas apt-get update will update the geoip-databases. perhaps we should work with Patrick to ensure that the packages are updated often?
18:50:52 <hellais> aagbsn: it would get updated if a new version of ooniprobe is released, because it would rerun the postinstall script, I think
18:51:01 <aagbsn> well, apt-get at least verifies the packages, and they are signed, etc, and we also provide ooni-probe updates via apt
18:51:23 <irl> it will re-run the postinstall script yes
18:51:24 <aagbsn> I spent a fair bit of time on this as well, so I am motivated to force the change :)
18:51:25 <hellais> aagbsn: we still need to use ooniresources to download the citizenlab test lists
18:51:48 <hellais> this is anyways something that has caused us lot's of pain in the past and I don't think will be solved before the next ooniprobe debian release
18:51:55 <irl> ok, there are bugs here, but none that *need* to be fixed before we can do an upload
18:52:05 <hellais> that is we should make it so what is well tested and we know works well is packaged ASAP
18:52:14 <irl> i could package the citizenlab test lists?
18:52:21 <irl> (in the future)
18:52:38 <aagbsn> that would be ideal, or perhaps citizenlab would like to contribute/help?
18:52:46 <hellais> irl: I  think the future is for having a service that we run that you query to get your inputs
18:53:14 <hellais> there is also the list of open DNS resolvers that is quite large
18:53:18 <aagbsn> hellais: afaik all we have to do is make the debian package depend on geoip-database-extra and drop the postinstall for the geoip data
18:53:57 <hellais> aagbsn: yes in future debian packages we could
18:54:27 <aagbsn> so, is there something I should do to convince you?
18:54:31 <hellais> irl: so are you or Lunar^ doing the packages?
18:54:34 <irl> ok, there are two weeks between now and the translations being finished. i'll look at testing the cron job stuff and porting the packages to ubuntu first.
18:54:56 <irl> if there's time i'll do the geoip stuff but won't include it unless i've tested it to a decent level.
18:55:03 <Lunar^> I don't really understand this story with translatiosn
18:55:05 <Lunar^> translations
18:55:18 <irl> hellais: Lunar^ did a lot of work, which looks good, just needs some tidying up.
18:55:35 <aagbsn> irl: let me know how I can help you test it to a decent level ;)
18:55:36 <irl> Lunar^: it is debian policy that debconf options are translated and you give the translation teams an oppertunity to do this.
18:55:44 <irl> aagbsn: i will do this.
18:56:21 <irl> we were going to use OTF translators, but hellais has not had time to make progress there.
18:56:32 <aagbsn> I could write a simple tool to compare the AS to /24 mappings for the maxmind and debian packaged datasets though they might not be congruent if we use say, the latest maxmind data and an older package
18:56:40 <irl> the reasoning for this is we may change the text as upadates are made from the lawyer people.
18:56:48 <irl> and i didn't want to abuse the translation teams.
18:57:14 <Lunar^> irl: well, usually you upload a first version of the package to unstable
18:57:27 <Lunar^> irl: get native speaker to test the package and have a look at the English phrasing
18:57:35 <Lunar^> irl: then it gets pushed to more translators
18:58:13 <irl> hmm. this wasn't the way i was recommended by debian-i18n.
18:58:20 <irl> i've just done this exact same thing with https://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=xastir;dist=unstable
18:59:10 <irl> lintian complains if you perform uploads without any complete translations
18:59:25 <irl> those packages should not be in unstable
18:59:56 <irl> we might need to grab those lawyers again to interpret debian policy for us
19:01:16 * poly is leaving
19:02:13 <hellais> ok, well if Lunar^ says that we don't need to reach out to the translation team before it's in unstable I think we should do that
19:02:24 <hellais> I mean this means we can push the package out sooner
19:02:42 <hellais> and not be blocking on translation for some text that still needs to be improved
19:02:53 <irl> it would migrate to testing without translations
19:03:00 <hellais> Lunar^, irl: when do you think we can manage to have the latest package in unstable?
19:03:09 <irl> non-english speakers would be presented with a debconf option that would be confusing
19:03:19 <hellais> irl: I don't think that's a problem
19:03:20 <irl> if they do apt-get upgrade
19:03:30 <Lunar^> that's why it's called testing
19:03:37 <hellais> ooniprobe users are advanced users, they probably know the basics of english
19:03:39 <hellais> and if they dont'
19:03:52 <hellais> then it's probably best if they don't run it, since all the docs on OONI are only in english
19:04:02 <hellais> and it would be hard for them to really understand the risks
19:04:14 <irl> ok, if we don't block on translations, i can finish the package off and test by wednesday.
19:04:24 <hellais> that would be epic!
19:04:29 <irl> hand it to Lunar^ who can look over it too, and then upload to unstable.
19:04:43 <irl> (wednesday evening, not morning)
19:04:53 <Lunar^> could work for me
19:05:04 <hellais> so by end of the week we will have ooniprobe 1.3.1 in unstable and backports?
19:05:13 <irl> not in backports.
19:05:21 <hellais> ah ok, just unstbale
19:05:25 <hellais> *unstable
19:05:35 <hellais> what is needed to get it into backports?
19:05:59 <hellais> (I have probably asked this already a million times, excuse my forgetfulness)
19:07:09 <irl> Lunar^: ?
19:07:36 <Lunar^> hellais: you get it to migrate to testing
19:07:41 <Lunar^> hellais: then you can upload to backport
19:08:09 <irl> so upload+5 days
19:08:37 <hellais> ah ok cool
19:08:41 <hellais> sounds good
19:08:51 <hellais> so we are now 1 hour overtime on the meeting
19:09:10 <irl> i mean, it was a productive hour
19:09:15 <hellais> indeed
19:09:20 <hellais> we should do more of these :)
19:09:26 <irl> heheh
19:09:44 <hellais> so who is interested tomorrow at 15:00 we shall do a voice or IRC meeting to discuss hackathon logistics
19:09:55 <irl> i can be available for that
19:10:00 <hellais> I would say who is interested should show up on IRC by 14:00 and we can coordinate for that
19:10:07 <irl> can you specify TZ?
19:10:09 <hellais> I think mumble may be the most likely candidate
19:10:18 <hellais> oh yeah 14:00 CEST
19:10:26 <irl> what is a CEST? :/
19:10:30 <irl> UTC+2?
19:10:33 <hellais> (when I say a time I usually mean it in European time)
19:10:44 <hellais> irl: yes
19:10:48 <irl> ok cool. yep.
19:10:52 <hellais> 1 hour more than your time
19:11:00 <hellais> rome, paris, berlin time
19:11:09 <irl> my office clock is still UTC. i refuse to acknowledge summer time.
19:11:31 <hellais> heh I think we should not use timezones
19:11:39 * irl agrees
19:11:47 <hellais> anyways I have to go an eat
19:11:52 <hellais> good talking to you all
19:11:57 <hellais> see some of you tomorrow
19:12:00 <irl> ok. i should eat too.
19:12:03 <irl> have fun all. (:
19:12:06 <hellais> #endmeeting