17:13:52 <meejah> #startmeeting
17:13:52 <MeetBot> Meeting started Wed Jun  4 17:13:52 2014 UTC.  The chair is meejah. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:13:52 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
17:14:11 <asn> nickm: i would also like to document the magic times in entry_is_time_to_retry(). do you know if these are spec'ed out, or are they implementation details?
17:14:13 <meejah> lucyd: okay, cool.
17:14:34 <luketheduke> I have been away for a month, so I'd like to find out what the current status of the tor-weather rewrite is.
17:14:57 <meejah> luketheduke: okay. 3. overall status
17:15:02 <nickm> asn: if they are documented anywhere, I would expect that it would be in path-spec.  But they might not be
17:15:08 <asn> nickm: thanks
17:15:17 <luketheduke> (if there is an internet page that is accurate and current I would also be very happy with a link to that)
17:16:27 <meejah> okay, lets start with 1. if anyone has more agenda, just speak up ;)
17:16:35 <luketheduke> https://trac.torproject.org/projects/tor/ticket/12085
17:16:38 <luketheduke> ಠ_ಠ
17:16:43 <meejah> #topic 1. updates from lucyd
17:16:49 <lucyd> meejah: should we consider adding a 'Progress' section in the weather-rewrite wiki?
17:17:15 <meejah> #agreed update wiki page with progress
17:18:23 <meejah> lucyd: okay, what did you make progress on this week?
17:19:03 <lucyd> okay. so I picked up some stuff from OnionPy and developed an initial wrapper
17:19:31 <lucyd> checkout the comment on the ticket for info regarding usage
17:19:46 <lucyd> as pointed out by meejah, it had some possible improvements
17:19:52 <lucyd> I made some of them
17:20:12 <meejah> #action i will post my review-email to trac
17:20:14 <lucyd> e.g., restructuring the OnionooRequest object
17:20:35 <baumanno> meejah: i'm here
17:20:40 <lucyd> adding some more docstrings etc
17:20:41 <baumanno> sorry -.-
17:22:24 <luketheduke> now that I've seen that I'd like to request adding "let's talk about OnionPy" to the topic list.
17:22:54 <lucyd> also made suggested style-related modifications like converting all objects to the new-style objects etc
17:23:04 <asn> nickm: i'm also intrigued by the fact that almost all calls to entry_is_live() have need_capacity toggled to true.
17:23:19 <asn> nickm: as an example, the call in entry_guard_register_connect_status()
17:23:35 <meejah> luketheduke: 4. onionpy
17:23:48 <asn> nickm: which toggles all the previous entry guards to 'up', if we just made contact with a new guard.
17:24:12 <asn> nickm: i'm not sure why the previous entry guards _must_ have need_capacity.
17:24:17 <luketheduke> meejah: (:
17:24:31 <feverDream> hello weether-rewrite team, did I miss the meeting?
17:24:40 <baumanno> should we move on to 3 (or 2, but that's clarified, right?)
17:24:41 <meejah> feverDream: it's happening now
17:24:46 <asn> nickm: because I'm checking out the calls to choose_random_entry_impl(). and some of them don't require need_capacity. so it's possible that some guards are not fast?
17:24:54 <meejah> sure, 3
17:24:55 <baumanno> hola feverDream :)
17:25:11 <meejah> you guys can put #info in front of things to, i think, have meetbot put them inminutes...
17:25:18 <feverDream> hola baumanno, long time :)
17:25:36 <meejah> so 3, "the wiki page" is this one:
17:25:41 <meejah> https://trac.torproject.org/projects/tor/wiki/doc/weather-in-2014
17:25:52 <meejah> does need a few updates
17:26:11 <baumanno> #topic 3. overall status
17:26:28 <nickm> asn: I think that perhaps there is no reason for that.  As least, not any reason that's immediately obvious.
17:26:36 <asn> ok. will note it down.
17:27:00 <meejah> yeah, so there's info about the repos in there, and in my "next" we have some vagrant stuff merged, some unit-tests merged from jondbaker so far
17:27:24 <meejah> lucyd: where are you at in your overall plan? anything to add?
17:27:35 <lucyd> I'm almost done with the onionoo document-attributes issue meejah mentioned and once I write some unit tests, the wrapper is set to go
17:28:16 <lucyd> I'll also try to add a couple of utility methods that use the wrapper to perform weather backend functionality
17:29:03 <lucyd> I'm thinking this will be done by june 6 and I can put it up on my report
17:29:17 <meejah> #info there's also lucyd' proposal: https://sites.google.com/site/sreenathadev/gsoc-2014-weather-rewrite with detailed plan
17:29:38 <baumanno> lucyd: what are you thinking of? what comes to my mind is basic checks, like is_stable and such
17:29:53 <meejah> so then you plan to use these util methods and incorporate the tshirt etc scripts using them?
17:30:07 <Phoul> nickm: was just going to look at the blog post, but noticed the email never got sent. I dont have scrollback to yesterday, is there any reason for this?
17:30:34 <lucyd> baumanno : right. I was planning to start off with your email-parser...since most of the code is already up at your github repo, I'll take a look at it
17:30:47 <lucyd> meejah : precisely
17:31:04 <meejah> okay
17:31:18 <meejah> i guess this feeds into 4. then, maybe? lucyd: sorry i missed last meeting, can you remind me why karsten and you decided to modify onionpy stuff "in" tor-weather?
17:31:27 <meejah> #topic 4. onionpy
17:32:54 <nickm> Phoul: I'm waiting for the message to get onto the list.  First, I need permission to post.  Trying to unravel whom it is who can give me permission to post.
17:32:59 <nickm> *who
17:33:07 <baumanno> i don't have meejah's mail at hand at moment, but fiddling with the wrapper, i'm getting weird responses
17:33:20 <baumanno> though this might be the documents-attributes issue mentioned earlier
17:33:28 <luketheduke> What I'd like to say is that I wrote Onion-Py with the new tor-weather in mind as the primary consumer.
17:33:37 <lucyd> we figured it would come in handy for all the backend scripts
17:34:05 <baumanno> e.g. "for relay in doc.relays: print relay.fingerprint" returns "None"
17:34:05 <meejah> yeah, i guess i'm asking: why isn't tor-weather just depending on onion-py?
17:34:07 <luketheduke> I also have gamambel as a potential "customer" now since he wanted some scripts to keep track of torserver things.
17:34:08 <baumanno> is this known?
17:34:59 <meejah> was karsten worried about deployment or something?
17:35:09 <meejah> baumanno: not by me ;)
17:35:46 <luketheduke> But I'd be very happy to make changes to onion-py as they are required by tor-weather, or to merge back whatever you do (within reason) if you fork onion-py for tor-weather development.
17:36:13 <luketheduke> As for deployment, whenever I have time, I will figure out how to get onion-py into PyPi, which shouldn't be too hard since everyone does it (:
17:36:18 <lucyd> I think we decided that we wanted something specific for handling the weather-rewrite issue
17:36:30 <feverDream> If I remember correctly, I think the plan was to make it more generic later
17:36:43 <lucyd> onion-py was a bit generic and hadn't been maintained for a while
17:37:08 <meejah> luketheduke: i think karsten greatly prefers debian-packaged things :/
17:37:29 <meejah> but yeah, it'd need to at least be on pypi for proper deployment
17:38:01 <meejah> lucyd: can we concentrate on testing/deving the utility methods you need in the immediate future?
17:38:03 <luketheduke> imho there are no debian-packaged things that are appropriate for development and deployment of a modern, sane python product.
17:38:51 <luketheduke> I seem to remember karsten wanted python 2?
17:39:02 <meejah> and ideally get the "onionoo query/caching layer" simply using onion-py? especially since the author is here and keen to merge things? ;)
17:39:03 <luketheduke> I don't know if that was related to tor-weather.
17:39:30 <lucyd> meejah: okay...we'll have to take a look at weather's functionality and estimate some helpful utility methods
17:39:30 <meejah> not sure.
17:39:51 <meejah> #action talk to karsten and figure out weather deployment
17:40:19 <meejah> lucyd: also working on incorporating the scripts will help guide that
17:40:45 <luketheduke> some "utility things" are in https://github.com/duk3luk3/onion-py/blob/master/bin/onion.py right now
17:40:46 <meejah> as in, should guide what methods you need as you try to eliminate duplicate code
17:41:20 <luketheduke> But those are just meant to be code samples showing how to use onion-py atm
17:41:27 <lucyd> ok
17:41:57 <meejah> lucyd: do you have a good idea how you're going to lay things out as you bring in the tshirt, etc things?
17:43:06 <lucyd> well, once the wrapper is done, the tshirt script for example, would have to be rewritten accounting for this change
17:43:09 <Phoul> nickm: ahhh, ok.
17:43:24 <lucyd> similar is the case for other scripts
17:43:30 <Phoul> nickm: If you think about it, could you ping me when it gets posted?
17:43:43 <meejah> luketheduke: okay, that looks good. i don't want to de-rail lucyd too much on his plan, but there's no need to have two python onionoo abstractions...
17:43:45 <lucyd> once the core functionality is implemented, we can work on the django controllers and views
17:44:10 <luketheduke> is the tor-weather rewrite currently intended to work in python 2?
17:44:21 <meejah> lucyd: okay, and you'll arrange for the scripts to run at the right time from within django?
17:44:31 <meejah> or something else?
17:44:36 <nickm> Phoul: okay.  If you haven't heard from me by tomorrow or so, feel free to ping me back again too. :)
17:44:44 <meejah> luketheduke: as far as i understand, yes
17:45:00 <meejah> i do want to discuss with karsten what his deployment needs are, though
17:46:02 <lucyd> meejah : most scripts will be run periodically using crontab
17:46:08 <Phoul> nickm: Sounds good :)
17:46:13 <luketheduke> yeah, I think that's what karsten wanted.
17:46:17 <luketheduke> hmm.
17:46:39 <luketheduke> I don't remember what I said back when I talked about this with karsten.
17:46:42 <lucyd> meejah : and then the documents fetched and parsed by the scripts should be stored in the database
17:47:05 <meejah> lucyd: okay
17:47:48 <karsten> hi. didn't read backlog, but what questions are there about deployment?
17:48:08 <meejah> karsten: hi. luketheduke wondering about python2 (vs 3)
17:48:21 <karsten> what's in wheezy + wheezy-backports?
17:48:22 <luketheduke> hi karsten.
17:48:25 <meejah> ...and why we imported onionpy instead of depending...
17:48:25 <karsten> hi
17:48:33 <meejah> both
17:48:47 <meejah> i'm 99% sure, sec
17:49:13 <karsten> importing seemed easier because of the needs-a-package-in-debian requirement.
17:49:23 <luketheduke> there is no python3 support to speak of in wheezy and wheezy-backports.
17:49:31 <karsten> if there's an easy way to depend on it that fulfills our deployment needs, let's do that.
17:49:59 <meejah> packages.debian.org says it's in there. don't know about libs.
17:50:03 <karsten> though I'm not sure how far lucyd is with rewriting the concept of onionpy. (again, didn't read backlog yet.)
17:50:32 <feverDream> hello karsten
17:50:38 <karsten> hi feverDream
17:50:38 <meejah> it's not *that* different from upstream's repo.  except the pep8-spacing stuff i guess...
17:50:40 <luketheduke> the preferred way to deploy a python3 product on debian is with setuptools.
17:50:59 <meejah> luketheduke: i'd say that's prefered for python2 as well ;)
17:51:12 <luketheduke> karsten disagrees, last I heard.
17:51:24 <meejah> ...however, that doesn't deal with updates as nicely as "apt-get update; apt-get upgrade"
17:51:31 <luketheduke> indeed.
17:51:52 <karsten> I'm just relaying what our sysadmins say, and it doesn't seem crazy what they want.
17:52:21 <karsten> maybe we should involve them in this discussion?
17:52:35 <luketheduke> well, tor-weather is not going to be in wheezy or wheezy-backports, but it could well go on pypi.
17:52:45 <meejah> karsten: i guess the question is: is it possible to depend on onion-py via pip/pypi?
17:53:23 <meejah> you're going to be doing "Something" to upgrade tor-weather, right? if that included "upgrade deps from pypi", is that sufficient?
17:53:33 <karsten> I'd feel more comfortable asking our sysadmins rather than making a statement here about things I don't know enough about.
17:53:44 <meejah> sure, okay. we should take this offline
17:53:48 <luketheduke> (you could also host tor-weather distribution somewhere else in a git repo, setuptools can pull distributions from git urls)
17:54:24 <meejah> true. and there's "peep" to verify SHAs of dependencies
17:56:05 <meejah> so, that's action-item for karsten and I
17:56:22 <luketheduke> I have to say I can't really make a strong argument towards "use setuptools for everything" since the documentation that I've found is so terrible I don't actually know what setuptools can or can not do except for the things I've tried myself :/
17:57:08 <meejah> for lucyd this week, finish what you're doing, and move on to bringing in the tshirt-etc scripts as soon as possible, to start guiding the util methods you need
17:57:13 <luketheduke> are you proficient with setuptools meejah?
17:57:19 <meejah> somewhat, yes
17:57:24 <luketheduke> excellent
17:57:32 <meejah> most of that knowledge is burned into txtorcon's setup.py ;)
17:57:48 <luketheduke> I will be consulting you on improving onion-py's setup.py
17:57:57 <meejah> luketheduke: it comes with it's own pluses and minuses
17:58:00 <lucyd> meejah : gotcha
17:58:01 <meejah> okay, cool
17:58:12 <luketheduke> meejah: of course, like everything (:
17:58:40 <meejah> this week, karsten and i will look into deployment questions etc.
17:58:47 <luketheduke> great (:
17:58:49 <karsten> yup
17:59:00 <meejah> baumanno: did you have anything to add?
17:59:05 <meejah> feverDream: ditto
17:59:54 <baumanno> meejah: absolutely not :) i'll be watching progress on onion-py and maybe i'll be able to give some thoughts about the util-methods
18:00:02 <meejah> okay, great
18:00:16 <baumanno> but, as lucyd aptly put it: most of what I thought I needed is already in the hacky scripts on github ;)
18:00:40 <feverDream> meejah: not much here, I have to play catch up with you guys for some time
18:00:53 <meejah> luketheduke: BTW, don't discount possibility of onionpy being debian-packaged -- that's always an option too
18:01:19 <luketheduke> meejah: there will not be any new packages in wheezy
18:01:31 <meejah> right, but there's wheezy-backports
18:01:39 <luketheduke> I don't know what the policy on that is
18:02:10 <luketheduke> also I feel like I'm far too un-noteworthy a person to have a package in an official debian repo o.O
18:02:29 <meejah> me neither, exactly, but txtorcon release magically appear in there
18:03:03 <meejah> i'm *pretty* sure that's not a needed qualification ;)
18:03:07 <luketheduke> interesting
18:03:08 <meejah> #endmeeting