11:27:32 <buxy> #startmeeting
11:27:32 <MeetBot> Meeting started Tue Aug 13 11:27:32 2013 UTC.  The chair is buxy. Information about MeetBot at http://wiki.debian.org/MeetBot.
11:27:32 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
11:27:55 <buxy> #topic review of the last week
11:28:05 <mlalic> Okay... well, firstly, the last story is almost finished, but not quite fully
11:28:18 <buxy> mlalic: I haven't had the time to look at your last push yet
11:28:20 <mlalic> The rewrite of existing todos/panels took slightly longer
11:29:13 <mlalic> I saw the email just a few seconds ago, so I figured you didn't catch the time yet :)
11:29:16 <pabs> hi buxy, mlalic
11:29:22 <mlalic> hi pabs
11:30:01 <buxy> mlalic: yeah, I'm offsite so I'm not always connected, I took the time to review everything I had last night but couldn't send the mails before now
11:31:09 <buxy> mlalic: there's one thing I forgot in my review emails, I belive you should include the meta-data severity/created/last updated in the header of the long version of the ActionNeeded
11:31:36 <buxy> you only put one of them IIRC
11:31:43 <mlalic> You mean the popup?
11:31:59 <mlalic> It is there in the shortdescription and the long description dedicated page, but not in the popup
11:32:00 <buxy> yes (and the full page for non-javascript users)
11:32:24 <buxy> in the shortdescription?
11:32:32 <mlalic> Ah okay... I thought it'd be kind of redundant in the popup since it's right there in the title
11:32:49 <mlalic> short description == the title preceding the question mark
11:32:57 <buxy> right
11:33:07 <buxy> but it's weird to put all those meta-data there
11:33:12 <buxy> we
11:33:33 <buxy> we wanted to shorten it to what's really relevant
11:33:51 <mlalic> Right, right... I'll move it to the title then, no problem
11:34:06 <buxy> the severity might be ok with a small icon or something, but the dates seem useless in the title
11:34:25 <mlalic> ah no, the date isn't there
11:34:43 <mlalic> just the severity, albeit in text form for now
11:35:34 <buxy> #todo mlalic to enhance the ActionNeeded popup
11:35:56 <buxy> #todo mlalic to enhance the ActionNeeded popup with creation date / last update date
11:36:54 <buxy> I'm looking at your todo items of last week
11:37:23 <buxy> in http://meetbot.debian.net/debian-qa/2013/debian-qa.2013-08-07-11.49.html
11:37:32 <buxy> the non-regression test I saw
11:37:38 <buxy> what about the rest?
11:37:56 <mlalic> The files are extracted on the server
11:38:04 <mlalic> I had to make a few improvements along the way, though
11:38:22 <mlalic> Such as automatically limiting the size of cached sources
11:38:31 <mlalic> Since the space runs out pretty damn quickly...
11:38:56 <buxy> I saw that, quite a few things that we didn't anticipate
11:39:32 <mlalic> Yeah, and Python2.7 didn't automatically untar lzma compressed archives
11:39:54 <mlalic> (I had read some time before that tar now handled .xz files, but didn't realize they only added that to python3)
11:40:07 <mlalic> s/tar/tarfile package
11:40:30 <mlalic> But anyway, now that's done, it all worked out pretty well
11:41:00 <mlalic> The other thing was exim settings... I read through quite a lot of stuff and experimented with the different settings
11:41:07 <mlalic> FIrst off, .forward files vs the current set up makes no difference
11:41:34 <mlalic> It may be cleaner for administration, but as for the delivery/process spawn rate, it's the same
11:42:32 <mlalic> However, there are settings which cause received emails to be put in a queue, instead of delivering them (and thereby spawning a python process) after a certain load is reached
11:43:04 <mlalic> Then, you need to configure how often a process which cleans up the queue runs (and how many of those processes are run simultaneously
11:43:15 <mlalic> There is an alternative of always queuing the received mail
11:43:22 <mlalic> This could be a bit safer, in my opinion
11:43:29 <mlalic> Since we can then always know the exact load
11:43:59 <buxy> assuming the machine doesn't do anything else :)
11:44:03 <mlalic> In my tests, the settings where it should automatically start queuing were sort of "random" in the amount of delivered mails before firing up the queuing
11:44:18 <mlalic> Well, yeah, I mean the PTS dispatch processes at least, right?
11:44:34 <mlalic> It's safer than that being random as well
11:45:00 <mlalic> although, to be fair, it's not random. It uses the system load average, but that is something I don't quite understand well enough
11:45:33 <mlalic> All I know is that it is based on the CPU time used
11:45:52 <mlalic> ...by all processes on the machine
11:46:37 <buxy> and on the number of processes waiting on it (and I/O is involved as well IIRC)
11:47:09 <buxy> I don't care if it's random, is it reliable enough to avoid triggering OOM ?
11:47:40 <mlalic> With a low enough value, it seemed good to me
11:48:33 <buxy> mlalic: so what settings did you put in production on pts.debian.net?
11:49:13 <mlalic> But, another aspect of it is setting it up in a shared environment. I tried looking for a way to set it up so that only certain addresses use these setting, but it seems it must be global
11:50:01 <mlalic> For now it is the option where it should queue after the system load reaches over 2
11:50:04 <mlalic> In the tests I ran
11:50:26 <mlalic> It meant it usually stops after about 20 processes
11:51:59 <buxy> hum, that value seems very low, with two background tasks, you could block all mail delivery
11:52:54 <buxy> and with multiple cores, the machine should not really be considered over loaded at that point
11:53:20 <buxy> mlalic: did you compare the memory usage of the new dispatch vs the old one ?
11:54:05 <mlalic> Ah, no actually. I was also looking at the todo items from the meeting and didn't think of that
11:55:13 <buxy> #todo mlalic to compare memory usage of old dispatch.pl to new dispatch implementation
11:55:20 <buxy> So
11:55:51 <buxy> you will take care of this this week, it shouldn't take long but it's interesting to know
11:56:13 <buxy> at least to be able to define limits in the number of process that we can allow depending on the memory available
11:56:27 <mlalic> Yeah, sure
11:56:52 <buxy> mlalic: can I reenable the forward from the real PTS according to you?
11:58:26 <mlalic> Well, I think the setup should not crash as it is, at least. The blocked delivery when a task is running is a background is a possibility, though...
11:58:49 <buxy> pabs: are there task from the backlog on https://trello.com/b/faDgzjwO/pts-rewrite that you wanted to take care of?
11:58:50 <mlalic> Hmm, but wait, actually
11:59:00 <buxy> mlalic: ok, reenabled then
11:59:23 <mlalic> The delivery is not blocked completely. It is just slowed down. The process(es) that clean the queue are not influenced by the load setting
11:59:32 <mlalic> The load setting just causes mail to be queued
11:59:36 <pabs> there are some that I could probably look at, not sure how much I will get done during DebConf though
12:00:06 <buxy> mlalic: ok, good
12:01:01 <buxy> pabs: ok, then I'll schedule his work assuming you
12:01:14 <buxy> won't do anything this week
12:01:23 <buxy> you can always pick up from the remaining task
12:01:43 <buxy> #topic next week planning
12:02:07 <buxy> so we're going to plan for 1 day more since next week's meeting will be on wednesday again
12:02:42 <mlalic> Right
12:03:45 <buxy> when does the gsoc end?
12:04:03 <buxy> ie how many weeks do we have left?
12:04:19 <mlalic> "Suggested pencils down" is Sept 16
12:04:26 <mlalic> Final evaluation Sept 27
12:04:37 <mlalic> And firm pencils down Sept 23,
12:04:54 <buxy> ok, 5-6 weeks then
12:05:52 <buxy> backlog still has 2 - 2.5 weeks of
12:05:55 <buxy> work
12:06:44 <buxy> and we have lots of stuff in draft form still
12:06:56 <buxy> we'll probably have to make some firm choices
12:07:26 <mlalic> The backlog is all the current PTS features, except the simple package subscription through the Web interface, as far as I've seen
12:07:57 <buxy> oops, sorry
12:08:03 <buxy> wrong column
12:08:10 <mlalic> No biggie :)
12:09:10 <buxy> well, the current PTS has this RDF thingie that almost nobody is using but olivier berger has put a lot of work on it
12:09:50 <buxy> the current PTS also has some SOAP api (though marked experimental)
12:10:03 <mlalic> Ah, yes, indeed. We've only included news RSS feed so far in the backlog
12:10:28 <buxy> but really I want to focus on
12:10:37 <buxy> where there's most value
12:11:04 <buxy> and some of the existing features have less value than some of the new ones
12:12:38 <intrigeri> Please pull from feature/mention-website-repo on git://gaffer.ptitcanardnoir.org/piuparts
12:14:58 <pabs> h01ger: ^
12:15:34 <buxy> mlalic: please revie
12:15:55 <buxy> review the estimation of the RSS feed story, I added a new point at the end of the acceptance test
12:17:16 <intrigeri> (I'll file a bug unless it's merged promptly.)
12:17:26 <mlalic> buxy: done
12:20:11 <buxy> mlalic: how did you adapt the buildd log check to the ActionNeeded framework?
12:20:54 <buxy> mlalic: people requested me that the severity of the message be adjusted based on what's reported
12:21:30 <mlalic> buxy: It displays a single item in the panel giving the number of errors/warnings. The full description provides the additional links
12:21:55 <mlalic> buxy: Extra data then contains the count of errors and warnings which are used in the template.
12:22:43 <mlalic> buxy: Supporting these adjusted severity levels is also quite possible on a case-by-case basis, of course, since that's quite item-specific
12:22:59 <buxy> mlalic: can you adjust the severity to be high when it has errors, and low when it has only warnings ?
12:23:38 <mlalic> buxy: Sure thing, that's no problem.
12:24:15 <buxy> #todo mlalic to enhance the build log checks to vary severity depending on presence of errors or not
12:25:10 <buxy> mlalic: I have put 31 points of story (not counting the one currently in progress), does that seem ok
12:25:11 <buxy> ?
12:25:20 <buxy> (given the supplementary day)
12:25:44 <mlalic> Yes, I think it's ok
12:26:49 <buxy> Good. I think that the rest of the backlog will not be dealt next week. Those are remaining low-value features and we'll keep them for the end and/or for others to implement (they are relatively easy).
12:26:55 <buxy> So
12:27:38 <buxy> I'll have to check with zack and others on future priorities and we will have to write up stories and you'll have to evaluate and add tests
12:29:06 <buxy> #topic misc questions
12:29:13 <buxy> mlalic: do you have further questions for me?
12:29:50 <mlalic> buxy: Since we're here, I'd want to check with you about the story which is in progress: watch file ActionItems
12:30:38 <mlalic> The way I've started implementing it is that I have 4 different ActionItemTypes: new-upstream-version, watch-filure, watch-file-broken, and watch-file-available
12:31:00 <mlalic> This more or less corresponds to what is written in the acceptance tests of that story (which is still using todos/problems nomenclature)
12:31:41 <mlalic> Does that sound right, or should I change something before finishing it.
12:32:03 <mlalic> (I've implemented the first two items so far, but it'd be easier to change it now, than after finishing it all...)
12:34:02 <buxy> mlalic: the first two are related to the output of udd-dehs and the two last are related to the qa.debian.org downloads is that correct?
12:34:21 <mlalic> buxy: Yes
12:35:56 <buxy> seems ok
12:35:59 <buxy> in terms of severity
12:36:10 <buxy> high, high, low,
12:36:12 <buxy> wishlist
12:36:26 <mlalic> Okay, great.
12:37:52 <buxy> mlalic: otherwise I want to let you know that the rewritten PTS has been getting quite some interest here
12:38:03 <buxy> Ubuntu is interested in deploying it for them
12:38:31 <mlalic> wow, that sounds really nice
12:39:16 <mlalic> I've got to admit that I'm glad that the work I've been putting in has an effect :)
12:39:25 <buxy> and other persons found it great to make it available to derivatives as a way to help them track their divergence with Debian
12:39:44 <buxy> (but that part still requires work that's not planned in the current project)
12:41:13 <buxy> so keep up the good work :-)
12:41:43 <mlalic> thank you very much! It really means a lot :)
12:42:36 <mlalic> We're only just over the half-way point of GSoC, but as I said before starting, I'd really like to keep contributing to the project even after it program ends
12:42:57 <buxy> mlalic: that would be really great!
12:43:34 <mlalic> Not 40+ hours a week with the school year starting, but definitely a part of my time
12:43:37 <buxy> I won't forbid you to continue working on the PTS or on anything else
12:43:48 <mlalic> Good to hear :)
12:44:03 <buxy> and if you need my help to get introduced to some other part of the project, I'll gladly do it for you
12:44:56 <mlalic> Thanks! That is possibly the biggest reason I regret not being able to attend debconf. But ah well, there's still time :)
12:46:10 <buxy> mlalic: can you configure apache to announce UTF-8? otherwise text files are badly displayed in the browser
12:46:49 <mlalic> yeah, sure
12:47:18 <buxy> mlalic: and move the action needed box above the ne
12:47:22 <buxy> above the news one
12:47:44 <buxy> bugs should also be above links
12:48:33 <buxy> and testing migration should also be above news
12:48:44 <buxy> just keep the current organization concerning panels
12:48:59 <mlalic> Eeeh, testing migration is a bit more difficult
12:49:16 <mlalic> If you remember, I originally had a settings value which gives the positions of panels
12:49:30 <mlalic> But then I changed it so they're automatically registered/displayed
12:49:42 <mlalic> So the order is given by the order in which the classes are loaded
12:49:58 <mlalic> Since testing migrations is not in core, it'll be loaded after the news panel
12:50:16 <mlalic> At that point, we said we wouldn't concern ourselves with this
12:50:21 <mlalic> But it seems now that time's come? :)
12:50:50 <mlalic> Maybe a panel could set some sort of prefered_position integer which is used to sort them?
12:51:25 <buxy> yes, that should be good enough,
12:51:36 <buxy> or maybe panel_importance
12:51:58 <buxy> and the most importants are at the top
12:52:51 <mlalic> Yeah, sounds good... I'll add that in now then
12:53:11 <buxy> #todo mlalic to add a way to sort display order of panels
12:55:01 <buxy> mlalic: drop the capital on "Homepage" to be consistent with other links
12:55:23 <buxy> otherwise I think we're good to go
12:55:36 <mlalic> Done :)
12:55:57 <mlalic> Yeah, as far as I'm concerned seems good
12:56:52 <h01ger> pabs, i have a highlight on piuparts :)
12:56:59 <pabs> ah
12:57:23 <buxy> hum
12:57:24 <buxy> #todo mlalic to enhance panels so that they are hidden when they are empty
12:57:24 <buxy> #endmeeting