13:11:16 #startmeeting 13:11:16 Meeting started Fri Jun 12 13:11:16 2015 UTC. The chair is zack. Information about MeetBot at http://wiki.debian.org/MeetBot. 13:11:16 Useful Commands: #action #agreed #help #info #idea #link #topic. 13:11:23 start time slipped my mind :) 13:11:32 matthieucan: thanks for the SMS! 13:11:36 sure 13:11:39 #topic roll call 13:11:44 * orestis 13:11:45 looks like everyone is here 13:11:54 (except me, that is :-P) 13:11:57 ack 13:12:13 #topic next meeting 13:12:23 shoudl be fine for me 13:12:27 does anyone have issue with next week, same time? 13:12:34 fine fo rme as well 13:12:42 no problem 13:12:57 fine here 13:13:00 #agreed next meeting as usual 13:13:13 so, who was first in the weekly review, again? :) 13:13:33 last time me :p 13:13:40 ok 13:13:47 #topic weekly review - orestis 13:14:05 orestis: please go through your items one by one 13:14:10 alright 13:14:28 well all items with PR#6 are done 13:14:32 i mean the PR is closed 13:14:37 nice job for that! 13:14:46 so they've been reviewed as well, right? 13:14:51 yes 13:14:56 that covers parse and render d/copyright 13:14:59 yes 13:15:00 ok, please move them to archived 13:15:16 sorry, to "reviewed" 13:15:34 next is the file by file api doc 13:15:53 which is ok i think 13:15:59 in which list is that? 13:16:08 Done 13:16:15 "license api file-by-file" ? 13:16:38 Submit proposal for file by file api 13:16:47 k 13:16:52 yes, we've consensus on that I think 13:16:56 and I've merged your code 13:16:59 so it's definitely reviewed 13:17:15 there is the issue raised by jpleau about where the api will be rooted 13:17:18 so there are two items for PR#10 which is under review 13:17:21 by I think it's kinda independent from the API design itself 13:17:43 yeah, we can easily change route roots 13:17:48 orestis: those are the 2nd and 3rd item under "done", right? 13:17:52 yes 13:17:55 matthieucan: ack, so let's avoid blocking on that 13:18:02 yep 13:18:10 matthieucan: jpleau: are you still on the review of those two? 13:18:19 I think the api doc needs to be updated with the /suites|version|all/ URI 13:19:00 orestis: ^^^ do you confirm? if so, can you please add an item for that? 13:19:01 yes, reviewing PR 10. It looks ok, I should be able to test it more deeply tonight 13:19:08 yes that's true. i am planning to do that once the PR is confirmed 13:19:09 Unless it was decided not to that of course :) (/api/file/package/suite|version|all/path) 13:19:16 sounds great 13:19:28 For checksums did we want to do similar or keep the package and suites in query string parameters? 13:19:31 perfect if it matches the sources api 13:19:39 hm 13:19:48 I think orestis suggested it was better to keep ?package and ?suite 13:20:01 * zack has no strong opinion on this 13:20:01 for the checksum i proposed not to change them since it would defer from the s.d.n api 13:20:34 orestis: oh nevermind, yes. I forgot we had checksum on sources api. :) 13:20:45 I want to do more tests tonight though bnefore confirming! 13:20:46 jpleau: which doesn't mean it cannot be improved of course :) 13:20:53 but the day we do that, we can do it more broadly 13:21:00 s/can/should/ 13:21:14 so, next item is "show license of individual..." 13:21:31 although I don't understand why that is specific of sha1 13:21:38 searching by package/version/path should be the same, no? 13:21:46 yes that i have everything ready with the new template etc.. i didn't pull request coz it depends on PR#10 13:22:21 orestis: but is it specific of sha256 searches? or more general? 13:22:27 orestis: great news, I'll hurry up with PR#10 13:22:31 well i was thinking it wasn't going to be exactly the same since searching by checksum gives much more results 13:22:56 right, but ideally it would be using the same code, except it will show multiple results in one page 13:23:00 or something like that, right? 13:23:03 and i tried to include some 'stats' for the checksum i.e which is the most frequent license of the checksum etc 13:23:18 zack: What I was proposing for sha256 was : /api/sha256/(package_name|all)/(suite|version|all)/checksum 13:23:39 If we can adapt the sources api, it'd mean all the APIs follow the same route scheme 13:23:50 zack: well yes the code will be almost the same with file search.. i just haven't had the time to do that as well 13:24:01 orestis: that's OK, it was just to understand 13:24:23 yes i know :) 13:24:27 cool 13:24:31 so, that's pending 13:24:42 orestis: maybe you can add "PR: waiting for PR#10" 13:24:42 so as soon as we have PR#10 merged i ll pull request that 13:24:48 (in the title) 13:24:53 yes sure.. 13:25:05 jpleau: given it depends on a change on the already deployed API, can we make that a separate feature/discussion? 13:25:19 zack: yep ! 13:25:29 so, next topic 13:25:31 and for the batch api doc i am working on it but i am waiting to have the final routes for the file-by-file 13:25:38 #topic orestis - next week 13:25:44 orestis: ack 13:25:50 orestis: sounds like a great link to next week planning ;) 13:26:13 what would you like to work on? 13:26:29 so next week i guess i ll do the batch api but besides that i don't know what you think its best to push for 13:26:54 uhm 13:27:10 that would be good already 13:27:23 then, I guess there are 2 independent topics that can go in parallel, and according to your pref 13:27:29 what about license statistics? you said you began doing that? 13:27:32 one would be the license scanner, client-side 13:27:41 another is looking into SPDX and export 13:27:52 matthieucan: right, I'm working on it for a research project, but i don't have the resulting data yet 13:27:59 it's taking longer than expected (no wonder) 13:28:17 a task related to do that would be deciding the DB schema/data model for storing license statements in the DB 13:28:27 matthieucan: no it is just a simple thing i did.. 13:28:36 (it's already on trello: DB schema for persistent copyright files) 13:28:51 yes i was thinking that its better to have the db before the stats.. but then wouldn't that interfere with clemux work? 13:29:13 hopefully, clemux work will be done without affecting the DB structure 13:29:24 so as long as your interface is the DB, it should be fine 13:29:33 ack 13:29:35 orestis: but really, it's about what you'd like to prefer to work on 13:29:42 above are macro areas to prioritize 13:29:52 feel free to pick from there, and mix and match other minor items 13:30:06 (in the future, it would be useful if these debates happen _before_ the meeting 13:30:15 so that we arrive at the meeting with already a proposal from you on what you'd like to work one) 13:30:22 ack 13:30:44 so, your preferences? 13:30:54 (or we can decide after the meeting, if you need to think about this) 13:31:39 well ok i think i ll do the batch api, fix the bugs that we discovered (the heuristics for machine readable, fail to dump when render fails) and yes i d like some more time to think about it and ping you about something else 13:31:48 ok, great 13:32:01 please assign to yourself those items, and ping us afterwords for the rest 13:32:16 #topic weekly review - clemux 13:32:21 clemux: you're on stage! 13:33:29 so, I submitted a first draft of the workflow graph for celery tasks 13:33:43 and we've reached consensus on it, right? 13:34:30 the only problem left is update_metadata 13:34:42 right, but it doesn't seem to be a blocker 13:34:50 I'm fine with leaving it as you designed for now 13:34:57 we can improve/split it later 13:35:06 ok 13:35:13 IIRC i'm waiting for your PR to merge 13:36:09 I have a commit to push, I'll do that immediately after the meeting 13:36:14 k 13:36:18 (quick question) Is it possible with the tools we have to have an html version of the graph with tooltips describing each task? It could be great addition in doc/, and it's visual 13:36:33 not sure if dot handles that 13:36:47 it can create html maps, IIRC, but of the old-style kind 13:37:11 oh, ok. I'll see if I can come up with something for that :) 13:37:18 awesome 13:37:26 clemux: what else from this week? 13:37:45 the next task was to determine how to use the sqlalchemy session in celery tasks 13:37:49 I'm still stuck on that :( 13:37:58 is that the nested transaction question? 13:38:02 I've thought about that a bit 13:38:10 yes that's the main problem 13:38:31 so, we can dig more deeply post meeting, but in a sense I think we can be way more "adventorous" than the old updater was 13:38:44 I think it'd be totally fine to have one transaction per package, for instance 13:38:53 without nesting transactions 13:39:06 clemux: that would solve most of the problem, right? 13:39:19 it'd be simpler, yes 13:39:26 clemux: yep sorry, couldn't help you on that :/ 13:39:41 zack: however, having all hooks in a single transaction is still a problem for me 13:39:49 for a given package, I mean 13:39:57 right 13:40:03 but we probably don't need that either 13:40:13 for instance, sloccount can run in its own transaction 13:40:25 as long as it knows the package has been unpacked already 13:40:33 would that help? 13:40:43 yes 13:40:58 that should unblock the task 13:41:08 and, if we reach a point where we cannot have smaller transactions any more, well too bad, we will group together tasks that you've currently split 13:41:13 it will just reduce parallelism 13:41:20 (but not by much) 13:42:04 indeed 13:42:19 anything else to report about, before moving to next week planning? 13:42:41 zack: so you're saying a package could be updated with the results from sloccount but not e.g. ctags? 13:42:48 (if ctags failed) 13:43:03 clemux: for instance, yes 13:43:11 as long as we can keep track of the fact the task has failed, that is 13:43:19 this should not be a problem 13:43:57 ok, I feel better, I should be able to move on faster now 13:44:07 more importantly, I guess I'm saying that even if we need --- for now --- to group all plugins together in a single tasks, 13:44:12 it wouldn't be such a big deal 13:44:28 because the increase in parallelism in treating one package independently from the other would be huuuge anyhow 13:44:53 zack: I thought one of the point of the new architecture was to be able to rerun e.g. only ctags on all packages 13:45:04 that's correct, yes 13:45:05 with new release, other options, ... 13:45:15 (and a good point) 13:45:21 that's why I hadn't considered going that way 13:45:34 ok, so let's settle with what I mentioned before :) 13:45:39 if a single plugin fails, then too bad 13:45:48 the others would have succeeded anyhow 13:46:01 deal? 13:46:09 it'll be retried later and the failures will be monitorable 13:46:12 yup 13:46:15 awesome 13:46:29 anything else to discuss before planning next week? 13:46:40 nothing else 13:46:48 #topic next week - clemux 13:47:05 there's plenty of stuff on "this week" for you already 13:47:09 should we just confirm all of it? 13:47:12 yes 13:47:29 ok, I guess the transaction issue has been an unexpected blocker this week, right? 13:47:32 yes 13:47:42 great it's solved now ;) 13:48:03 ok, so last topic, a great classic 13:48:05 #topic misc 13:48:14 anything else people would like to discuss? 13:48:31 nope 13:48:39 I'm fine 13:48:54 nothing here 13:49:12 let's adjourn then, and sorry again for my delay! 13:49:35 nothing else from me 13:49:41 #endmeeting