18:00:22 #startmeeting trac migration 18:00:22 Meeting started Tue Oct 1 18:00:22 2019 UTC. The chair is gaba. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:22 Useful Commands: #action #agreed #help #info #idea #link #topic. 18:00:33 our agenda is here: https://pad.riseup.net/p/e-q1GP43W4gsY_tYUNxf 18:00:52 please review it and add/edit/comment when necessary 18:02:06 looks OK to me 18:03:01 hello! 18:03:06 o/ 18:03:49 should we do item #1? 18:03:56 yes, let's start 18:04:01 cool. i can give a status there 18:04:06 update on what we said we were going to look into 18:04:11 thanks ahf 18:04:59 so friday i did a small experiment with the model that anarcat suggested from last week: we move tickets from 1 ... N from trac to gitlab into a dip.torproject.org/xxx/legacy project, fill in the "gabs" with dummy tickets and then "move" the tickets out to the actual gitlab projects (such as dip.torproject.org/tpo/tor) 18:05:12 the suggestion anarcat came up with works that way we hoped to, so i think we should do that 18:05:32 and it does the smart redirection that we hoped for which will make irc ticket handling via the bot and bugs.torproject.org migration easier 18:05:35 that is it i think 18:06:01 sounds good 18:06:14 * anarcat oneline 18:06:27 I think we should work in a new "tpo" group (not use the group 'the tor project' we have now) and do that 18:06:29 i havent rewritten migrate.py yet to do this. that is a task for friday 18:06:38 ahf: that's awesome news! 18:06:38 gaba: agreed 18:06:41 tpo/legacy project 18:06:49 anarcat: it is! it was a very good idea even though i was critical to it at first 18:06:56 and then the script will also move stuff from legacy into its own group/project, right? 18:07:19 yes, it can move things to everywhere based on the config we make 18:07:25 and leave tickets we dont move at the leagcy project 18:07:26 anyone disagreeing on this? 18:07:29 which i think is a good feature too 18:07:35 so we can slowly see what we havent migrated 18:07:40 sounds good 18:08:14 yes, and we can test 18:08:15 gaba: i think moving to tpo is a good idea 18:08:41 for people that are using 'the tor project' then they/we can merge them manually into the right tpo group/project. 18:08:42 my goal for this friday is to see if i can copy ALL tickets from trac over to gitlab, also to have an idea of how long it takes 18:09:25 sounds good 18:09:25 yes gaba 18:10:02 2 thing we said the were going to do is rename torproject to tpo. I didn't do it as I think is better migrate to a new fresh group/project. 18:10:34 maybe we should keep the current one around and then when we do the switch from trac we start using tpo? 18:10:41 I think if you rename you will have to update the git hooks 18:10:42 that allows me to regenerate the tpo namespace in the script 18:10:52 hiro: the git *hooks*? not the URLs? 18:11:05 i think renaming projects is safe - it leaves redirections in place and all 18:11:08 yes I meant the urls configured for the git hooks 18:11:11 we should do it earlier than later 18:11:24 what are those git hooks? 18:11:35 the sync hooks? 18:11:37 sync from tor git to gitlab 18:11:43 yes. I say let's not mess with 'the tor project' and migrate into a new one. We will need to update the offical tor project group and merge projects that people are using in gitlab. 18:12:04 yes the redirect works but if someone makes a change I think we should make sure we update those 18:12:24 ok? 18:12:29 i would argue that the current gitlab instance is not designed for production and can change at any time, so we shouldn't offer stability garantees like this just yet 18:12:34 but i don't have a strong opinion on this 18:12:44 anarcat: what do you mean? 18:12:49 we can change it if we want! I just wanted to say to not forget to let gitadmins know 18:12:54 okay 18:13:04 gaba: i'm worried that people start using dip in production before it's ready 18:13:11 it would make our lives much harder during the transition period 18:13:15 it already does, i feel 18:13:16 oh, I see. I think some people are using it 18:13:19 people are using dip in production 18:13:22 the git hooks is an example 18:13:23 okay 18:13:26 i didn't realize that 18:13:27 web and ux are using it 18:13:28 but it is clear that is only an instance for test 18:13:43 the footer does say "DANGER WILL ROBINSON! MIGHT EXPLODE AT ANY TIME! TESTING THIS PLATFORM RIGHT NOW!" 18:13:56 the ux and web team is living on the edge 18:14:05 typical of web folks ;) 18:14:09 yeah I like danger 18:14:13 ready to let us know when everything breaks 18:14:43 it is very good we have some people experimenting with this so we get some feeling and catch early issues. i am very glad those teams are doing that 18:14:44 :) 18:14:50 but only for tickets to be honest. because git is just a sync from tor git 18:14:57 yep 18:15:29 ok 18:15:42 3rd? how to deal with spam in gitlab 18:16:05 yep 18:17:04 the only thing about spam in gitlab I found is this one: https://about.gitlab.com/handbook/support/workflows/managing_spam.html that is the integration with akismet... 18:17:17 yeah, and akismet wants the users IP 18:17:25 and the user IP is probably a tor exit node often :-/ 18:17:30 yep 18:18:19 bridgedb has a captcha that doesn't used akismet 18:18:36 some bot are breaking it... but I am not sure those are spam bots 18:18:51 so maybe we can have our own captcha thingy 18:18:59 yeah. it's almost like there are two totally different audiences we need to consider for the spam. 18:19:00 we had a conversation about this, it was interesting.. there was at least one user that was saying they couldn't solve the bridgedb captcha after multiple tries 18:19:13 there are ways to report users and then admin can remove them... 18:19:23 one is people using automated tools to spam our ticket tracker. for those, some really stupid thing, like "type this word, it's the same word every time" trick would work for them. 18:19:32 last time we talked about this with nickm, we did identify two different requirements: one was abusive users and the other was spam bots, which are subtly different 18:19:52 (e.g. #14680) 18:20:32 i spend some time last friday thinking about the meeting we had last time, around the following: preventing abusive spam bots to sign up, handle signing up, and handle anonymous users 18:20:43 but the other audience is actual jerks. and putting up a captcha that is easy to break en masse is... it depends how we want that arms race to go 18:20:47 i wrote this that i shared with gaba, but that other people can read now too: https://pad.riseup.net/p/MMEeAUdiUQ-XwhDV-Eww 18:20:51 it is a long read and i might be wrong 18:21:05 and it is a bit of work, but i also have some time each friday the next long time to do things 18:21:27 ahf: how much work you think it would be? 18:21:34 gaba: 4 fridays or so 18:21:54 arma2: we *could* separate those concerns and have two mechanisms 18:22:04 i am more worried about spam than jerks right now 18:22:15 i know that jerks are also a problem, of course 18:22:26 but spam is a clear and present danger to the stability of the service 18:22:35 while jerks, well... gitlab would still run 18:22:49 the proposal in that document allows us to continue to have anonymous users, with moderation applied, but requires us to have a moderation team that approves and rejects comments and new users signing up 18:23:02 ahf: it would have been great to see this document before the meeting, because right now it's unlikely everyone will have time to digest it in a reasonable timeline now :) 18:23:05 i think that will prevent most issues, but will be more work than just taking debian's salsa isgn up form 18:23:21 anarcat: yeah, sorry, i had noted it down, but forgot to share it before i went out to shop 18:23:28 anarcat: we dont have to decide on that now 18:23:36 cool 18:24:01 i just think the user sign up/anonymous users is related to spam and jerk handling 18:24:17 i like the goals it describes, so i am cautiously optimistic that i like the very long document :) 18:24:18 and i don't think gitlab itself have a good way for us to handle that build in, so i think we need to build something on top 18:24:51 ahf the only thing I am afraid of is that we build a tool that we need to maintain and keep updated as gitlab changes 18:25:02 I'm only worry on how long it could take with not many resources if we go that route 18:25:11 is cerberus kind of like an elaborate salsa signup app but with moderation? 18:25:18 ie. do people register on gitlab or cerberus? 18:25:42 even if it's not that complicated, still there will be libraries to update too and thigns might break 18:25:53 hiro: that is a concern for me too, but i looked at the gitlab api around issues, notes, and user signups and very little have changed there in the lifetime of gitlab 18:25:58 and they do versioned api's 18:26:10 i think cerberus looks quite complicated - you'd have to reimplement all forms, authentication and controls on top of the existing issue system 18:26:17 gaba: it is volunteer time IMO and volunteer time is something i have a bit of right now, which i wont have in a year 18:26:24 anarcat: yes 18:26:37 anarcat: i disagree that it is very complicated 18:26:45 it is a webapp that uses the gitlab api 18:26:54 ahf: let's say you use twisted, or flask, or rails or pyramid to build cerberos... there will be updates to this libraries that you will have to consider 18:27:15 hiro: yeah, i was planning on doing it in django if people think it is worth it 18:27:34 but let's talk about 1 year from now 18:27:37 wouldn't you be reimplementing all of gitlab on top of gitlab, but with django? 18:27:52 when you will not have volunteer time maybe 18:27:53 i'm sorry, i don't want to sound negative, but it feels like a lot of work :) 18:27:55 nope. that is in the document. only a small subset of comments/ticket handling 18:28:04 well that's the first step 18:28:09 but then people will ask you for wiki edits 18:28:10 I would be quite surprised if this can be done in 4 days 18:28:29 and "just comments/tickets handling" is already quite a piece of work' 18:28:40 anarcat: did you read teh whole pad? 18:28:41 "just a commenting system" are famous last words i heard a web developer say ;) 18:28:48 next thing you know, they wrote drupal :p 18:28:57 if it gets to that i would drop it 18:29:00 gaba: i'm at around line 90 18:29:15 gaba: bootstrap and django gets you very far very quickly 18:29:36 i think cerberus and (incidentally) the salsa registration form are interesting solutions we could deploy shortly to fix the spam/signup problem 18:29:41 ahf: just finished reading 18:29:45 looks good to me 18:29:48 (your proposal) 18:29:52 ahf: i'm concerned about long-term maintenance costs, and what happens if something breaks and you're not around to fix it? 18:30:09 ahf: sign me up for a moderator 18:30:14 i think it's worth considering in the abstract 18:30:15 what i'm worried about is feature (3) in the cerberus spec, that seems to be designed to workaround the "we do not want a cypherpunks" requirement, but at a large maintenance cost 18:30:23 ahf: I think the first prototype will be easier to implement 18:30:49 catalyst: there is some bus factor issue for sure, but the python part of it wont be hard for others to look into i think, if there is a code problem. if there is a service problem then i don't know 18:30:54 hiro: yep 18:31:15 catalyst: if something breaks it will impact anonymous users and user signup 18:31:18 i would also be worried about having two distinct user interfaces for the same data... it could be very confusing for new users who wouldn't quite know where to engage 18:31:30 which can wait a day or two before i return i think, in the absolute worst case 18:31:37 catalyst: this issue would be there with salsa signup as well 18:31:47 except salsa signup has at least one other deployment 18:31:50 ahf: I think the first prototype will be a 10th of what we will need 18:31:55 this would be our own precious pet :) 18:31:56 only anonymous users would deal with this interface 18:31:59 the rest is just gitlab 18:32:07 anarcat: yep, that is true 18:32:16 hiro: 1/10 of what we need? 18:32:17 right? 18:32:22 yeah gaba 18:32:23 gaba: i understand that's the desired goal, but it might not be obvious for users 18:32:28 only anonymous users would need to use this 18:32:32 how *would* that be made obvious? 18:32:37 well, that is a UI/UX question 18:32:46 yes, it is :) 18:32:51 UX kind of matters :) 18:32:59 gitlab is hard enough as it is with all its javascript stuff 18:33:02 agreed. i havent done in visual prototypes of this 18:33:23 can we have custom pages? i mean, can we edit templates for that scenario? 18:33:41 the real question is: can i embed a lisp interpreter ;) 18:33:48 anarcat: it will be outside of gitlab the application itself; so we decide entirely how it looks 18:34:09 ahf: I mean that we will find out that what we need is more complex of what we anticipated 18:34:10 https://signup.salsa.debian.org/ like this 18:34:23 just think of this as: Want to sign up? Want to be anonymous? in those two boxes 18:34:30 hiro: yeah, that might totally be 18:34:42 so just to take the salsa signup form as an example here... 18:34:51 it would be only for anonymous users. If people move into having an account then they forget about it. I do not think it will be confusing there. 18:34:56 this is a piece of software that is basically *one* form: register a user 18:35:03 we had hesitations about deploying it because of maintainability issues 18:35:08 because it's a custom thing on top of gitlab 18:35:30 and because we arent sure if we want to handle user registration at all for external users as we discussed in the last meeting 18:35:31 now we're talking about implementing that, plus moderation (so an admin dashboard), plus comment submission 18:35:41 correct 18:35:45 and i know django gives us a lot of that stuff for free 18:35:58 but i'm skeptical it's a worthwhile project just to solve the "cypherpunks problem" 18:35:59 salsa does not satisfy the requirements we identified at last meeting 18:36:07 maybe i underestimated the seriousness of that problem 18:36:07 right 18:36:12 okay, maybe i missed some reqs 18:36:20 anarcat: that is the real question, but that is a question that i am the wrong person to ask, because i think cypherpunks is a pretty lame thing 18:36:25 but there are people in the org who thinks it is great 18:36:34 and this thing is trying to come up with something that works for everywhere 18:36:42 which a large amount of this whole gitlab excercise is about 18:36:45 * catalyst thinks cypherpunks is a terrible thing and drives away people who would actually contribute 18:37:04 right, that is what i hope moderators can prevent 18:37:07 * anarcat agrees with catalyst 18:37:17 i guess that's my fundamental concern here 18:37:38 i'm concerned we'd spend a lot of effort satisfying a use case many people are fundamentally objecting to 18:37:55 many people also wants this feature 18:38:00 especially if we make signing up hard 18:38:05 * gaba thinks is important to have lower barriers for reporting issues 18:38:15 i agree with having low barriers to reporting issues 18:38:16 ahf: this feature in particular? or just some low barrier way for outsiders to engage with us? 18:38:23 that doesn't necessarily mean having a cypherpunks account :) 18:38:33 we need to clear up what this requirement is 18:38:36 catalyst: i think they want an easy way of getting tickets in 18:38:37 i think the need not to create an account is important 18:38:37 because i'm getting mixed signals 18:38:53 like not yet another account in a project just to contribute 18:39:08 GeKo: that's one reason i prefer us to engage with new people over github.com, etc 18:39:27 yeah but github has other issues 18:39:34 which make it less suited for us 18:39:47 a third-party hosted service takes care of the registration issue and the quality filtering issue to some extent 18:40:05 GeKo: which issues with github do you think are blockers here? 18:40:58 blockers for what? 18:41:09 we are about to move to gitlab 18:41:13 and not github 18:41:14 GeKo: you said github has other issues 18:41:45 github has the issue that is a hosted service that does not protect people's privacy 18:41:46 i meant use our self-hosted gitlab as an issue tracker of record, and monitor github.com (or gitlab.com even) for new people 18:41:53 non-self-hosted, some countries not able to reach it 18:42:04 oh 18:42:05 right 18:42:17 i think we can continue to do that even with this. that will be up to each project/team 18:42:27 yeah 18:42:46 i am almost sure the network team wants to continue to monitor github.com 18:42:47 i would not want to require users to create a github account 18:42:52 to contribute to our project 18:43:05 just because we don't want to deal with user registration 18:43:10 but you're ok with making them create an account on our service? 18:43:15 * gaba agrees with GeKo 18:43:27 what if they already have a github.com or gitlab.com account? 18:43:30 i like ahf's idea 18:43:33 FWIW, the salsa signup thing is a flask app, and maybe we could just start with that and extend it as needed 18:43:50 catalyst: if they already have an account that is fine. I don't think we should require them to have an account 18:43:59 exactly 18:44:02 anarcat: the features of the salsa app is a 2h thing to do from scratch in a framework i dont know 18:44:17 ahf: ah, i thought you mentioned flask, sorry 18:44:21 if we decide to do this idea it is going to be in a framework that allows us to do it in a sensible amount of time and not have to think of an upstream 18:44:29 i'm just trying to keep this small 18:44:34 i am almost sure nobody else than us wants this feature :-) 18:44:37 cerberus feels like a moonshot right now 18:44:37 yep 18:44:40 we need to solve user registration 18:44:41 mm, starting with salsa and expanding from there may give us a compromise of something short term and that can be contribute to later 18:44:49 yes, that we can do 18:44:52 that's the immediate problem, i think 18:44:54 we can do salsa registration now 18:45:03 and then move to this if/when it is done, if we want it 18:45:04 cypherpunks is another problem, that i don't think is immediate 18:45:07 maybe i'm wrong! 18:45:12 i do think it is a blocker 18:45:14 but what i hear is people want lower friction 18:45:22 does salsa allow us to explicitly approve actions by users before they become visible? 18:45:27 no 18:45:27 that doesn't need to be solved with anonymous users 18:45:28 catalyst: no 18:45:32 salsa is just fill out a form and you have access 18:45:42 no captcha or anything 18:45:51 so it's the IRC problem instead of the cypherpunks problem? 18:46:41 the irc problem being anybody can join? 18:46:42 Why can't we separate core contributors from people that just want to send a bug report? 18:46:57 how hiro? 18:47:01 and handle that via a forum like discourse 18:47:18 and when people send many bug reports or patch we invite them to dip.tp.o 18:47:51 and discourse has all the moderation we need to handle abuse and users scores and stuff 18:47:59 +1 18:48:09 +9000 even 18:48:14 but this prevents them from participating on existing issues in our bugtracker 18:48:15 hiro: so like our blog comments, except maybe better focused? 18:48:18 and I think has already an anonymous user account kind of thing 18:48:25 it would solve the "blog is hell" problem as well and would eventually allow us to get rid of a dynamic site for that :p 18:48:34 and also RT 18:48:49 boklm: correct 18:49:00 we need some compromise here 18:49:04 gitlab doesn't allow moderation 18:49:05 I like that BUT we woud have to update two places on how an issue is going 18:49:20 so unless we write a moderation plugin for gitlab, we won't get all that we want 18:49:25 nono gaba: you don't have to keep two issues 18:49:30 if people want to participate in teh bugtracker then we give them accounts 18:49:51 people will send their issues and if these are valid we add them to gitlab 18:50:01 yeah, i like your idea hiro 18:50:08 and if they want to participate we give them a -guest account 18:50:15 i dont know discource, but is it like irc or more like a forum? 18:50:22 it's like a forum 18:50:23 discourse is like a forum 18:50:32 so people reporting issues would create a new topic? 18:50:33 https://www.discourse.org/ 18:50:37 yes 18:50:40 yeah there's no inherent reason to let unknown people have write access (even restricted) to our bug tracker, especially if we're good about entering stuff that we hear about 18:50:43 can anonymous people do it? 18:50:48 ahf: https://forum.securedrop.org/ 18:50:53 that's how secure drops uses it https://forum.securedrop.org/ 18:50:56 lol ggus 18:51:04 :P 18:51:05 also python core uses I think 18:51:27 it's spreading like the plague :p 18:51:29 and you can just write there without user registration? 18:51:38 ahf: no 18:51:44 ahf: but there are nice registration options 18:51:53 like you can signup with github, gitlab, facebook, gmail, whatever 18:52:00 but yes, it's yet another signup form 18:52:16 would we run our own discourse, in this proposed model? 18:52:45 they have a hosted thing 18:52:48 i am not against it. i wonder if we are going to keep track there though? i have the feeling we sometimes miss bug reports even on irc where we hang out day and night (minus weekends) 18:52:49 arma2: i can see arguments either way 18:52:50 but yes, i guess that would be another service :/ 18:52:55 https://blog.discourse.org/2015/06/discourse-1-3-released/ 18:53:18 ahf: i think IRC tends to be too noisy for reliable issue reporting 18:53:20 we have, for a while, thought of replacing our drupal blog with a static blog content + some external commenting discussion thing like discourse 18:53:21 ^ anonymous mode 18:53:32 community team is worried about forum moderation 18:53:41 arma2: i would be happy to see that happen, and think it would solve many problems 18:53:50 ggus: does the community team worry about blog moderation? :) 18:53:50 can discourse embed in mostly-static pages like disqus does? 18:53:55 catalyst: yes 18:54:05 blog moderation is handled by the author of the blog post, no? 18:54:11 catalyst: example: https://blog.codinghorror.com/is-your-computer-stable/ 18:54:34 to not totally derail this meeting, i wonder if maybe we should get k people to write up the k proposed ideas 18:54:35 anarcat: different issue. one is support 18:54:56 (we have 5min to the hour) 18:55:01 ggus it will be the same moderators that will have to interact with cerberus 18:55:02 arma2: i think we went off the rails a while back and you provide much needed re-railing ;) 18:55:13 arma2: i'm trying to add this to the notes 18:56:03 i especially want us to make sure that the proposed options actually work the way one of us is guessing they do, so we don't say "oh wow option 3 does everything we want, it's decided, we'll never think of it again" and then later we learn that option 3 doesn't work that way 18:56:10 mm, not sure how we should move forward with deciding which way to go with this 18:56:32 arma2: yeah, that was my concern with the proposal i had too. the idea there was the people who wants the anonymous user also wants to help moderating it 18:56:54 ahf: i'm really glad you took the time to document clearly your proposal, it's very useful! 18:57:11 anarcat: np! sorry for dropping it like that in here. i had completely forgotten to send it out before 18:57:36 * catalyst thinks we chronically underestimate the effort it takes to do useful moderation 18:57:42 About the other stuff we had in the agenda, it seems things are resolved. IRC ticket number and redirection will work as we want, right? 18:57:43 arma2: i think the requirements are not clear either, at least to me. i'm a bit confused as to how strong the "cypherpunks" requirement is or whether it's more a reflection of the desire to have registration-less bug reporting or at least "less friction to collaborate on the bugtracker" 18:57:54 catalyst: agreed 18:57:57 one angle worth trying to capture is how smooth the transition is from "external thing" to "is now using new gitlab account". like, if it's not smooth, we will end up with two bug trackers, one used by internal people and one used by everybody else. 18:58:36 ok. Let's discuss next steps for the next two weeks to be able to make a decision. 18:59:21 i read that as "let's discourse" :p 18:59:24 * anarcat totally biased 18:59:34 friday goal for me is to do a trial run of a full migration of tickets 18:59:56 and i think next friday i will look into irc bot integration. i wonder where the code that runs bugs.torproject.org is or is it just a redirect? 19:00:16 mike perry had an interesting use case with some keywords i need to come up with some idea for that i have no clue how to fix, but is related to bugs.torproject.org 19:00:36 bugs.tpo is just a redirect, iirc 19:00:39 ahf: i am close to knowing that answer re bugs.tpo. i'll hunt it down and tell you. 19:00:43 Does anybody have some time in the next two weeks to help ahf with any of those tasks on Friday? 19:00:46 thanks arma2 19:00:51 ok 19:00:56 i could spare some cycles 19:01:03 because i want this to liiiiiive 19:01:07 :) 19:01:08 same 19:01:13 the migration stuff i think i can do alone but i need some people soon to look at random samples of tickets on gitlab 19:01:20 as long as we do it my way of course 19:01:20 and see if something isnt being migrated right 19:01:21 ;) 19:01:25 anarcat: :-D 19:01:45 about discourse, any volunteer to write down how this would work? 19:02:00 yeah, that would be really nice! i have no clue about that 19:02:03 hiro, want to do that^ ? 19:02:06 i could help 19:02:20 i'm registered on like a dozen of those frigging things so i have some experience with it (although never as an admin) 19:02:43 ok. I can help with that too. 19:03:26 very cool! 19:03:43 i love at these meetings that we have so many people from so many different teams <3 19:03:44 * catalyst can try to get some info on how various marginalized people experience discourse.org forum moderation? 19:03:45 about plan for migration (taht we mentioned in the last meeting) I started drafting something at the end of the blessed trac to gitlab document 19:03:52 i'm tempted to just spin up a docker image of discourse so people get a feel of it 19:03:52 thanks catalyst 19:04:00 catalyst: that would be awesome 19:04:24 i have yet to hear that about discourse - i heard a lot about mastodon and other platforms, naturally, but not specifically discourse 19:04:41 anything else for today? 19:04:44 discourse.org was made by the same folks who did stackexchange.com, for what it's worth 19:04:49 gaba: coffee? :) 19:04:54 hehe 19:05:00 anarcat: i'm not sure that's a strong recommendation :-/ 19:05:28 next meeting will be in 2 weeks. si se puede! :) 19:05:31 network team people have a meeting in 55 min. and i think everybody is eager to get away from irc for a bit :-P 19:05:34 thanks all o/ 19:05:39 thanks! 19:05:42 #endmeeting