17:59:30 #startmeeting DebConf Video Team 2020-06-25 17:59:30 Meeting started Thu Jun 25 17:59:30 2020 UTC. The chair is tumbleweed. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:59:30 Useful Commands: #action #agreed #help #info #idea #link #topic. 17:59:36 #link http://deb.li/oNCD Agenda 17:59:41 #topic Roll Call 17:59:46 Say hi! 17:59:57 hi! 18:00:18 is it just the two of us? :) 18:01:00 I guess so 18:01:07 #topic Actions from previous meeting 18:01:12 #link http://meetbot.debian.net/debconf-video/2020/debconf-video.2020-06-16-17.59.html minutes 18:01:23 pollo: you had 2 actions 18:01:30 I haven't had time to do either 18:01:32 olasd, RattusRattus? 18:02:05 I haven't seen progress on either of those items 18:02:20 #topic Tracking progress in Salsa 18:02:27 #link https://salsa.debian.org/debconf-team/public/data/dc20-online/-/issues 18:02:33 Nothing there yet 18:02:49 Should we start by turning those action items into issues there? 18:02:58 I think we should 18:03:10 not that I volunteer to :) 18:03:20 #action tumbleweed to create issues for the 4 pending action items 18:03:21 pff 18:03:26 sorry, I was distracted 18:03:41 haai :) 18:04:01 olasd: anything to say about latency, or moving on? 18:04:09 no progress 18:04:42 #topic video stack 18:04:42 I doubt I'll really have time before we sprint, tbh 18:05:00 are we having a sprint? 18:05:10 I'm confused between videostack and streaming setup 18:05:20 they are closely tied to each other 18:05:42 tumbleweed: AGREED: online sprint during early July (paddatrapper, 18:18:03) 18:05:54 We need to set actual dates 18:06:11 ok, that's later in the agenda 18:06:23 anyway, back on topic, we did talk about OBS a little since our last meeting 18:06:38 I wasn't really following that discussion closely though 18:06:49 I don't think any conclusions were reached 18:06:52 we need to experiment 18:07:14 Yeah. There were some notes on playing recorded media 18:07:20 o/ 18:07:27 vocto has been finicky with sources coming and going, so we really need to test that 18:07:35 Essentially vocto isn't really designed to do it 18:07:49 I'd be especially concerned with audio in that scenario 18:08:00 (I mean, keeping it in sync) 18:08:34 I hope this is on topic, 18:08:50 OK, doubt there's much more to say on this point in the meeting. We really need to experiment 18:09:07 but I was thinking that the jitsi instances that we'll set up could be at $roomname.debconf.org and set up specifically for debconf 18:09:35 pollo said something like he does have an ansible role that we can recycle 18:09:36 highvoltage: if there are more than 1 track 18:09:36 you think one jitsi instance wouldn't cut it? 18:09:55 highvoltage: nope, I have a puppet module :P 18:10:03 pollo: aaaah, ok 18:10:14 Would need a server per Jitsi instance then 18:10:20 I guess if there's just one track then jitsi.debian.social is fine and we don't have to do much jitsi work then 18:10:36 anyone know what the talk count is? 18:10:50 (oh wait there's that handy stats page that tumbleweed usually reminds us of right about now) 18:10:52 https://debconf20.debconf.org/talks/statistics/ 18:10:56 there's stats 18:10:57 highvoltage: let me ask again 18:11:06 you think we need more than one jitsi? 18:11:16 isn't one easier? 18:11:43 tumbleweed: if we have more than one track, then it's easier yes, although if the streaming script is made a bit more clever then it might also be ok even with 2 tracks 18:12:02 ah 23 talks submitted so far 18:12:08 ok I think the one jitsi is fine 18:12:16 highvoltage: I'm not seeing what makes it easier? 18:12:37 I assume a lot of submissions will come in at the last minute (or later) 18:12:51 paddatrapper: well for one, overload in one session wouldn't hurt other sessions on other jitsis (that was my initial thought) 18:13:18 How close have we come to overloading the Jitsi instance? 18:13:20 redundancy is certainly useful 18:13:32 I agree redundancy would be good 18:13:47 paddatrapper: and re: the streaming script if it worked like last time we wouldn't have to add much to the script (to know where to stream through, but I suppose that's trivial enough to fix too for next time) 18:14:05 Yeah that is trivial to base on the entered key 18:14:16 paddatrapper: last time CPU use were about 50-65% for a general ballpark 18:14:26 Hmm ok 18:14:30 paddatrapper: I think more of that was for jibri than jitsi itself 18:14:41 Yeah sounds about right 18:14:56 We will need other servers with jibri running anyway 18:15:03 I think if we want multiple jibris per jitsi (omg) then we would need a much beefier machine 18:15:14 would redundancy really help? If our jitsi server goes down, I feel trying to fix it will take less time than trying to switch everything (and everyone) to another instance 18:15:26 pollo: I tend to agree 18:15:37 less infra to maintain means we can fix things faster 18:15:55 Jibri will require running on additional servers from what I've seen. Containerising doesn't play nicely, nor does VMs 18:16:04 I thought it helps in the sense that if one track goes down, it doesn't affect other sessions, but I'm not married to the idea and have no strong feelings about it 18:16:14 tumbleweed: real replication would be great, but it's shit ton of work 18:16:21 putting jirbis on separate machines sounds sensible 18:16:32 just having a spare jitsi instance doesn't seem worth it imo 18:16:38 I'm happy to get them set up and scripted 18:16:48 pollo: yeah no one suggested that 18:17:01 Let's move on 18:17:15 #topic Streaming Setup 18:17:22 Anything to discuss here? 18:17:34 I suspect this is dependant on the investigation olasd was going to do 18:18:34 I'll take that as nothing to discuss 18:18:48 #topic Advice/training for directors/talkmeisters 18:18:56 *nod* 18:19:12 This would depend on RattusRattus' docs I guess 18:19:25 is anyone else helping with that? 18:19:28 I'd say his docs are more about the next one 18:19:48 pollo: nattie and I 18:19:55 hmm, it's hard to pin down anything yet since we are still flying blind though 18:19:58 #link https://storm.debian.net/shared/iVvMB3RMXFtTen_jbtZ_FIh0-Ob54DaYvXvxaZx7_Ls the note sI started 18:20:55 So, who is responsible for director and talkmeister training? 18:20:58 in slide design, it might be worth while saying that 16:9 slides are best 18:21:15 (it does mention resolution, but I noticed at mdco that some people had 4:3 slides) 18:21:21 highvoltage: added 18:21:25 I'm happy to deal with it once we have a firmer idea of what our stack is 18:21:31 please add to this yourself, too 18:22:00 #agreed paddatrapper can do training once the stack is finalized 18:22:15 #topic Advice/training for presenters 18:22:17 cool 18:22:22 (this is really where that #link should have gone) 18:22:51 This is not dependant on stack 18:22:58 we should be getting this done ASAP 18:23:27 Maybe a few of us can spend 10 mins on it after the meeting? 18:23:41 I'm in 18:23:47 Cool 18:23:56 #topic Questions wrt to language tracks 18:24:06 We were asked: What are the minimal requirements for people who want to organize a language track 18:24:27 I think the answer given at that point (1 director + 1 talkmeister) was about right? 18:24:30 My thoughts: directors, video reviewers, talk meisters 18:24:37 they're still submitting via wafer right? or is it more independently organised? 18:24:42 are language tracks completely separate from the "regular" talks? 18:24:54 as in, they can be in parallel ? 18:24:55 Yeah, I think 3 is really the minimum. All need decent Internet 18:24:57 highvoltage: yes, a track is created for them 18:25:18 so... we would use more than one jitsi room at a time? 18:25:24 pollo: they can be. Or they could be scheduled in a convenient timethat doesn't overlap with english events 18:25:29 or will they kind of interlock and not overlap time-wise? 18:25:56 I suspect we can make the schedule have a single concurrent event at all times 18:26:03 I've said it before, but depending on the location of those people, moving the jitsi server closer might not be a bad idea 18:26:04 s/can/will be able to/ 18:26:04 (getting jibri off of the jitsi host will probable solve my performance concerns will talk to paddatrapper again later) 18:26:17 olasd: yeah, if we want to put our foot down on that, we certainly can 18:26:24 highvoltage: yeah, can split that out, no problem 18:27:10 highvoltage: I think this is going to come down to the level of interest in the event 18:27:32 I'd hope that, like any debconf, lots of ad-hoc discussions happen in jitsi during conference events. But those don't need streaming 18:27:45 depending on the timezone, I think it would be a good idea to have someone from the videoteam "on call" 18:27:53 yeah 18:27:57 tumbleweed: we can hope for the best and plan for the worst, and shape our plans as we get closer to the event and have more data 18:28:05 yep 18:28:38 as guidelines, I'd say we can manage 2 concurrent tracks during Europe/NA daytime, but not in other timezones 18:28:53 #agreed language tracks should have at least 1 video director, 1 video reviewer, 1 talk meister. They also need a core video team person to be on call in their timezone 18:29:03 pollo: sounds reasonable 18:29:28 #agreed the video team thinks we may be able to support 2 concurrent tracks during EU/NA daytime, but probably not in other timezones 18:29:46 I'll forward that bit on to content team 18:30:02 #topic Test Setup 18:30:21 What infra are we wanting to use for the conference? 18:30:35 highvoltage: you were speaking about reorg of the jitsi.debian.social instance 18:30:36 heh, is there a pad for that? 18:30:47 but I think we'll need quite a few VMs 18:30:51 I've got a good ping to the current Hetzner server 18:31:00 yeah, I think we need a list of components and spec them 18:31:01 so probably we should be doing the whole thing in a cloud? 18:31:05 so I think staying in that datacenter would be great 18:31:08 (to give some flexibility) 18:31:12 pollo: yeah a good portion of the world has reasonable pings to there 18:31:24 sometimes their are DDoSed though 18:31:31 there are days when hetzner sucks 18:31:38 highvoltage: is the current jitsi.debian.org server a baremetal hypervisor? 18:31:41 tumbleweed: I spoke about a bunch of different things so not sure *exactly* whare you mean, but for now getting jibri off is at least the initial plan 18:31:49 pollo: it's a hetzner cloud instance 18:32:03 my bouncer has 280 us ping to that server :P 18:32:15 (yes, same dc) 18:32:26 heh 18:32:39 pollo: that's basically a VPS with 8 ryzen cores and 16GB RAM 18:32:40 tumbleweed: haven't seen much of that in the last few years of using them; maybe I've been lucky? 18:32:44 paddatrapper: you said something about jibri needing baremetal? 18:33:00 pollo: not me 18:33:11 I doubt anything *needs* baremetal 18:33:12 pollo: not bare metal, but with a public IP 18:33:21 I also wondered about that since it's not running on baremetal now 18:33:39 ah, yeah, we'll do more VPS's with their own IPs for jibri that's fine 18:33:50 I think it would be great if we could get an EPYC server and manage KVM ourselves 18:33:50 So running our own hypervisor causes issues that I really don't want to have to try fix 18:33:57 heh 18:34:35 #link https://storm.debian.net/shared/9x6jml4j5QW_qyJWtJs_ogIRKjcwirMZQX0ZXfdByPF pad to write up infra requirements 18:34:37 pollo: that's what we do on the debian.social box, but there are cost implications if you want additional IPs, etc, so for small things, especially short lived VMs, a short lived vps instance is ok 18:34:37 But this is purely for jibri and Jitsi, the rest of the stack could be on our own hypervisor 18:35:05 pollo: longer term we will likely get an openstack instance for debian with lots of IPs and space for lots of VMs then we can run this kind of thing there 18:35:26 I have no experience with openstack though... 18:35:32 me neither 18:35:33 it's just like any other cloud 18:35:53 anyway, we'll need to flesh out the specs for sure, and there is a pad 18:35:57 what further do we need to cover in the stack? we can get people together in jitsi, capture it with jibri, stream to the debconf CDN... 18:36:07 we just need to imrpove some glue afaict 18:36:11 paddatrapper: I'd tend to agree. hypervisors are not our concern 18:36:16 .oO(Debian had an openstack instance that noone ever used) 18:36:39 olasd: I wonder if it was set up by the same person who wants to do it now (not impossible) 18:36:52 highvoltage: stick OBS or similar in the middle and yeah, I reckon that's about it 18:37:27 highvoltage: so, is hetzer cloud an option here? 18:37:38 could multiple people in the debconf video team get access to this account? 18:37:42 tumbleweed: yes, certainly 18:37:51 (or do we need a new one?) 18:38:10 tumbleweed: I can get philhug to create a debconf video team group, add us, and then we can create/destroy VMs as we please 18:38:49 otherwise aws / some other thing is an option 18:38:51 if you want, you can send the DPL an updated delegation that includes some budget for videoteam to create VMs as needed 18:38:56 and zigo was also volunteering infra, last week 18:39:05 then the videoteam doesn't have to ask for approval to do that again 18:39:29 there isn't a debconf20 budget yet, but I'd expect one to be in place before the event 18:39:35 could also do a combination of things and try it out 18:39:39 I guess the problem is we should be experimenting now, not then 18:40:14 what budget should we request? $1k? 18:40:17 tumbleweed: if the global latency to the Infomaniak datacenter is similar, I don't mind, but we have to keep that in mind 18:40:38 I'd expect it to be similar 18:41:51 tumbleweed: sounds reasonable 18:42:12 worst scenario we can ask for a budget update 18:42:33 but I agree it'd be nice to have machines to play with, I'm tired of stealing $work ressources 18:42:46 #agreed video team to ask DPL for a $1k budget to allow experimentation before debconf20 has a budget 18:42:48 then let's consider this a high priority 18:43:12 #topic Any Other Business 18:43:45 I've never used DO droplets in a team before, can you also set up teams / billing in a way that debian.ch could pay for it and video team could create those as needed? 18:43:57 How keen are we into putting everything into ansible? 18:44:11 highvoltage: yes, and we have an account set up like that 18:44:12 I somehow doubt we'll be able to do it in time 18:44:16 (except DLange pays, I think) 18:44:31 pollo: Probably a good idea for reproducibility though 18:44:36 tumbleweed: maybe we can also consolidate all of that so that the TO directly pays 18:44:37 pollo: would be nice, but not critical 18:44:43 tumbleweed: SPI pays 18:44:55 paddatrapper: I agree it would be great, but I'm not sure it's realistic 18:44:56 this is just a team I set up at the time 18:45:24 i.e. you can empty SPI with the DO instances but not my company, separate team 18:45:33 heh 18:45:59 pollo: at least parts can be then. I can do jibri. The CDN stuff is already there. It is missing jitsi and OBS 18:46:26 at least jitsi isn't that hard. it's a ppa and a few packages and a few config files 18:46:40 hi all. We gave up to organize a portuguese track on DC20 18:46:53 phls: probably a content issue at this point 18:47:01 mleh, not my experience... If we go with ansible for jitsi, let's please consider looking at the Ansible Galaxy first 18:48:16 I guess that isn't any other business then 18:48:25 #topic Upcoming Meetings 18:48:30 IRC next week? 18:48:31 pollo: ah, was easier for me, did you use Debian or some other shitty distro :p 18:48:46 tumbleweed: wfm 18:48:50 +1 18:48:57 #agreed IRC meeting same time next week 18:48:59 online sprint 18:49:04 We said early July. When? 18:49:11 highvoltage: well, I did things "right" in puppet, and that means managing prosody correctly in a separate module, etc. 18:49:12 early july is this time next week :P 18:49:33 After 5 July (state of the Map) 18:49:37 tumbleweed: I think we said after SOTM 18:49:51 So, weekend of 11th? 18:50:31 fri-sat-sun ? 18:50:37 5th is when the content team will start putting together the schedule 18:51:01 tumbleweed: cool. I won't be around the Sunday, but can do the rest 18:51:11 pollo: WFM 18:51:27 olasd, highvoltage? 18:51:34 +1 18:51:37 yeah, that probably works 18:51:56 #agreed video team to sprint on infrastructure 10-12 July 18:52:10 Test runs 18:52:14 presumably those come after this 18:52:19 I guess we're going to want to work on EU time as much as possible, I'll try to shift my schedule to get up earlier 18:52:29 we'll probably get speakers wanting to test around that time, too 18:53:07 pollo: yeah, I can try that too 18:53:17 the weekend after for a test run? 18:53:24 pollo: +1 18:53:45 wfm 18:54:07 #agreed can tentatively aim for some video test runs on the weekend of 19-19 July 18:54:12 fwiw I'm available most of the weekend to try stuff too (and probably will regardless) 18:54:34 last call for AoB? 18:54:54 #endmeeting