18:29:44 <TheSnide> #startmeeting
18:29:44 <MeetBot> Meeting started Wed Sep 26 18:29:44 2018 UTC.  The chair is TheSnide. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:29:44 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
18:29:52 <TheSnide> Hi all ;-)
18:29:59 <Kvisle> !
18:30:03 <TheSnide> Let's wait for ppl to arrive gently
18:31:37 <TheSnide> #topic warmup
18:31:59 <TheSnide> Do someone has any topic to be added to the agenda ?
18:33:30 * h01ger just waves and expresses his hope to *soon* get to upload sumpfralle's work to debian
18:36:51 <TheSnide> h01ger: finger crossedç
18:36:53 <TheSnide> h01ger: finger crossed!
18:36:59 <h01ger> :)
18:38:57 <TheSnide> #topic async
18:40:07 <TheSnide> I'm currently looking closely at the munin-async(d) pair. And, while the master<->async protocol is quite clever, the async-asyncd is pretty horrible ;)
18:41:03 * TheSnide pleads guilty on that one... it started more an hugly hack done while waiting at an airport
18:41:27 <TheSnide> ... and that hack got shipped :)
18:42:18 <TheSnide> Actually, it's not that bad, but it currently doesn't offer much protection against async/asyncd concurrent execution. And this results in partial data delivered
18:43:55 <TheSnide> ... which is quite bad IMHO. Best workaround for now is the one found by bcg : "delay the asyncd collecting by a certain offset". This is to artificially avoid collision, or at least make it unlikely.
18:44:38 <TheSnide> So, I'm thinking about rewriting the whole async/asyncd spooling protocol, to make it much more robust
18:44:58 <TheSnide> I basically have 3 options:
18:45:37 <TheSnide> 1. add locks, and add enough metadata to make it work (this is already working in a PR)
18:47:05 <TheSnide> 2. change the spooldir format completely. Either using many files (1 per plugin per run), or a SQLite DB that handle atomicity & concurrency
18:47:25 <TheSnide> 3. do 2. but also rewrite it in C via the munin-c project.
18:48:42 <TheSnide> ... I'm currently leaning towards merging 1. into master, and 2+multiple files. As it should not be that a huge deal to handle less than 1k files, even for anemic nodes.
18:49:41 <TheSnide> I don't really want to rely on sqlite as it will be a pain to install, and we won't be able to target anemic nodes.
18:50:07 <TheSnide> what do the audience think ?
18:52:08 <bcg> is it one file per plugin or one per value per plugin?
18:52:13 <kjetilho> if locking entails creating/deleting a file for each plugin per run, there isn't that much to gain from doing that
18:52:22 <kjetilho> resource wise
18:52:28 <TheSnide> locking is done via flock()
18:52:40 <kjetilho> ok, good :)
18:52:44 <TheSnide> on existing files.
18:52:54 * TheSnide doesn't care about NFS :D
18:53:10 <TheSnide> (for async/asyncd IPC that is)
18:53:15 <kjetilho> people still using NFS v2 can stay in the 1990s :)
18:53:30 <TheSnide> bcg: no, i'm planning 1 file per plugin per run.
18:53:33 <TheSnide> like :
18:53:36 <bcg> If I have 60 plugins, it will create 17k files per day?
18:53:41 <TheSnide> yes
18:54:25 <bcg> it might be best to use subdirectory per plugin then.
18:54:26 <TheSnide> hmmm ... 60 * (24 * 3600 / 300)
18:55:24 <bcg> on top level 60 directories, 288 files on each (per day)
18:55:40 <TheSnide> something like spool/$timestamp/$plugin.txt
18:56:05 <TheSnide> as, it would enable the asyncd to know directly which directory to scan & emit
18:56:14 <TheSnide> async
18:56:31 <bcg> or spool/$plugin/$timestamp ? ... ok, good point, spool/$timestamp/$plugin
18:56:39 <TheSnide> spool/${timestamp}/${plugin}.txt
18:57:04 <TheSnide> yeah, i thought about a plugin dir, but then I have to scout all those subdirs
18:57:37 <TheSnide> a timestamp dir would work even with completely loose asyncd process
18:57:44 <bcg> spool/time/plugin is good
18:58:09 <TheSnide> yep. because a design that is nice from asyncd is :
18:58:50 <TheSnide> if you specific "--fork" then asyncd will fork() just prior to fetching a plugin infos
18:59:17 <TheSnide> In order for every plugin to be fully isolated from the other.
18:59:41 <TheSnide> you'll have a parent asynd, that does the timekeeping thing, and fork().
19:00:30 <TheSnide> and then the childs that do just "connect to node, issue a config $plugin and a fetch $plugin if needed, die"
19:01:29 <TheSnide> This was done to:
19:01:33 <TheSnide> 1. keep it very simple
19:01:51 <TheSnide> 2. be able to merge the async feature directly into the node in the future
19:02:46 <bcg> back to options: do 1 + 2 (spool/$timestamp/$plugin) now, 3 after 3.0 is released.
19:07:02 <TheSnide> yup
19:07:53 <TheSnide> #idea merge the lock PR & hack spool/$timestamp/$plugin for 2.999.
19:08:04 <TheSnide> #topic misc
19:08:19 <TheSnide> I ran out of topics. Q&A ?
19:08:23 <TheSnide> ahh..
19:08:33 <TheSnide> #topic coding style
19:08:53 <TheSnide> This is as usual a controversial topic ;)
19:09:50 <TheSnide> I'll have to look into what is currently done in Munin and merge some perlcritic & perltidy prior to the next meeting.
19:09:56 <h01ger> just use any automatic code formatter and be done with it
19:10:13 <TheSnide> h01ger: my point.
19:10:22 <kjetilho> there already is a .perltidyrc in project
19:10:34 <kjetilho> my suggestion was to add it to munin-contrib as well
19:11:04 <kjetilho> but of course, it is a good idea to see if it matches current coding standard :)
19:11:14 <TheSnide> #action TheSnide checks perlcritic & perltidy in munin.git & pushs them to contrib.git also
19:11:21 <TheSnide> ;)
19:11:23 <h01ger> TheSnide: glad we agree
19:15:38 <TheSnide> #endmeeting