13:01:06 <hiro> #startmeeting network-health 2025-08-11
13:01:06 <MeetBot> Meeting started Mon Aug 11 13:01:06 2025 UTC.  The chair is hiro. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:01:06 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
13:01:14 <sarthikg[mds]> o/
13:01:51 <hiro> ok, I do not have much, fixed access to clickcloud so you can test. This will will try to finish the code to extract onion services stats
13:02:00 <hiro> please do let me know if you need me for anything
13:04:27 <juga> also not many news, continuing with collector and small things
13:04:44 <hiro> thanks juga
13:05:06 <juga> np
13:06:30 <M4xy0m[mds]> I think I can expect the parser to work by the end of the week
13:07:33 <hiro> oh wow 4xy0m great news let me know if you want us to test anything
13:09:03 <sarthikg[mds]> there's little progress on data-model side. we have tag-release pipeline (with permissions), publishing to crates.io. the sea-orm compatibility with clickhouse doesn
13:09:15 <sarthikg[mds]> * clickhouse doesn't look good in the initial testing
13:09:58 <hiro> sarthikg: we can try to draft a plan tomorrow w reg/ to move data from and to clickhouse and psql
13:10:03 <sarthikg[mds]> [EDITED] just incase someone did not see the edited message (bcoz matrix):
13:10:03 <sarthikg[mds]> there's little progress on data-model side. we have tag-release pipeline (with permissions), publishing to crates.io. the sea-orm compatibility with clickhouse doesn't look good in the initial testing
13:10:08 <M4xy0m[mds]> hiro: I gonna rewrite tor fusion using the library to see if everything works good and it would be nice if you could review the doc
13:10:30 <hiro> sure thing 4xy0m !
13:10:43 <sarthikg[mds]> sure hiro!
13:11:42 <hiro> 4xy0m: tor_fusion parses mostly csv files and onionperf data though, what is your plan with it?
13:11:58 <hiro> parse the data from descriptors instead of csvs instead/
13:11:59 <hiro> ?
13:12:33 <M4xy0m[mds]> Yeah that's it
13:12:55 <hiro> ah nice! looking forward to it!
13:13:01 <M4xy0m[mds]> Just, why did the json is converted into a csv ?
13:13:10 <M4xy0m[mds]> That's for data analysis with polar ?
13:14:04 <hiro> the csv data comes from the DB, since it was "expensive" to parse the descriptors into memory in java and we didn't have the rust library I was using the csv in order to load them into memory with polar
13:14:22 <hiro> not doing much at the moment with the json that comes from onionperf
13:14:31 <hiro> but I think we might drop some of the data we store there
13:14:39 <hiro> we have a ticket that we have been working with juga
13:14:42 <hiro> hold onm
13:15:15 <hiro> https://gitlab.torproject.org/tpo/network-health/metrics/tor_fusion/-/issues/10
13:15:23 <hiro> I think we are saving too much
13:16:06 <M4xy0m[mds]> Let me know if you plan to change the datamodel
13:16:51 <hiro> yeah for onionperf for sure! but have a look at the other module instead... https://gitlab.torproject.org/tpo/network-health/metrics/tor_fusion/-/tree/main/src/stats?ref_type=heads
13:17:18 <hiro> here I think you could try to have the same data from those csv by parsing the documents directly and then producing the aggregates
13:18:56 <M4xy0m[mds]> Ok I see
13:20:00 <M4xy0m[mds]> I think that saving into a csv isn't a bad idea since we cannot really choose which data we want so I gonna do this
13:20:33 <hiro> sounds good! let me know if you want to try it out or you'd rather like to have me scoping you an issue so you can work with some guidelines
13:20:47 <hiro> the general gist is that you parse the document for one day and then you create the net aggregates
13:20:54 <hiro> *documents
13:22:02 <M4xy0m[mds]> That's good for me
13:23:36 <hiro> you prefer a scoped issue or you want to hack it out a bit?
13:24:33 <M4xy0m[mds]> A scoped issue would be nice
13:24:35 <M4xy0m[mds]> Thank you
13:24:43 <hiro> ah ok perfect! I'll get to do that
13:24:50 <hiro> I'll open it out in tor-fusion then
13:25:38 <hiro> ooook everyone! if we are all groot! we can go back to talk async
13:25:43 * hiro is groot!
13:25:50 * juga is good
13:25:52 <M4xy0m[mds]> * 4xy0m is good
13:26:33 <hiro> I AM GROOT🌲lol
13:26:36 <hiro> #endmeeting