13:02:19 <hiro> #startmeeting network-health 2025-09-29
13:02:20 <MeetBot> Meeting started Mon Sep 29 13:02:19 2025 UTC.  The chair is hiro. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:02:20 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
13:02:30 <hiro> #link https://pad.riseup.net/p/tor-nethealthteam-2025-keep
13:02:42 <hiro> xD
13:02:43 <GeKo> we got our dates wrong, no?
13:02:50 <GeKo> on the pad i mean
13:02:52 <juga> +1
13:03:02 <GeKo> *scratches head*
13:03:19 <hiro> I think I wanted to update the next meeting date
13:03:22 <GeKo> i don't believe i am ready for oct 6 yet :)
13:03:26 <hiro> and my brain switched to october
13:03:46 <juga> sorry maybe i confussed the pad adding the date myself earlier
13:04:16 <hiro> sorry sorry
13:04:24 <hiro> :)
13:04:35 <juga> no worries :)
13:05:49 <hiro> oook
13:06:31 <hiro> I am starting data ingestion in the new clickhouse on metricsdb-03 and I hope to coordinate everything with @sarthik to make it work together with the statuses creation
13:07:01 <juga> i have a question about clickhouse
13:07:06 <hiro> go ahead
13:07:14 <juga> if everything is migrated to clickhouse, would i be able to do postgresql queries via clickhouse interface?
13:07:22 <juga> (interface for postgres)
13:07:45 <hiro> I think so to a certain extent, sarthik will know more for sure
13:08:12 <sarthikg[mds]> from what I have seen, clickhouse tries to imitate the postgres interface, so basic queries work fine, but anything which is complex, doesn't work
13:08:27 <juga> i see, hmm
13:09:00 <juga> and the plan with metrics-sql-tables repo is also to migrate all to clickhouse syntax?
13:09:11 <hiro> but also @juga I think you will be able to do less complex queries, if the ingestion works fine we can go through the data together on thursday
13:09:40 <juga> sounds good hiro
13:09:48 <sarthikg[mds]> datamodel will ideally replace the metrics-sql-tables repository. in datamodel, we will have separate structs for postgres & clickhouse
13:10:11 <juga> sarthikg[mds]: ah, that's awesome, that there'll be 2 different structs
13:10:45 <juga> ok, i got my questions answered, at least for now ;)
13:11:41 <juga> last week on my side, just more work on sbws and same for this week, until releasing 2.2...
13:11:51 <sarthikg[mds]> i also think it might also be reasonable for us to temporarily pull data in postgres from clickhouse for analysis. though i'm not sure how that sounds
13:12:10 <hiro> I think clickhouse is supposed to be faster
13:12:11 <GeKo> why would  we need that
13:12:13 <hiro> for analysis
13:12:15 <GeKo> yeah
13:12:31 <hiro> especially since I am trying to have the timeseries there too
13:12:52 <juga> maybe we can pull data from clickhouse to postgres when we have concrete experiments
13:13:04 <juga> but not regularly
13:13:15 <GeKo> but why would we need that? :)
13:13:30 <sarthikg[mds]> ohh ohk, i just thought that since complex queries might work better with postgres. anyways, clickhouse should work fine
13:14:01 <juga> GeKo: i don't know right now :), maybe if some query doesn't work well in clickhouse, we'll see
13:14:06 <hiro> complex queries work with clickhouse too, it just has a slightly different syntax
13:14:56 <juga> ok
13:14:58 <GeKo> to be honest i have some hope that we don't need to deal with postgres AND clickhourse
13:15:03 <GeKo> *hose
13:15:05 <GeKo> hah
13:15:08 <GeKo> *house
13:15:09 <juga> horse :P
13:15:13 <sarthikg[mds]> lmao
13:15:18 <GeKo> yeah, that one was missing
13:15:55 <GeKo> anyway, i don't like the idea of suddenly having to deal with both of them
13:16:23 <hiro> @geko my idea is that all the analysis can be done in clickhouse
13:16:23 <juga> tbh, i feel lazy to learn clickhouse syntax, but i'll do it, shouldn't be too hard ;)
13:16:51 <hiro> and there might be some app data that will live in postgres for a bit at least (like tagtor or api)
13:17:26 <GeKo> fair
13:17:28 <hiro> but let's see
13:17:35 <GeKo> okay
13:17:53 <GeKo> i did p112 work last week, the usual bad-relay stuff and p183 (analysis#80)
13:17:53 <tor> Uhm, which one of [tpo/network-health/analysis, tpo/network-health/metrics/analysis] did you mean?
13:18:12 <GeKo> no metrics, tor
13:18:40 <sarthikg[mds]> my status update: i am trying to modify the aggregator to be able to read byte-stream & json columns from clickhouse directly, and compare the performance improvements in this process
13:18:42 <GeKo> this week i'll get back to anomaly research, will continue p112 stuff and then be at the tor community gathering
13:18:46 <GeKo> that's it
13:19:00 <GeKo> *anomaly detection
13:19:22 <hiro> last week there were some dns errors but I sent an email about the net-health meeting have you received it, otherwise I'll send again
13:19:40 <hiro> feel free to add ideas in the pad, I won't post it because there are some info there that we might not want to share
13:19:44 <juga> i received it, thanks
13:20:04 <juga> oh, community gathering, i forgot about it
13:21:06 <GeKo> got it
13:21:15 <hiro> ok, that's all from my side, if we are good we can all go back to what we were doing
13:21:46 <sarthikg[mds]> i'm good!
13:21:57 <juga> me too
13:22:55 <GeKo> me as well
13:23:22 <hiro> oook!
13:23:25 <hiro> #endmeeting