13:01:44 <hiro> #startmeeting network-health 2025-03-10
13:01:44 <MeetBot> Meeting started Mon Mar 10 13:01:44 2025 UTC.  The chair is hiro. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:01:44 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
13:01:56 <hiro> and the pad: https://pad.riseup.net/p/tor-nethealthteam-2025-keep
13:03:34 <hiro> my update for this week is that I have started testing object storage and loading and offloading of data to and from the db
13:04:01 <juga> \o/
13:04:12 <hiro> I am also thinking I could move the statuses and family creations from object storage as well and just use the parser for actual documents
13:04:39 <juga> interesting
13:05:04 <hiro> yeah I thought maybe we can save of the queries that we run
13:05:46 <hiro> and the ingestion could be a lot faster this way
13:06:05 <hiro> anyways that's all for me
13:06:29 <GeKo> nothing from my side besides what is already on the pad
13:06:39 <hiro> thanks @GeKo
13:07:07 <juga> i managed to implement some query parameters for the /details endpoint and will continue with that this week
13:07:22 <hiro> \o/ thanks @juga
13:07:44 <juga> yw
13:07:54 <hiro> sarthikg[m]: want to go next?
13:08:27 <sarthikg[m]> i got the /router/details route working with most of the information. just finishing that one up.. will continue working on vm-based routes after this
13:08:40 <hiro> @GeKo btw I see you will be starting to test the imported data. I have processed till October 21 2024
13:08:51 <hiro> from june 2024
13:09:15 <hiro> and also from Jan till feb 20 something... I have some gaps that I need to fill in the first part of the year
13:09:21 <hiro> thanks @sarthikg[m]
13:10:16 <GeKo> hiro: yeah, i'll test as i am going. so, no rush
13:10:24 <hiro> @GeKo @juga another thing that I think we might think about is if we want to keep all the data in the db (given that the object storage mechanism works fine)
13:10:36 <GeKo> so far, nothing to complain about :)
13:10:39 <hiro> what I mean is: for example there is one table that we do not need that often (like votes)
13:11:05 <GeKo> hrm
13:11:23 <GeKo> so, you mean we could keep other stuff longer instead?
13:11:27 <hiro> yeah
13:12:02 <GeKo> i'll think about that
13:12:02 <juga> i'm not sure which data to leave, maybe we can think about it together next days
13:12:07 <GeKo> yeah
13:12:11 <juga> s/data/tables/
13:12:23 <GeKo> i am a bit worried about the added complexity to that scheme
13:12:58 <GeKo> which we then would need to document given that we want to provide db dumps at some point for external folks
13:13:04 <GeKo> assuming that's still the plan
13:13:27 <hiro> yep
13:13:37 <GeKo> which would work more easily by just saying: you are welcome to use that db with X years/months of past data
13:13:52 <hiro> but also if we can open up object storage we could serve all the data we have there (that we want ofc)
13:14:06 <GeKo> indeed
13:15:20 <hiro> so you could do COPY table FROM 's3://mybucket/2024/10/13/relay-descriptors/server-descriptors.parquet';
13:15:26 <hiro> something like that
13:16:39 <hiro> anyways let's see how that goes first
13:17:08 <juga> wfm
13:17:19 <hiro> @ggus do we have a meetup scheduled or to schedule for mar/apr?
13:18:22 <ggus> hiro: nope
13:18:50 <hiro> ok
13:18:55 <hiro> anything else for today's sync?
13:18:59 <ggus> (there are some internal comms to be discussed first)
13:19:21 <hiro> ok sounds good
13:20:32 <GeKo> nothing from me
13:20:42 * juga is good
13:21:01 <juga> oh, we have p183 sync at 14utc, right?
13:21:18 <hiro> yes
13:21:21 <juga> kk
13:21:30 <sarthikg[m]> nothing from my side!
13:21:38 <hiro> ok thank you everyone let's end the meeting
13:21:45 <hiro> #endmeeting