15:58:19 <hiro> #startmeeting network-health 07/31/2023
15:58:19 <MeetBot> Meeting started Mon Jul 31 15:58:19 2023 UTC.  The chair is hiro. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:58:19 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:58:40 <hiro> meeting pad: https://pad.riseup.net/p/tor-nethealthteam-2023-keep
15:58:58 <RishadBaniya[m]> just opened the email right now, seems a lot of issues to be worked on..
15:59:11 <mattrighetti[m]> o/
15:59:12 <RishadBaniya[m]> i'm currently finishin the onionperf one
15:59:15 <RishadBaniya[m]> right now
15:59:30 <hiro> @RishadBaniya[m] I am also parsing onionperf files
15:59:36 <RishadBaniya[m]> can i continue with those issues after i finish the onionperf one juga
15:59:42 <juga> RishadBaniya[m]: sure
15:59:43 <RishadBaniya[m]> yep, the parsing one
15:59:47 <RishadBaniya[m]> of the json files
15:59:58 <RishadBaniya[m]> the problem seems to be with the reason for failure of circuit creation
16:00:00 <juga> you can start with the new ones next week if needed
16:00:01 <RishadBaniya[m]> that is absent in
16:00:09 <RishadBaniya[m]> onionperf analysis file
16:00:17 <RishadBaniya[m]> what should i do about it
16:00:26 <hiro> I think you could find that in the logs
16:00:38 <hiro> in the torctl logs
16:00:45 <hiro> so you should parse that instead of the json
16:01:09 <RishadBaniya[m]> sure i'll take a look at it too
16:01:46 <juga> RishadBaniya[m]: if you have any problem parsing the json logs, as you see hiro is also working on that, so you know who to ask ;)
16:02:10 <RishadBaniya[m]> oh, can i get a link for that parsing work
16:02:11 <hiro> yes this is the repository I am working on https://gitlab.torproject.org/hiro/tor_fusion/-/blob/main/src/main.rs it's not finished not even in beta... just a start
16:02:13 <mattrighetti[m]> Hi guys, any updates on the metrics db hiro ?
16:02:36 <hiro> @matt I haven't deployed the changes but sent you an issue with the sql change
16:02:38 <hiro> hold on
16:03:11 <hiro> https://gitlab.torproject.org/tpo/network-health/metrics/networkstatusapi/-/issues/4
16:03:23 <hiro> let me know if I should just deploy or you want to update the api first
16:04:06 <mattrighetti[m]> You can deploy if you want, I’m adjusting the queries later today
16:04:09 <mattrighetti[m]> No big deal
16:04:19 <hiro> ok I didn't want to break anything for you
16:05:09 <mattrighetti[m]> I’ll ask you some questions on uptime and clients later, I’m not sure how to get those and we wanted to pause uptime for the moment
16:05:14 <mattrighetti[m]> Is that still the case?
16:05:42 <hiro> yes uptime is paused... maybe we won't even create that endpoint for now
16:06:01 <hiro> clients I think I have the query in grafana ...
16:06:07 <hiro> I'll create an issue
16:06:12 <hiro> and give you the query
16:06:29 <RishadBaniya[m]> recently i had tried to parse by creating struct and implementing the serde seri/deseri and did through it..arrow_schema
16:06:32 <RishadBaniya[m]> seems interesting
16:06:40 <RishadBaniya[m]> how different does it
16:06:46 <RishadBaniya[m]> handle things, i'm curious
16:07:15 <mattrighetti[m]> Thanks Hiro, I think I’ll fix summary and bandwidth as soon as the db is ready and then make a MR, it’s already pretty big
16:07:28 <hiro> yep let's do this @mattrighetti[m]
16:07:56 <hiro> @RishadBaniya[m] I wanted to leverage the sql query over the df object once the json is parsed
16:08:17 <hiro> seems pretty similar to how I would do that in python with panda dataframes
16:09:08 <RishadBaniya[m]> and about fetching the data..collector API is the goto place right?
16:09:32 <hiro> collector doesn't have an API, just the webserver
16:09:52 <hiro> on collector we just share the analysis files
16:10:03 <hiro> but all the onionperf clients have a webserver too
16:11:27 <RishadBaniya[m]> so getting data from all the hosts running the onionperf analysis directly the best practice? or through collector itself?
16:11:28 <hiro> and some expose the tgen and tor logs
16:11:40 <hiro> the analysis files you can get from collector
16:12:01 <hiro> but you mentioned you were interested in the reasons for why a circuit has failed to start
16:12:25 <hiro> https://op-de7a.onionperf.torproject.net:8443/ I think you would have to look at the tor logs for that
16:12:31 <hiro> this client exposes those
16:12:48 <hiro> so you can start testing your code and if you need I can expose the logs for the other clients too
16:13:02 <RishadBaniya[m]> i understood now, thanks :)
16:13:36 <juga> RishadBaniya[m]: if getting the reason why a circuit failed complicate things too much, just ignore it, the first step is just to know if a circuit fails or not, but we don't really need the why
16:14:03 <juga> (as the first step)
16:14:58 <RishadBaniya[m]> I'll do this work as a  milestone, first being if a circuit fails or not and second being adding info on why they fail through the logs
16:15:14 <RishadBaniya[m]> now that i've got the idea on where to look for the things needed
16:15:46 <juga> RishadBaniya[m]: sounds good, if you see that would take more than a week, just leave it to the basics...
16:16:00 <juga> RishadBaniya[m]: with respect all the new issues i created, they are related to the databases, to help with the partition analysis
16:16:04 <juga> some might not be needed or are small changes, i just tried to create small issues
16:16:50 <juga> it'd be great if you could start this or next week, but no rush
16:17:43 <RishadBaniya[m]> Sure :), i'll also handle the issues soon
16:17:50 <juga> thanks!
16:18:48 <mattrighetti[m]> All good on my side
16:19:04 <hiro> thanks all
16:19:20 <hiro> I'll close the meeting then
16:19:22 <mattrighetti[m]> o/ ciao ciao
16:19:25 <hiro> #endmeeting