18:31:08 #startmeeting tor-dev 18:31:08 Meeting started Wed Nov 18 18:31:08 2015 UTC. The chair is mikeperry. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:31:08 Useful Commands: #action #agreed #help #info #idea #link #topic. 18:31:23 hello im here 18:31:25 but not for long! 18:31:46 may i go first with the reportbacks? 18:31:47 Let's start with status reports? 18:31:54 Please! 18:32:06 over the past week, i worked on prop250 18:32:09 Afk 5 min 18:32:16 i did a few refactorings and moved the implementation forward. 18:32:35 i think we are done with the voting logic 18:32:41 and the commitment/reveal logic 18:32:54 the next step is computing the actual shared randomness 18:33:13 i made a post in the ML about making this more robust: https://lists.torproject.org/pipermail/tor-dev/2015-November/009903.html 18:33:30 this is what i was up to. 18:33:37 i also have space in my review queue 18:33:55 so if nickm_mobile or someone else has something that you want reviewed please tell me. 18:34:23 and that's that 18:34:31 i will be off now, but i will check backlog 18:34:38 so please let me know which ticket you want reviewed!! 18:34:40 cheers 18:34:50 next? 18:35:41 I have been testing #175292, along with the rest of my netflow stuff 18:36:11 I've been running into issues where chutney does not behave very much like a real tor network 18:37:00 primarily because of the continual consensus creation+downloading. I also just hit an issue where it tends to use two guards very often, despite the consensus params. this might be considered a tor bug, though 18:37:43 rehi 18:37:49 back from bus 18:38:00 circuit_establish_circuit() chooses an exit before it chooses the rest of the path, so if it chooses your guard as an exit first, you have to use a second guard. in small testing networks, it happens frequently that this exit is your guard, so you often end up using two guards 18:38:29 makes sense. I seem to remember a really long arguument as to whether it leaked more info to have a guard as your exit sometimes, or a guard as your exit never 18:39:04 otherwise #17592 testing is going fairly well. it is mostly behaving how I want, modulo those issues 18:39:45 I might play with exactly when it samples timeouts and stores them. I'm running into the max(X,Y,Z...) distribution bias problem again 18:39:54 hmm 18:40:35 as for the testing issue, I would love to have a couple of chutney networks with default + accurate timing. The ones we use for the default test really do need to be accelerated though, or otherwise they'd take hours to run right 18:40:51 I think it probably shouldn't be storing per-circuit timeouts, but should have one single timeout associated with port prediction that also governs circuit idle timeouts 18:42:16 my turn? 18:42:38 I think that's it for me. I was going to comment more on circuit_establish_circuit(), but it is probably a rabithole 18:42:41 I've been going a bit mad with auditors and accountants and trying to get an ED hired and making bills get paid. Hope hope hope. 18:43:15 I've been trying to revise and merge old patches from others, and merge other stuff 18:43:27 I think I've finally got the patches all written for the first round of the blob-removal: 18:43:43 with all of them merged, we will no longer have any SCC in our callgraph over 19 functions in size 18:44:00 next step on that plotline will be to take function pointers into account 18:44:50 i'm taking a break there to review a few more things, and finish #15055 at long last 18:44:53 so tired of that one 18:45:03 also got to get 0.2.7.5 out 18:45:07 hm 18:45:11 I think that's it for me 18:46:44 anyone else around? 18:47:07 ok. Any discussion topics? 18:47:14 hi meeting! 18:47:21 I'm the only one who likes the new meeting times it seems :/ 18:47:22 yes I can go quickly 18:47:41 great! 18:49:01 I'm back from DC which was the R meeting so I'm catching up on things, mostly emptying queue of "people need my attention", I've started this morning also review on Yawning's #13737 (will finish soon), been busy yesterday also with our jabber server that needed immediate attention, I'm expecting things to cool down by Friday so I can focus on little-t tor code and more dev. 18:49:17 yay for review 18:49:23 done -- 18:50:16 any more? 18:50:20 anyone else? 18:51:26 ok. any discussion topics? 18:51:40 I'd like to talk about the meeting time, but I think maybe that's a better discussion to have with more people. :) 18:51:45 I am wondering if I should try to write an official chutney test for the connection lifespan stuff and netflow padding.. it would take minutes-hours to run, even if I coulod figure out what to instrument to check for success/good operation 18:52:09 right now I've just been watching log lines 18:52:40 I'd love to have a minutes/hours-long tests that covers this stuff, though the instrumentation is hard. 18:52:59 I wonder if it's an option to make the tor code self-testing somehow, so it can LD_BUG if there's a mistake 18:53:03 not sure if that's doable here though 18:53:45 armadev suggested such a thing for #17604, but that seems expensive.. we'd have to iterate through all the connections and look for dups 18:54:24 there's already an index of conns by identity I hope... 18:54:33 Or we could check only once every N minutes 18:54:42 for the netflow padding itself, its tricky.. cause what I really want to know is how long orconns live, and when the padding actually happens, and how much of it there is, independent of what tor actually things 18:54:58 but I could settle for logline scraping or something, if we already had tests that do that sort of thing 18:55:13 yes, there is an index of orconns by identity digest 18:55:16 sounds more to me like you'll have to do some kind of capture on it 18:55:34 and none of that exists right now? 18:55:45 (capture in chutney) 18:55:50 there's no "chutney + protocol capture" framework yet 18:55:52 how about control port logging/monitoring? 18:56:04 that could work, but it relies on there being no bugs in tor's controlport stuff 18:56:36 does that exist yet either? 18:56:56 there's lots of stuff in stem you could adapt, but it's not integrated into chutney 18:57:05 I'd be interested in collaborating if this is something you want to work on 18:58:29 possibly. I think I need to think about what I even want here. packet capture itself might be useless due to TLS.. that's why I asked about the control port.. somehow using the two together or something 18:58:36 sounds complicated though 19:02:05 if I checked for duplicate connections, I guess I'd just add that to run_connection_housekeeping() or on a less frequent schedule in run_scheduled_events()? 19:04:28 run_scheduled_events is now refactored, but yeah 19:08:10 oh, I have another random question. how do I check for the number of currently scheduled libevent timers, and how many is "too many"? 19:08:13 mikeperry: could you have a quick re-look at my #17590 btw? I fixed the XXX and added a little more to it 19:08:38 mikeperry: there's not a way to do that, I believe. Too many is when you start getting towards "one per connection" or "one per circuit" 19:08:59 there are tricks to make those work, but they're probably best avoided 19:10:07 is it just as expensive to cancel a timer in that situation as it is to add one? or what is the expensive operation? delivering the event for the timer callback? 19:10:59 it uses a heap-based priority queue, so basically everything is O(lg n) 19:11:07 add, remove, de-queue 19:11:11 check-next is O(1) 19:12:55 I could do something crazy like make a bitarray with 1000 bits in it, and stack the things if more than one of them is scheduled for a given millisecond 19:13:11 what the F kind of events are you thinking about here? 19:13:59 there's a crazy data structgure that can be more efficient if you really want, and if the timeouts in question are always constant, you can tell libevent to put those events in a doubly-linked-queue rather than a heap... 19:14:45 basically, the way the netflow patch works is that once per second, I call channelpadding_decide_to_pad_channel() from run_connection_housekeeping(). if the time-to-send-padding is less than 1 second remaining, I need to schedule a timer to send the packet when it should be sent 19:15:03 otherwise, I wait until the next call to run_connection_housekeeping() and do not schedule a timer 19:15:55 what horrible thing would happen if padding was added on 1-second boundaries? 19:15:56 and active connections will never come close to that point, so it will only be relays with a lot of inactive orconns that will end up with lots of these timers scheduled 19:16:01 or 200-ms boundaries 19:16:02 or whatever? 19:16:56 against the default netflow configs, not much, but I didn't really like the padding to be completely predictable, because it may also obscure things like HS circuit fingerprinting 19:17:22 maybe batch the connections into buckets and add the padding at different offsets into the second? 19:18:35 and pick those bucket offsets randomly every so often? 19:18:43 yeah 19:19:14 could work 19:19:59 is there anything that already happens on sub-second scheduling inside of tor that I could use instead, btw? 19:20:09 out of curiosity 19:20:20 If you'd like to see the improved algorithm which allegedly libevent should use instead, which could handle way more events, see http://www.25thandclement.com/~william/projects/timeout.c.html 19:20:23 It's a cool algorithm 19:20:48 resurrected from 1987 19:21:00 not sure if it's a good idea though 19:21:23 as for sub-second: I'll be adding it soon now that 3199 is mereged. There are some kludges too. one second... 19:21:44 btw endmeeting maybe? I think it's just us two now 19:21:51 ok. sure 19:22:21 #endmeeting *baf* (couldn't resist using the e-gavel :)