20:59:25 <isis> #startmeeting prop#239 Consensus Chain Hashing
20:59:25 <MeetBot> Meeting started Thu Feb  8 20:59:25 2018 UTC.  The chair is isis. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:59:25 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
20:59:42 <isis> #info https://lists.torproject.org/pipermail/tor-dev/2018-February/012897.html
20:59:52 <isis> #info https://gitweb.torproject.org/torspec.git/tree/proposals/239-consensus-hash-chaining.txt
21:01:49 <isis> hello! this is a meeting to discuss consensus chain hashing
21:02:04 <isis> would someone like to start by summarising their understanding of the proposal?
21:02:46 <nickm> (shouldn't be me, since apparently I'm a coauthor :) )
21:04:05 <isis> ok
21:04:14 <nickm> who is here besides me and isis?
21:04:36 * teor4 is here
21:05:01 <tjr> Now I am here. Thanks nick :)
21:05:10 <Samdney> hi
21:05:17 <isis> let me go check who else was interested
21:06:36 <isis> hmm komlo, ln5, and paul were interested
21:07:42 <tjr> I will try to summarize
21:07:51 <isis> ok thanks!
21:08:08 <tjr> Dirauths include <some number> of previous-consensus lines in their votes that correspond to <some number> of previous consensuses they know about
21:08:29 <tjr> When calculating a consensus, if >half of the votes have an identical previous-consensus line in it, it makes the consensus
21:08:54 <tjr> When a client gets a consensus, it checks all previous-consensus lines against all previous consensuses it has.
21:09:03 <tjr> The only reporting/complain mechanism is in the tor logfiles
21:09:19 <tjr> The client complains in many situations:
21:09:30 <tjr> - When it has prev consensuses and can't do anything
21:09:53 <tjr> - When it has a gap in consensuses so it can't build a full chain up to the most recent consensus from it's oldest consesnsus
21:10:07 <tjr> - When it has a consensus whose hash doesn't match the reported hash
21:10:41 <tjr> DirAuths complain if they get a vote with a prev- line whose hash doesn't match the the consensus they have for that time
21:10:54 <tjr> There are options to disable these checks
21:11:06 <tjr> fin
21:11:21 <tjr> Sorry:
21:11:26 <tjr> When it has prev consensuses and can't do anything -> When it has NO prev consensuses and can't do anything
21:12:12 <tjr> Ah I think <some number> = 24 + 2
21:12:21 <teor> 3 + 1
21:12:26 <teor> ugh, 3 + 2
21:12:49 <teor> 3 valid consensuses, plus 2 older ones
21:13:21 <tjr> Is 'valid' "Valid as defined by the consensus" or "valid as defined by the code"? Since IIRC tor will use a consensus for up to 24 hours?
21:13:41 <komlo> hello, just joining
21:13:45 <teor> that's "reasonably live"
21:14:48 <isis> komlo: hi!
21:15:14 <teor> anyway, you're right, the proposal should clarify
21:15:31 <nickm> #item proposal should clarify how many older consensuses to hold
21:16:55 <isis> are there any devices where we should be worried about keeping 5? 6? consensii around on disk?
21:17:20 <teor> yes, most of the embedded, mobile, and disk-constrained ones
21:17:26 <tjr> The answer to "Are there any devices where we should be worried about X" seems to always be mobile. :)
21:17:27 <isis> cached-microdesc-consensus is ~2MB each
21:17:38 <teor> #item why not just keep the hash(es) of each consensus
21:17:54 <nickm> you don't need to keep 6 consensuses around; only 6 digests
21:18:19 <isis> yeah, the prop made it sound like you cache the whole document
21:18:29 <nickm> one thing this doesn't specify is whether it is looking at the "digest" or the "digest as signed" of the consensus
21:18:39 <teor> yes is does
21:18:43 <nickm> oh, it does?
21:18:48 <Samdney> yeap
21:18:55 <nickm> ah yeah, "the signed portion".
21:19:18 <nickm> Should we change the mandatory hash from sha256 to sha3-256, to match the consensus-diff code?
21:19:47 <isis> yes, imo
21:20:01 <nickm> #item maybe use sha3-256 instead.
21:20:07 <isis> i was a bit confused about the part which says "all available hashes"
21:20:17 <tjr> Well... this kind of gets into my biggest question/concern: are there future thoughts/plans for what to do when you find a hash mismatch. Some sort of automated submission mechanism?
21:20:18 <tjr> Because that seems to be the most important, valuable thing; and that thing is only valuable if you submit either the whole document (for investigation) or hash+dirauth signatures (to prove validity of submission)
21:21:03 <nickm> hm
21:21:21 <nickm> maybe it needs to keep the hash and signatures, and maybe it needs to use sha256 then
21:21:42 <nickm> (since the signatures are currently over sha256)
21:22:09 <nickm> #item maybe store only the hash and the signatures.  Have the chaining hash match the hash used in the signatures?
21:22:18 <nickm> except.
21:22:36 <nickm> no, that's not enough info to prove dishonesty
21:22:46 <nickm> since it doesn't include any proof that the timestamp is wrong
21:23:05 <nickm> or rather, that the timestamp is what you say it was
21:24:06 <nickm> I think unless we change the directory signature format (a good idea but not for one day) we have to keep any old consensus we want to be able prove fraud about
21:26:11 <teor> ok, so is it sufficient to just keep a chain of diffs?
21:26:27 <armadev> would the idea that's been kicking around, of having the dir auths produce a "there was no consensus" consensus when they can't produce one, be helpful here? it seems like a lot of problems show up when there is no consensus for an interval but it's hard to prove the negative.
21:27:07 <nickm> you can't have a "no consensus" consensus
21:27:18 <nickm> you _can_ have a set of "no consensus" statements.
21:27:32 <nickm> rather, you can't rely on a no-consensus consensus working
21:28:05 <tjr> The logfile may get messy when there is no consensus, but at first glance it doesn't appear to be an issue with a future reporting mechanism...  We assume an attacker can prevent a tor client from receiving a consensus anytime they want
21:28:21 <armadev> right. it would be something like "dir auths sign an "i didn't find one" doc if they didn't find one, and maybe you cobble together enough such sigs to convince yourself that there couldn't have been one.
21:28:29 <nickm> I think that the proposal should make it clear that the "wrong digest listed" error is the one to shout about.
21:28:39 <nickm> the other warning conditions are mildly bad
21:28:54 <nickm> "wrong digest listed" means that the authorities are either confused, subverted, or have their keys stolen
21:28:57 <tjr> #item Instead of keeping full documents, can we keep a chain of diffs?
21:29:55 <teor> #item expand "some categories of attacks" to explain what the attacks actually are
21:31:01 <nickm> tjr: we'd have to keep reversible diffs, or build reverse diffs.
21:31:13 <nickm> our current diff format (ed) is very easy to apply, but completely unidirectional.
21:32:10 <tjr> Well, is the goal of using diffs to reduce disk and/or memory? If so always construct and keep document n, and keep the data to go from n-> n+1, n+2.... Each hour, construct n+1 into a full doc and drop n?
21:32:30 <tjr> If you ever need to submit, you can always construct any doc you need
21:33:29 <nickm> sure, that could work
21:36:27 <tjr> So do we want to speculate wildly about a future reporting mechanism, or just write in the doc that we want to design the system to support a future unspecified mechanism that will allow a client to submit a consensus document it deems suspicious?
21:36:54 <nickm> we could do both
21:37:10 <Samdney> oh nice idea :)
21:37:22 <nickm> I think v0 of this should at least contain a "here is the exported object that proves misbehavior" implementation.
21:37:44 <nickm> like, "I have saved the file to tor_trouble_evidence.0001; please get it to the tor team somehow!"
21:37:51 <tjr> Yea, it could write the (full) document to disk, and reference the filename in the logfile.
21:37:58 <tjr> ++
21:38:44 <tjr> #item Provide an export of a suspicious consensus that would prove misbehavior. (Probably in a file on disk and a mention in the logfile)
21:39:28 <tjr> I have a sinking suspicion that any reporting endpoint would need to be hardcoded in the source code. Anything specified in the consensus could be co-opted by a compromised consensus
21:39:48 <tjr> So the options would be: a) submit to dirauths b) submit to fallback dirs c) submit to a new endpoint specified
21:40:03 <tjr> And always d) <something I haven't thought of>
21:40:17 <nickm> "put on pastebin, scream loudly"
21:40:36 <teor> b) would make fallback dirs more of a target in that scenario
21:41:02 <tjr> That's a great v0 solution; but any mechanism that relies on users reading the logfiles I think is insufficient. Automated reporting is the way to go IMO.
21:41:17 <Samdney> +1
21:41:38 <teor> I think we have another proposal for automated reporting, aka consensus transparency
21:42:28 <teor> prop#267
21:42:48 <Samdney> but could somebody overflood the network with willful wrong "misbehaviour reports"?
21:42:51 <tjr> Yea. It is considerably more complicated, and I'm not actually sure it enables automated reporting.
21:43:21 <teor> Samdney: only if they controlled a majority of dirauth keys
21:43:36 <tjr> Samdney: The reporting endpoint could be DOSed; yes, but the first thing you'd build it to do is check for valid consensues and discard them, and check the validity of submitted data and discard that which is not.
21:43:51 <tjr> So a CPU-based DOS.
21:44:18 <Samdney> thanks, for the answer :)
21:45:31 <tjr> #item It would be good to look at prop267 again in relation to this document.
21:45:58 <tjr> I suspect either this or 267 should be pursued, but not both. So we should look at them and figure out what, if anything, one enables that the other doesn't
21:46:39 <tjr> I have some thoughts there (I think 267 gives you safety even when you have no prior consensuses which is very nice) but I really need to reread it closely.
21:47:46 <isis> sorry, i scheduled this meeting separate to prop#267 because i thought this was simple and we'd be able to come to some agreement about it
21:48:06 <isis> i can make the prop#267 one be also about deciding between these two and contrasting them
21:48:23 <isis> (if that would be a useful way to arrange it)
21:49:12 <tjr> More meetings is... well meetings. But I think it is probably best to think about them separately and then compare/contrast them in some sort of third meeting....
21:49:41 <tjr> If we're 50 minutes in on this relatively simple one, 267 will take every bit of an hour to get through not even considering the compare/contrast element.
21:50:58 <tjr> In my mind, I see a very good path forward to finalize this proposal into something well specified and complete (I think) as a result of this.
21:53:55 <isis> ok, so i think prop#239 should probably be revised, and while that's happening we can schedule a time for prop#267 discussion, then a contrast/choose meeting
21:54:26 <tjr> ++
21:55:01 <isis> would anyone be excited to revise 239?
21:55:53 <nickm> i could, but i'm not really excited :)
21:56:31 <isis> warning that if i revise it, i will find a way to use the phrase "put tor on the blockchain"
21:56:44 <nickm> we could also wait till we've talked about prop#267
21:56:56 <isis> ok that could work too
21:57:16 <nickm> another thing to be aware of: if nobody is enthusiastic to review the proposal, maybe nobody wants to implement it either.
21:57:55 <nickm> so we should figure out whether there's a potential implementor around
21:58:23 <isis> this would possibly make a good gsoc project?
21:58:44 <Samdney> I would be on board then ;)
21:59:02 <teor> we should also try to re-use as much of the shared random code as we can, including the line formats
21:59:10 <nickm> isis: conceivably!  It's a bad idea to rely on gsoc though.  You have about a 60% probability of winding up with working code...
21:59:30 <nickm> ... and about a 30% probability of winding up with working code with less effort than it would have taken you to write it yourself
21:59:45 <Samdney> *lol*
21:59:47 <nickm> gsoc is good for mentoring and evangelism and new perspectives; less so for free labor
21:59:57 <nickm> teor: +1
22:00:04 <isis> true
22:00:06 <Samdney> teor: +1
22:00:54 <isis> ok, so i will send out these notes, and also an email to schedule the prop#267 discussion
22:00:58 <isis> anything else?
22:01:58 <isis> #endmeeting