18:02:45 <mikeperry> #startmeeting tbb
18:02:45 <MeetBot> Meeting started Mon Sep 29 18:02:45 2014 UTC.  The chair is mikeperry. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:02:45 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
18:03:14 <mikeperry> ok
18:04:17 <mikeperry> so last week I drove the chemspill release, and then did some emails and calculations for scaling the tor network, and dealt with some org issues
18:05:08 <mikeperry> this week, my plan is to finally get arthur's esr31 branch into our main repo and prepare the tor-browser-builder.git for making some builds
18:06:00 <mikeperry> I think that's it for me
18:06:42 <arthuredelstein> Is your scaling work online? I'm curious about that problem.
18:06:58 <Yawning> There's a huge tor-dev thread
18:07:14 <mikeperry> yeah, that thread, plus the tor-relays thread on the cost and value of the network
18:07:23 <arthuredelstein> Cool, thanks
18:07:38 <GeKo> I can go next
18:07:42 <mikeperry> I'm going to take all those results and roll them together into a blog post, I think
18:07:58 <mikeperry> but probably will focus on esr31 first
18:08:33 <GeKo> last week I helped getting the release done and tried to handle its fallout as good as possible
18:08:40 <GeKo> then I finished #13186
18:08:54 <GeKo> and I wrote a first test for #13024
18:09:22 <GeKo> and made a first try to backport the patch for #13027 which failed
18:09:54 <GeKo> we might need a special patch here as some huge worker rleated changes landed in Fx 32.
18:10:30 <mikeperry> heh, fabulous
18:10:31 <GeKo> this week I try to finish #13024 and #13027 and work other things related to ESR 31
18:10:43 <GeKo> *on other
18:10:51 <GeKo> that's it.
18:12:03 <Yawning> quick question, does the bundle build process change significantly with ESR31?
18:12:33 <mikeperry> I think GeKo had to upgrade a bunch of the toolchain, and make some other mods
18:12:46 <GeKo> well, the process is still the same: |make build| :)
18:12:51 <armadev> last week i met with two groups of developers in DC who maintain browser extensions to help visualize your recent browsing history
18:13:17 <armadev> the main one is in chrome, and it basically looks at all the pages you visit, to see which links you didn't follow, and then has a backend machine learning thing to tell you which links you'll find most interesting
18:13:29 <armadev> the backend thing does thousands of ajax calls to a remote db
18:13:38 <GeKo> Yawning: but yes under the hood there are some more or less significant changes
18:13:46 <armadev> and he wants to integrate it into tor browser so his users can have tor style privacy
18:14:12 <Yawning> armadev: oh dear god
18:14:17 <armadev> he's now thinking of rewriting his thing for firefox so it will integrate better with tor browser. but the ajax calls will probably not want to go over tor (he needs ms latency)
18:14:25 <Yawning> GeKo: hmm, will I be sad the next time I rebase?
18:14:32 <mikeperry> if there is an identifier attached to those ajax calls, that's fail
18:14:50 <GeKo> Yawning: I don't think so.
18:14:51 <armadev> the other dev group similarly does visualization of stuff, and does ajax to localhost:8080
18:15:04 <Yawning> the obfs4 branch is against 4.0-a3 atm (works great btw ^_^)
18:15:05 <mikeperry> which there almost certainly is. I think these people are confused about what private means
18:15:22 <armadev> they want privacy from the destination websites. not from their local network or their ajax server.
18:15:32 <Yawning> "private in the sense that only us, and the people that wrote the NSL can steal ur sekrits"
18:15:33 <Yawning> ?
18:16:15 <armadev> also, if the database is on localhost, then the local observer doesn't see those transactions. unless the remote website tricks the browser into making use of the fact that they set 'no proxy for' localhost.
18:16:34 <armadev> anyway. i don't want to derail your meeting. which of you should i drag into this pit with me?
18:16:46 * GeKo hides
18:16:53 <Yawning> *poof*
18:17:19 <mikeperry> is there lots of money at the bottom of this pit for us or something? it seems like a silly distraction otherwise
18:17:37 <GeKo> armadev: they should files tickets and we'll see how the discussion goes?
18:17:40 <armadev> it is indeed part of SponsorR, who will like us more if we hang out in the pit sometimes.
18:18:34 <armadev> i also met a fellow who does trainings for DEA to use tor browser when investigating fugitives by web browsing
18:18:53 <armadev> we all have our opinions on DEA, but i figure having them understand that tor browser has good uses could be a win
18:19:05 <armadev> ok, i have derailed your meeting enough. carry on. :)
18:19:58 <mikeperry> yes, I am actually less wary that the DEA's use of Tor Browser will lead us to do crazy things in terms of development as compared to this SponsorR idea, which is kind of surprising :)
18:20:07 <qwerty1> they don't need someone to train them when they have their own training which takes their own time and money...
18:20:38 <armadev> i don't expect us to do development for the crazy ajax users. i think step one, and maybe that's enough, is to tell them what's likely to go wrong with their plan.
18:21:28 <mikeperry> yeah, that seems like a long conversation, and highly dependent on what their plan is, and what they want out of it
18:23:30 <mikeperry> ok, who wants to go next?
18:24:15 * MarkSmith can go
18:24:24 <MarkSmith> Last week, Kathy Brade and I finished #13021 (status is now needs review).
18:24:35 <MarkSmith> Then, as soon as the 4.0-alpha-3 candidate builds were ready, we did some testing of the updater with 4.0-alpha-2 and 4.0-alpha-3 and we found several bugs.
18:24:43 <MarkSmith> #13241 and #13245 have been fixed; #13247 remains open
18:25:01 <MarkSmith> This week we plan to help with ESR31 patches and test the updater in a 4.0-alpha-3 to 4.0-next (based on ESR31) update scenario.
18:25:11 <MarkSmith> Also (and I know we said this last week too), if we have time we will work on generating incremental MAR files as part of the Gitian-based build process.
18:25:19 <MarkSmith> That's all for us.
18:26:04 <MarkSmith> Oh, and I saw the message boklm posted to tor-dev regarding creating incremental MAR files outside of the gitian-based process.
18:26:18 <MarkSmith> We could discuss that here or on tor-dev.
18:26:43 <boklm> yes, it seems we don't need to do it in gitian to be able to reproduce them
18:27:08 <MarkSmith> While that is true, there are other reasons to make this stuff part of the automated process.
18:28:11 <MarkSmith> e.g., we would want to compare hashes of incremental MAR files among different developers/release engineers and that infrastructure is already in the existing process.
18:28:20 <GeKo> boklm: you mean it is reproducible outside but not inside gitian?
18:28:31 <GeKo> that would be quite surprising
18:28:33 <boklm> GeKo: it's probably also reproducible inside gitian
18:28:51 <GeKo> it wasn't the last time
18:29:11 <mikeperry> well, it does seem like a post-processing step. it seems like there should be some scripts in the builder repo that add incremental mars to your current version dir based on the previous one, and then signs them
18:29:27 <mikeperry> how big are the incrementals?
18:30:15 <boklm> the MAR between tor-browser-linux32-4.0-alpha-2_fr.mar and tor-browser-linux32-4.0-alpha-3_fr.mar is 3.3M
18:30:58 <MarkSmith> Makes sense for the MAR file creation to be optional, maybe even for full MARs (makes for a longer build that takes more disk space).  I guess I am saying I would not want the scripts to be too far away from the other builder stuff.
18:31:05 <MarkSmith> Just another make target.
18:32:15 <GeKo> boklm: ah, no, the mar-tools were the issue, see #13041. I mixed that up.
18:32:32 <boklm> GeKo: ah, ok
18:32:41 <MarkSmith> It could be part of the update-responses script or something tied to it (since that is now in the builder repo)
18:33:05 <GeKo> how does Mozilla do this exactly?
18:33:16 <mikeperry> MarkSmith: yes, that seems the best place
18:33:30 <MarkSmith> And we don't necessarily need to have reproducible mar tools if we just used them locally (but it would make me feel better if we understand and fix #13041).
18:34:32 <boklm> I think adding that to the update-responses script would not be too difficult
18:34:33 <MarkSmith> Mozilla has a database driven system to do all this stuff.  I think it is all part of their automated build system, but that is something bigger than the Firefox build system.
18:35:28 <GeKo> ok, the update-responses script sounds good then, I agree
18:35:58 <MarkSmith> Probably the most important things are (1) reproducibilty and (2) making it easy for developers/release people to generate everything.
18:36:22 <GeKo> yes
18:36:32 <MarkSmith> And I think update-responses will fit those requirements if everything is driven by a config file that is in git
18:36:53 <MarkSmith> (I assume that would be the plan)
18:37:07 <mikeperry> yes, we'll want to somehow reproduce all of the update response xml and mar files, too... that verification should also be easy
18:37:33 <arlolra> boklm: did you see sukhe's message last week?
18:37:46 <arlolra> he's looking for a place to build instantbird
18:38:52 <boklm> I think we can add to the config.yml file the infos about which incremental update paths we want to support, and let the update-responses script generate the missing mar files
18:39:24 <MarkSmith> that sounds good to me (config.yml)
18:39:48 <boklm> arlolra: ah yes, I didn't answer yet this email
18:39:59 <MarkSmith> boklm: if you have time to work on this, please go ahead and ping me and brade if you have questions
18:40:15 <boklm> MarkSmith: ok, I can do this during this week
18:40:23 <MarkSmith> Thanks!
18:41:27 <arlolra> boklm: ok, just confirming that you're aware
18:41:32 <boklm> arlolra: I'm not sure which machine we should use to build instantbird
18:41:44 <mikeperry> boklm: I noticed that the 3.6.6 tests still randomly failed for some locales for fte_httpproxy and other things. is there a way to make these more reliable?
18:41:54 <mikeperry> are any of those actually failures?
18:42:53 <arlolra> boklm: we can discuss after this meeting (sorry everyone)
18:43:18 <boklm> mikeperry: it seems tor_fte_httproxy is an actual failure as it failed on all bundles
18:43:54 <GeKo> I tested the ko bundle locally and everything was/is fine
18:43:56 <boklm> mikeperry: and tor_fte which failed only on 2 of them is a temporary failure
18:44:06 <mikeperry> hrmm, not on tor-browser-linux64-3.6.6_de.tar.xz and a couple others..
18:44:22 <boklm> ah
18:44:25 <mikeperry> pt_PT and zh-CN
18:45:10 <boklm> I was thinking about retrying those tests 2 or 3 times when they fail, to remove temporary failures
18:45:46 <mikeperry> I wonder if this means that fte+httpproxy actually has bugs where it fails sometimes
18:45:57 <boklm> I don't know why tor_fte_httpproxy fails most of the time, but not always
18:46:00 <mikeperry> or if it is something specific about our tests
18:46:21 <mikeperry> Yawning: do you want to try to go down this rabbit hole? :)
18:47:06 <boklm> the result page includes the config file that is used in the test
18:47:21 <Yawning> wut
18:47:32 <Yawning> it's common code
18:48:06 <Yawning> (also we have automated tests, since when?)
18:48:11 <Yawning> >.>
18:48:28 <Yawning> file a ticket about it and I can take a look at it probably
18:48:33 <Yawning> not sure when I'll get to it though
18:49:48 <mikeperry> boklm: can you file that, ideally with some test output for the instance where it is failing?
18:50:21 <boklm> mikeperry: ok
18:50:29 <Yawning> please?  I could try to repro it, but that would simplify things
18:50:30 <Yawning> ty ^_^
18:52:44 * arthuredelstein can go next
18:53:02 <arthuredelstein> Last week I worked on #11955, (still in progress).
18:53:12 <arthuredelstein> And I ported #3455 to tbb-esr31 and also to mozilla-central.
18:53:25 <arthuredelstein> Also I wrote a patch for torbutton (#13198).
18:53:35 <arthuredelstein> And I squashed and ported tbb-esr31 to esr31.1.1.
18:53:58 <arthuredelstein> This week I'll continue to work on #11955 and hopefully other patches for ESR31.
18:54:09 <arthuredelstein> That's it for me
18:57:13 <mikeperry> ok, I think we're down an mttp
18:57:36 <mikeperry> I think that leaves anything else boklm wants to discuss, and then that's it?
18:57:37 <mttp> just arrived
18:57:45 <mttp> sorry for being tardy
18:58:42 <mikeperry> np
18:58:44 <mttp> One pretty popular request was users asking us what to put for their proxy address and port
18:59:11 <mttp> So I put some ideas in #13271
19:00:04 <mttp> One person said they got a hard crash on Windows when they tried to open Tor Browser from their USB drive
19:00:18 <mttp> Complete with "Windows is searching for a solution to the problem"
19:00:38 <mttp> They said that Tor Browser opened normally when run from the Desktop for them
19:01:52 <mikeperry> as in the browser crashed, or windows did?
19:02:33 <mikeperry> we've had issues with USB in the past.. somtimes it ends up owned as the administrator or other user, and then Firefox dies because it can't write the lock file or the profile directory
19:03:08 <mttp> They said they received the windows-style "program x just crashed" dialog when clicking "Start Tor Browser" before tor-launcher opened
19:04:10 <MarkSmith> I agree with Mike that a permission problem or lack of write access is likely (but hard to know)
19:04:37 <mttp> Ok that seems likely
19:04:52 <mttp> So does that mean it's a known issue or not a bug?
19:06:28 <mikeperry> well, we could try to handle the error better.. I think some time ago I added some code in Torbutton to translate the exception into something human readable, but prhaps Tor Launcher is hitting it now before that code runs
19:08:01 <MarkSmith> It sounds like you should file a ticket.  If we have steps to reproduce it, we should be able to generate an error alert.
19:08:26 <mttp> Sure will do.
19:09:06 <mttp> Pretty often the questions that get asked on the support desk
19:09:22 <mttp> are answered pretty directly on the FAQ page.
19:09:55 <mttp> It is a big page, but I wonder if at some point in the future, we could get a link to it on the about:tor page
19:10:25 <mttp> or if instead we should maybe wait until Lunar gets the tb-manual ready for us
19:11:36 <MarkSmith> Offhand, it seems like adding some kind of support link (or links) or about:tor would make a lot of sense.
19:11:36 <mttp> I guess we shouldn't overload the about:tor page with links or it would just look like the tpo home page
19:11:45 <mttp> ok
19:12:33 <mttp> Oh also
19:13:01 <mttp> I think support people have expressed their frustration before at SOPHOS
19:13:02 <MarkSmith> The remote check page (https://check.torproject.org/?lang=en_US) has some support links (Short User Manual, Tor Q&A Site).  So maybe open a ticket for adding to about:tor and we can discuss there.
19:13:20 <mttp> MarkSmith: sure
19:14:07 <mttp> a la https://www.torproject.org/docs/faq#SophosOnMac
19:14:11 <mikeperry> it seems like either the FAQ or the manual could be linked inside the "What Next" box
19:14:48 <mttp> Someone emailed the help desk to say that SOPHOS developers have finally taken note
19:15:04 <mttp> and stopped breaking Tor Browser in  SAV 9.1.5
19:15:28 <GeKo> re linking to FAQ stuff see #10534
19:15:55 <GeKo> we had this discussion there already
19:16:42 <GeKo> hmm or maybe not
19:16:58 <mttp> Yeah that seems like a longer term big project
19:17:24 <mttp> Linking to the FAQ page would definitely not solve all support requests
19:17:35 <mttp> or the majority of them
19:17:40 <MarkSmith> Kathy and I think the about:tor thing should be a new ticket
19:17:51 <GeKo> yes, I agree
19:17:53 <mttp> Ok cool
19:18:51 <mttp> I think that's all I have for this week
19:19:32 <mttp> I'll be taking October off, so I'm not aware of anyone having agreed to fill in as a help desk correspondent
19:19:39 <mttp> during that time
19:19:57 <mttp> for these IRC meetings
19:21:28 * boklm can go next
19:21:54 * MarkSmith Has to step out for a minute (will read traceback)
19:22:08 <boklm> I have two patches in my update-responses branch of user/boklm/tor-browser-bundle.git that need to be merged
19:22:18 <boklm> Should I open a ticket about this ?
19:23:33 <mikeperry> yeah, GeKo or I can merge it. If you end up making lots of changes, we can add you to commit for that repo
19:23:44 <boklm> ok
19:23:53 <boklm> I have also been looking at #13015 (using the version from git tag to set the release version), and I was wondering two things:
19:24:12 <boklm> - if it's a possibility that we build more than one release from the same commit (a stable and an alpha for instance), or if we should not expect that to happen
19:25:34 <boklm> - if sometimes we do builds on untagged commits (other than nightly builds), and what should be the version in this case
19:26:10 <mikeperry> hrm.. sometimes we build from commits just to see how far the build will get before tagging
19:26:34 <mikeperry> and after the switch to esr31, I can imagine a time where we're using master for both alpha and stable..
19:26:51 <GeKo> but that does not mean the same commits
19:27:41 <mikeperry> it may. I could see the same commit with two tags..
19:27:49 <GeKo> sure
19:29:34 <GeKo> but my gut feeling tells me that won't happen in the near furture given how we currently do the releases
19:29:44 <GeKo> which have more incremental character
19:29:56 <boklm> if we do that, then it may be difficult to find the right tag to use to guess the version number. Or we could expect the alpha one to contain an "a" inside the version, and the beta one a "b" ?
19:32:07 <mikeperry> yeah, normalizing our tags in some way like that seems like a good plan
19:34:03 <mikeperry> we could make the version number from the versions file actually be a substring of the build tag..
19:34:22 <mikeperry> like ${TORBROWSER_VERSION}-build1
19:34:31 <mikeperry> that would be more sane than what we do now
19:36:05 <boklm> should we document our stable/beta/alpha versionning inside tor-browser-spec.git ?
19:36:28 <mikeperry> yeah
19:36:55 <boklm> ok, I will send a patch for that
19:37:21 <boklm> I think that's it for me
19:39:21 <mikeperry> ok. I will be going through https://trac.torproject.org/projects/tor/query?keywords=~MikePerry201409R&status=!closed this week, and prepping esr31. I'll send a mail out if it looks like we're ready for a release before next Monday
19:40:11 <GeKo> huh? release?
19:42:03 <mikeperry> yeah, a 4.0a4 with esr31. non-security release
19:42:25 <mikeperry> might be optimistic to think we'll have it by Friday though
19:42:58 <GeKo> indeed. I'd be happy with some working nightly builds to test by then actually.
19:43:26 <GeKo> but, sure, if we can get a 4.0a4 with esr31 even better
19:43:37 <arthuredelstein> Let me know how I can help.
19:45:01 <mikeperry> ok
19:45:15 <mikeperry> that's it for today then. thanks everyone
19:45:19 <mikeperry> #endmeeting