jlm-blog
~jlm

29-Jun-2010

Misuse of conditional probability

Filed under: math — jlm @ 13:27

When explaining conditional probability, the “two child” puzzle is often brought in, which is too bad because it’s a terrible example. Case in point, this recent blog post by Keith Devlin.

I tell you I have two children and that (at least) one of them is a boy, and ask you what you think is the probability that I have two boys.

(In a model where children’s sexes are independent and equiprobable,) he applies conditional probability to calculate the probability of them being the same sex at 1/3, in contrast to the “wrong” “intuitive” answer of 1/2. Let’s run with this 1/3 answer for a bit. The same analysis will show that if he tells us that one is a girl, the probability that the other is a girl is 1/3. He’s going to tell us either that he has a boy kid or he has a girl kid, and in both cases the probability of both children being the same sex is 1/3, so the unconditional probability of his children being same sex is 1/3, which is crazy. Where’d we go wrong?

The problem in the analysis is that him telling us he has a boy is not the same as conditioning the probability space on him having a boy. If you like calculating conditional probabilities, the proper condition to apply when he tells us he has a boy isn’t “he has a boy” but “he tells us he has a boy”. For the puzzle, we can assume he’ll be honest, so if he has two boys he’ll say he has a boy, and for two girls he’ll say he has a girl. The interesting case is if his children are mixed. Maybe he’ll say “boy” and “girl” half the time each in that case — but then, the probability of two boys turns out to be the intuitive 1/2 after all. Maybe he’ll always say “boy” then — this gives 1/3 for the quoted puzzle, as it “should”, but having the boy always trump the girl is unappealing and it’d give a 100% chance for two girls if he says he has a girl.

More importantly though, conditional probability is just trotted out there, with no consideration if it’s appropriate to the model. Is it appropriate? In the puzzle (“I tell you I have two children and that (at least) one of them is a boy, and ask you what you think is the probability that I have two boys.”) a reasonable model is he picks a kid of his at random and tells us its gender. He’d say he has a boy 50% of the time, and half of that time his other kid will also be a boy, as the kids’ genders are independent. Conditional probability doesn’t enter into this model. What would be the model where conditional probability was appropriate? It’s hard to come up with one which matches the wording of the puzzle, which is why I think using this puzzle for showing how conditional probability works is a mistake.

Keith Devlin’s article goes on to analyze the new “Tuesday birthday” puzzle:

I tell you I have two children, and (at least) one of them is a boy born on a Tuesday. What probability should you assign to the event that I have two boys?

once again blindly trotting out conditional probability. But let us first ask ourselves if it’s appropriate, what the proper model is. The least surprising model IMHO is that he picks one of his kids at random and tells us that child’s day-of-week and gender. In this model, independence between the children again applies and the probability of two boys is 1/2. What model could have conditional probability apply? Conditional probability applies when the other possibilities in the probability space are removed from consideration, so that’d be something like… a majordomo at a large gathering of parents of two children flips a coin to choose a gender (boy) and spins a spinner to select a day of week (Tuesday), sends away all the parents who don’t have a Tuesday-born son, and selects one of the remaining to tell you that they have a Tuesday-born son. The conditional probability of a second son is indeed 13/27, but models where conditional probability applies to the puzzle are farfetched, so applying conditional probability is an error. In this puzzle intuition is correct after all, the proper answer is 1/2.

[Update: Keith Devlin has pre-emptively addressed some of my criticism in his next post, The Problem with Word Problems, by saying that “I tell you X” is word-problem code for “the probability space is conditioned on X”. I still don’t like equating conditioning on “X” with conditioning on “he said X”: As above, if you have “One of my two kids is a boy, therefore the probability of my children being the same sex is 1/3.” then you can’t have “One of my two kids is a girl, therefore the probability of my children being the same sex is 1/3.”. I find the embrace of this asymmetry absurd.]

13-Jun-2010

On Alvin Greene

Filed under: politics — jlm @ 09:09

One of the news stories from Tuesday’s many primary elections which has stayed in the news is about unemployed veteran Alvin Greene trouncing unpopular member of the party machine Vic Rawl in the race for South Carolina’s Democratic nominee for Senate, 59-41.

Greene self-funded, meaning he used his own savings to pay the filing fee: his campaign consisted of calling up his friends and asking them to spread the word to their friends. Now Rawl is asking how Greene, a political nonentity, who didn’t campaign, could have defeated him. (Fox News, New York Times, The Root) Part of the blame for Rawl’s defeat surely goes down to how his own campaign was itself minimal: Charleston City Paper. But from reading these stories, I think there’s something simple which is getting lost: Rawl was unpopular. (From Public Policy Polling[pdf]: “Do you have a favorable or unfavorable opinion of Vic Rawl?” — favorable 5%; unfavorable 14%; not sure 82%)

To defeat someone unpopular, an unknown needs only to run. And with his unfavorable numbers at triple his favorable, Rawl should consider himself lucky to have picked up 41% of the vote! I’ve seen this happen several times with minor local offices, and sometimes even judgeships. The only news here was that it was an up-ticket race: US Senator.

Greene’s opponent come November is incumbent Sen. Jim DeMint, who doesn’t have the millstone of unpopularity around his neck that Rawl did. Greene was able to win round one just for showing up, but in round two he’s up against a goliath.

23-May-2010

RIP Martin Gardner

Filed under: obit — jlm @ 16:18

Passed away yesterday, at 95.

I remember Gardner’s “Mathematical RecreationsGames” well, it was great at showing sides of math not seen in school. I have many of the books collecting those articles, plus others of his like Ah! Gotcha. They had a big impression on me during my formative years — Gardner’s love of mathematics was infectious.

Scientific American has republished a profile of him.

(Edit: Wired had it right, the Scientific American column is indeed “Mathematical Games”. Mathematical Recreations is a book honoring Gardner and the “Mathematical Games” column.)

21-Apr-2010

Death Valley gallery

Filed under: travel — jlm @ 17:02

I put a photo gallery from my Death Valley trip up. (Wishing for a left-handed camera…)

20-Apr-2010

Day in the life post

Filed under: biking, so. cal — jlm @ 13:50

Not my usual day, so perhaps postworthy.

Last night I tried to head out to Hollywood, but my car wouldn’t start. Didn’t even do the “rrr-rrr” thing, and I verified I hadn’t left the lights on. So much for hitting the nightlife, spent the evening online instead. This morning was my annual physical, including a cholesterol test, so I was fasting that evening/this morning. I didn’t sleep well, woke up at 5 o’clock, had trouble getting back to bed and my alarm woke me. (Usually I have no problem sleeping and I wake on my own before the alarm.) The physical went fine, weight stable, no alarm bells, the doctor gave me cream for my athlete’s foot and probiotics for my GI. Get home, call Gabriel Towing, they refer me to Hillcrest Towing, who say they’ll be there in 15 minutes, and 15 minutes later they’re there. Amazing, I’m impressed, I’ve never had a tow come promptly before. Go to Subway for a lunch to break my fast (any “healthiness” of their sandwiches likely lost by me opting for the drink & cookies, ’cause I’m hungry). Bike around southern Pasadena, a light rain starts, I get twisted around by the twisty streets around Oak Grove and Oak Knoll. It’s a lot harder to navigate under low cloud cover, can’t see shadows or the mountains. Go home, wet, and the dealer says the non-hybrid-system engine battery bricked, looks like I’m out around $200 for repairs and towing.

18-Apr-2010

Economic musings

Filed under: econ, philosophy — jlm @ 20:05

There’s been noise and worry about deflation lately. The fear deflation sparks has always seemed strange to me, with the industry I know best — electronics — being both very energetic and highly deflationary. I’m familiar with the theory of how a deflationary spiral saps economic growth, with Japan’s economy being the prime example.

But looking closer at Japan’s economy: It was “stagnant”, by which economists mean its production was steady, and that steady production was actually quite high. If it weren’t economics, meeting an objective well and steadily would be considered very good, but the norm for economics is growth, so steady first-world level production is considered a failure. (Coming out of a deep recession, a steady production level doesn’t sound that bad after all.)

How much production do we want? Is this even the right question? I’m remembering Dijkstra’s complaint that programmers were proud of how large a program they had written, when instead they should have been ashamed at needing so much code to accomplish their goal. Is GDP as faulty a metric as LOC? Instead of being proud that we produce $47,000 while Japan only manages $33,000, should we instead be looking at why it takes us $47,000 to have a full and fulfilling life while Japan only needs $33,000? Are we really full and fulfilled with our lives, and getting a better life out of that $47,000 than Japan is out of its $33,000? Is increasing that number to $50,000 the best way to improve our lives? Or is there a better measurement that we should be looking at? Certainly more GDP helps immensely, it gives us more resources to spend on our goals, but I worry that treating GDP itself as the goal, we foolishly sacrifice “life value” for GDP, instead of spending our GDP to improve our lives.

13-Mar-2010

Urban peafowl sighting

Filed under: animals, biking, so. cal — jlm @ 13:11

I was just biking along Hill, by the PCC lots, and there braving traffic was a peacock and peahen. Bright blue necks, long (furled) train on the cock, quite the surprising sight, traffic stopped for the spectacle of them crossing the street.

7-Mar-2010

Prisoners’ dilemma and web advertising

Filed under: web — jlm @ 19:20

I use AdBlock Plus; I installed it after certain weight loss ads featuring ugly caricatured jiggling fat bellies started appearing all over the web, making browsing disgusting. Many people find that web ads make their browsing experience unpleasant in various ways, so adoption of AdBlock Plus and other blockers have been increasing significantly recently. This in turn has web publishers worried, because they see all these visitors showing up on their sites with the ad blockers, so they’re not getting the ad revenue they planned on, and that means more red ink on their balance sheets. (See this current Ars Technia article.) But when I turn ABP off, the web becomes intolerably hostile: Sites are slower, and when they do load they’re full of flashing ads and I get ad copy playing over my speakers and popping up over the text I came to the page to read. So now visitors are fed up, they block all that crap, and publishers can’t make their ad money, they shut down, visitors have no where to go, and everybody loses.

What happened? It was supposed to be visitors get “free” content because the publishers are ad supported, and this worked for a while. But this relies on the implicit social compact that visitors don’t block ads, which has as its unstated counterpart that publishers don’t make their ads so intrusive that visitors get annoyed by them. So I think we have a situation similar to a prisoners’ dilemma: visitors can defect by blocking ads, improving their own browsing experience but denying publishers their ad revenue; publishers can defect by showing intrusive ads, bringing in more revenue but destroying the visitors’ browsing experience. (It’s not quite a PD, because the payoffs are the same when the visitors defect regardless of whether the publishers do.) It seems to me that the publishers defected first, coveting the additional money from the worse ads, and we’re now seeing a tit-for-tat from the visitors, fueled by annoyance at the publishers’ defection.

And my experiment of turning ABP off shows that the publishers are still defecting, and defecting “harder” than before. Whatever your position on the morality of browsing with ads blocked, I think we’re going to see more visitors turning to blocking as long as intrusive ads are ubiquitous in web publishing. If the publishers go back to simple ads, which don’t move and flash and pop-over and piss users off, then publishers won’t see new internet users installing ad blockers, and us existing ad block users will be more inclined to turn our blockers off. But we’re going to continue defecting as long as the publishers are.

The Ars Technia article is titled “Why Ad Blocking is devastating to the sites you love”, but from the other side it could be called “Why intrusive advertising is destroying your users’ tolerance for your business model”. The publishers defected long before the visitors did, and turned a deaf ear to user complaints, and so I’m unsympathetic to cries from them that we’re now defecting too. Publishers have been pissing in the pool for years, and now they’re surprised users are putting on full-body wet-suits? Blame us ad-blockers all you like, but until you look in the mirror and make the web tolerable to surf without an ad blocker, the situation won’t get better for either of us.

4-Mar-2010

twitcode: automatic AFS Kerberos ticket renewals

Filed under: programming — jlm @ 17:29

If you’re a user of AFS with Kerberos, you’ve no doubt been annoyed at your ticket expiring and having to run aklog to get a new one. You may have tried to script running aklog automatically periodically, but been stymied that aklog won’t refresh an about-to-expire ticket. So, you have to scrape the output of klist for the expiration time and wait until then — easy enough. So simple that my script to do so, nicely indented and with declared integer variables for now and then weighed in at a paltry 353 bytes.

So, I figured if I inlined everything, I could fit it in a twitter post. And lo, it does:

while sleep $((`date -d "$(klist -5 | tail -1 | awk '{ print $3 " " $4 }')" +%s` - `date +%s`)); do aklog; done

That’s 112 characters, including unnecessary spaces for readability and the trailing newline, leaving plenty of room for extras you might think up.

Now, if only I could fit all this commentary in a twitter post…

16-Feb-2010

Dire time for webcomics?

Filed under: web — jlm @ 13:06

This recession has been bad news all around, with many businesses shuddering their stores and swathes of jobs lost. I’ve noticed several webcomics I’ve been following for a long time have recently shut down too. Bruno the Bandit announced its effective cessation mid-story. Angband: Tales from the Pit had an ending storyline for its wrap up. User Friendly has gone into repeats. Homestar Runner has quietly stopped updating. There’s a constant rise and fall in this “industry”, low entrance costs means competition is heavy, but I’m sad to see long-time players bow out in succession.

Powered by WordPress