April 30, 2004

Life imitates art

Josh Marshall notes Bush's suggestion that people who don't like his Iraq policy are racists:
There's a lot of people in the world who don't believe that people whose skin color may not be the same as ours can be free and self-govern. I reject that. I reject that strongly. I believe that people who practice the Muslim faith can self-govern. I believe that people whose skins aren't necessarily -- are a different color than white can self-govern.

Am I the only one who's reminded of this line from Full Metal Jacket

We are here to help the Vietnamese, because inside every gook there is an American trying to get out.

Now, obviously all humans have pretty much the same brain firmware, so it's not like there's any inherent reason that Iraqis aren't capable of self-government, at least given the right cultural background. But that doesn't mean that we can impose it on them.

Posted by ekr at 12:11 PM | Comments (72) | TrackBack

April 29, 2004

Canadian history

Marcus Leech pointed me to this hilarious article from The Toque about how Canadians don't understand their history.
Canada is a nation rich in heritage, but are our young Canadians forgetting about our roots? Are our children spending too much time floating in the pop culture flotsam that drifts through our country, that even the most common historical Canadian facts are washed away on a wave of cultural apathy? What does it say about the sorry state of our education system when seven out of ten children cannot even name the Canadian president? When 95% of our kids are able to recognize Pamela Anderson, while less than 10% can identify the face of Sir Lanny MacDonald, it is a sure sign that we are losing the ties to our strong Canadian past.

Of course, as everyone knows Canada doesn't have a president. In fact, a good way to annoy Canadians is to ask "If Canada's a real country, who's your president?"

Also, check this out.

Posted by ekr at 10:19 PM | Comments (64) | TrackBack

Automatic license plate reading

The city of Manapalan FL has decided to install cameras that automatically scan people's license plates and run background checks on them [*]. Remind me not to take my stolen car to Florida.
Posted by ekr at 09:43 PM | Comments (2) | TrackBack

April 28, 2004

The IETF mentality

More or less all you need to know about the way IETFers think is contained in the following two facts:

1. People at IETF meetings wear t-shirts that read:

"We reject kings, presidents and voting.
We believe in rough consensus and running code."
- Dave Clark (1992)

2. The IETF has an actual published standard (RFC 2777) that describes the random number generation algorithm used to select the committee that chooses IESG and IAB members.

Posted by ekr at 07:29 PM | Comments (42) | TrackBack

Is there anyone who doesn't have my email address?

I know a lot of people who worry about denying their email address to spammers. They lie about it when they fill out forms, or maybe have some temporary address they use only for commercial transactions. They don't put it on their web page or obfuscate it when they do. Now, this may make some sense if you don't send a lot of email, but my email address is splattered all over the Internet from a zillion mailing list postings, Internet-Drafts, etc.

My guess is that trying to conceal my email address at this point is fairly pointless. I already get about 1000 spams a day. It's hard to believe there's any spammer on the planet who doesn't already have my address. I guess one question to ask is how efficient the address-trolling systems are. If they're very efficient, then you'd expect to see the expected number of spams rise very quickly as a function of the number of times your address is published and then level off fairly soon as everyone has at least one copy. On the other hand, if they're less efficient then the curve will be a lot slower and may never level off.

As far as I know, noone has actually studied this question, though. If there's any work on this topic, I'd be interested to see it.

Posted by ekr at 07:14 PM | Comments (23) | TrackBack

April 27, 2004

Gene therapy for Alzheimers

This is interesting. A very small trial suggests that gene therapy has some promise for treating Alzheimers.
Posted by ekr at 09:32 PM | Comments (7) | TrackBack

You look just like Daddy

This New Scientist article reports the not-entirely-surprising result that women tend to marry men who look like their fathers. The result holds even for adoptive parents, so it looks like it's a matter of picking someone who looks like whoever filled the father role, not of picking people who share your genes.
Posted by ekr at 09:24 PM | Comments (3) | TrackBack

Another HUD technology

The BBC has an article on Microvision's Nomad heads-up display technology. Instead of being a screen, the Nomad uses lasers to scan the retina, so you can overlay the displayed information on your vision instead of having it replace your ordinary vision. It's only monochrome (displays is red) but has high resolution (SVGA). Still doesn't pass the "not stupid looking" test, but I guess some miniaturization would fix that...
Posted by ekr at 07:27 AM | Comments (3) | TrackBack

April 26, 2004

Speedbump DRM and downloading incentives

Ed Felten on talks about "speedbump DRM".
A real speedbump doesn't stop drivers from following a path that they're deterrmined to follow. Its purpose, instead, is to make one path less convenient than another. A speedbump strategy for copyright holders, then, tries to make illegal acquisition of content (via P2P, say) less convenient than the legitimate alternative.

There are several methods copyright owners can (and do) use to frustrate P2P infringers. Copyright owners can flood the P2P systems with spoofed files, so that users have to download multiple instances of file before they get a real one. They can identify P2P uploaders offering copyrighted files, and send them scary warning messages, to reduce the supply of infringing files. These methods make it harder for P2P users to get the copyrighted files they want -- they acts as speedbumps.

These kinds of speedbumps are very feasible. They can make a significant difference, if they're coupled with a legitimate alternative that's really attractive. And if they're done carefully, these measures have the virtue of inflicting little or no pain on noninfringers.

This last point is really critical. If we want people to do X instead of Y, we need to ensure that X is more attractive than Y. There are two ways to change the rate of copyright violation: make it more attractive to obtain content legally and make it less attractive to obtain it illegally. So far, the content companies have focused almost exclusively on the second, and had to be dragged kicking and screaming into allowing an even halfway decent downloadable music store. Believe it or not, there are now stories that the record labels want to jack up the price of songs.

Presumably the content industry thinks that the current rate of copyright violations is too high. If they want to lower it and simultaneously increase the price of downloads, they are going to have to make downloading much less attractive. Considering that, as Ed points out, their ability to do so technically is quite limited, the only remaining option is to increase the legal pressure. Maybe the death penalty for music pirates will do the job.

Posted by ekr at 08:38 AM | Comments (61) | TrackBack

10 guilty men and all that

Jennifer Granick addresses the question of false convictions [*]:
A policy question remains. How many false positives (i.e. false convictions) is society willing to tolerate to have an otherwise functional justice system? One District Attorney quoted in the New York Times article on this study said "We all agree that it is better for 10 guilty men to go free than for one innocent man to be convicted. ... Is it better for 100,000 guilty men to walk free rather than have one innocent man convicted? The cost-benefit policy answer is no."

I, of course, think that's wrong. Criminal prosecution destroys a person's life and family. If an error can be prevented, it should, and if an error occurs as a regular function of the system, e.g. by relying on unreliable evidence like uncorroborated confession, then the system should change.

I think it's more complicated than that. It's true that being prosecuted is very unpleasant, but then so is being a victim of crime. From the economic perspective, at least, (required reading: The Economic Approach to Human Behavior) the purpose of prosecuting crime is to lower the overall crime rate. After all, we could easily reduce the false positive rate to zero by prosecuting nobody. Of course, then we wouldn't have any law enforcement either.

So, the question I have is, what level of enforcement is the best policy given the elasticity of crime with respect to enforcement and our capabilities with respect to the accuracy of the process? I don't know what the answer is (though I suspect it's a lot higher than a false positive rate of 1/100,000), but admitting that we're willing to accept a certain rate of false positives--and that that's an inevitable consequence of law enforcement--is the first step towards working out what that rate should actually be.

Of course, none of this is to say that we shouldn't strive for increased accuracy. It's particularly attractive to find accuracy improvements which don't lower the true conviction rate. It's in that context where the eyewitness studies that Granick cites are so important.

Posted by ekr at 07:42 AM | Comments (46) | TrackBack

April 25, 2004

Non-monetary incentives

A while back I injured my ankle running and I've been nursing it abck to health with the usual combination of ice and NSAIDs. After a few months on ibuprofen I decided that it might be easier on my gut to convert to a COX-2 inhibitor. The best choice seems to be Bextra (valdecoxib), which has the additional advantage of once-a-day dosing.

Like most insurance companies, mine doesn't have any of the COX-2 inhibitors on the formulary, so you've got to jump through some hoops. Unfortunately, the pharmacy thinks that I need prior approval and when I call the insurance company they tell me that I don't need approval but that they'll only pay 50% of the cost of the drug. Apparently, the pharmacy gets their information from some different place from the health insurance reps on the phone. So, the pharmacy can't fill the scrip without prior approval, but my doctor can't give prior approval because whenever they call the insurance company they get told it's not necessary. Next step is to actually have the insurance company call the pharmacy.

Now, I'm not saying that the insurance company does this on purpose, but if I wanted to disincentivize customers from asking for non-formulary drugs, one of my first steps would be to make the process as inconvenient as possible.

Posted by ekr at 05:17 PM | Comments (2) | TrackBack

April 24, 2004

Why is the retail price of milk controlled?

According to the SF Chronicle, milk is about to go up by $0.50/gallon. This isn't happening because of normal market reasons but because the California Department of Food and Agriculture, which controls the price, has decided it should be:
The California Department of Food and Agriculture has ordered milk processors to start paying dairy farmers in Northern California an additional 47 cents per gallon beginning May 1, an increase that is likely to be passed to consumers.

Department spokesman Steve Lyle said it is the highest month-to-month increase on record. It will bring the farm price of raw milk in Northern California to $1.83 a gallon, setting another record. The department raised Southern California prices to $1.85.

The department cited several reasons for the sudden increase, including slight decreases in the size of the dairy herd, flat milk production and strong demand for milk, butter and cheese. The forecast suggests that tight supplies could continue through the rest of 2004.

The department also monitors retail prices to ensure that milk isn't sold below cost as an unfair loss leader. In April, the minimum price for a single gallon of whole milk in Northern California was $2.38, while the floor price for a two-gallon pack was $4.75.

Ok, I get why we have price controls on milk. I don't like it, but it's part of the general pattern of large-scale rent-seeking by American farmers. But why exactly do we need to be in the business of controlling the retail price? As long as the price paid to farmers is constant, why does it matter what the retail price is?

Posted by ekr at 07:49 PM | Comments (38) | TrackBack

Maybe we shouldn't treat people with smallpox

This week's NYT magazine has an article about a girl who came into the SwedishAmerican hospital in Rockford Ill. with monkeypox, a relative of smallpox. Only a few doctors were willing to treat her. The article is fairly balanced but basically treats the first responders who volunteered as heros for ignoring their own personal risk, and the risk to their families.

I'd like to take a more global view. Sure, it's heroic to help other people at personal risk, but remember that there are more people involved than the patient and the health care provider. The problem is that health care providers are a major vector for the disease to spread from one patient into the community at large. With something really communicable that has only infected a few patients, we have to ask whether the cost/benefit analysis supports treating them. At the very least, we need to consider quarantining the first responders as soon as they are potentially infected rather than waiting for them to actually get sick.

Posted by ekr at 07:57 AM | Comments (39) | TrackBack

April 23, 2004

Paris Metro Pricing

Matthey Yglesias writes about using differential prices to provide price discrimination for initially identical goods:
Now I remember that one of the several economists on my mother's side of the family -- my grandfather, I think, but maybe my uncle Paul -- once said that he thought gas stations should try offering different prices. You've got an Exxon station selling gas at $1.50 a pop and I own the Gulf place across the street and set my price at $2.00 -- am I crazy? Maybe not. Sure, "everyone" will go to your station, except that once everyone's there, you're going to have a very long line. People who are willing to spend more in order to save time will go to my station. You'll have higher volume, but I'll have higher profits-per-unit.

At any rate, I always thought that was an interesting idea -- make people pay for the higher prices. So I tried it out on this blog. Ads on the right cost more than ads on the left. Result -- more people buy the ads on the left. But that makes the right less crowded, and hence more desirable as an advertising medium and some people pay for the more expensive ads. Strange but true.

Apparently this is exactly the price scheme that the Paris Metro uses and Andrew Odlyzko has proposed a similar approach for controlling access to congested networks. Of course, this only applies for goods where there's a lot of congestion--as more people desire them the quality of the good goes down. It doesn't work well for, say, cans of Diet Pepsi at your local Safeway (though charging people for access to certain checkout lines certainly would work). What I'd really like to see, though, is for-fee access on certain freeway lanes.

UPDATE: Fazal Majid says in the comments section that the Paris Metro abandoned this kind of tiered pricing in 1981.

Posted by ekr at 08:16 AM | Comments (36) | TrackBack

April 22, 2004

Remind me again why I need quantum cryptography

Oh goody, someone did a commercial transaction using quantum cryptography [*]. The QC guys talk a lot about how it's absolutely secure, but check this out:
When these photons arrived at their destination, their state of polarisation was observed. This provided both ends of the link with the same data, either a one or a zero. In this way, it is possible to build a cryptographic key with which to secure the full financial transaction.

Quantum entanglement ensures the security of communications because any attempt to intercept the photons in transit to determine the key would be immediately obvious to those monitoring the state of the other photons in each pair.

And because the resulting key is random it can be used to provide completely secure link even over an unprotected communications channel, provided a new key is used each time.

This system can be guaranteed secure. By contrast, most existing non-quantum cryptographic systems rely on extremely time-consuming mathematical problems to create a code that are impractical - but not impossible - to break.

Uh, so, you use the quantum crypto to generate a key for your symmetric cryptosystem. So, it's true that the exchange of the symmetric key is "guaranteed secure", but as soon as you use it to secure some actual data, it's open to attack via the usual cryptographic techniques. And since it's the transaction you're trying to secure, not the cryptographic key, that's a pretty marginal improvement.

And then to make things more entertaining:

"If you are talking about large sums of money, people are interested," Zeilinger told New Scientist. He adds that the system should not be much more expensive to implement than current technology.

Except, of course, for the fact that it's only usable over distances of about 20 kilometers:

The photon-encrypted money transfer saw the Mayor of Vienna transfer a 3000-Euro donation into an account belonging to the University of Vienna team. The two buildings are just 500 metres away from one another, but fibre optics had to be fed through 1.5 kilometres of sewage system to make the link.

Zeilinger says in principle it should be possible to extend this link to 20 kilometres. Beyond this distance it becomes difficult to transmit single photons reliably.

I guess it's not more expensive, provided that you only want to do transfers over distances of no more than 20 kilometers. And then, of course, there's the minor detail that you need a dedicated point-to-point fiber optic link between the communicating parties. Other than that it's a great technology, though...

Posted by ekr at 09:33 PM | Comments (41) | TrackBack

The TCP windowing/RST flaw

A lot of electronic ink has been spilled about the TCP vulnerability discovered by Paul Watson. See here for a particularly overwrought treatment. Unfortunately, amid all the alarmism it's hard to tell how bad things really are.

The basic problem is really simple and obvious: TCP doesn't have any cryptographic protection. That means that if someone can observe your TCP connections (such a person is called on on-path attacker because they're on the path from one peer to the other) they can impersonate one of the peers to the other and inject data into the connection or shut it down. This has been known about ever since the initial design of TCP. It's intentional, at least in the sense that no attempt was made to stop it. The general consensus in the Internet community is that if you want to prevent attacks by people who are able to directly observe your network traffic you should use encryption, probably IPsec, SSL/TLS, or SSH.

However, it's been known for a long time that there are a variety of blind attacks that can be mounted by off-path attackers. The most well-known such attack is TCP sequence number guessing. For background, it's worth describing the sequence number guessing attack:

We start with the objective. Back in the bad old days, it was very common for UNIX systems to base security decisions on the client IP address. In particular, the "r" protocols (rlogin, rsh, etc.) could be configured to let you log in to a server purely with no password purely on the strength of having the right client IP address. So, it could be very useful to be able to impersonate a machine you didn't have access to.

To understand the attack you need to know how TCP connections are initiated. The process, shown below, is called a "3-way handshake".

In TCP each byte has a "sequence number" which allows it to be addressed for the purposes of ACKs. In order to prevent confusion between multiple connections between the same host/port pairs, these sequence numbers do not start at 0 each time. Rather, each side of the connection chooses an Initial Sequence Number (ISN). In the first message of the handshake, the client tells the server his ISN. In the second message, the server acknowledges the client's ISN and tells the server his own. In the third message, the client acknowledges the server's ISN and the peers are ready to communicate.

As long as you know the server's ISN, then, you can forge a connection to the server. However, for a long time TCP implementations used predictable ISNs (basically a counter). This made it fairly easy to mount an attack, as shown below:

Effectively, the attacker initiates a genuine TCP connection with the server, but this connection is just a probe to get the state of the server's ISN. The attacker then poses as another host and initiates a second connection. The attacker can't see the server's response but that doesn't matter because it can guess the server's ISN and therefore forge a plausible-looking ACK packet. This doesn't work every time because the ISN isn't totally predictable, but in old TCP implementations it was predictable enough that you could typically forge a connection with a fairly small number of attempts. At this point, the attacker can send any data it wants to the server--though of course it can't see any of it.

This is only the simplest of a variety of attacks that you can mount. For instance, if you know about a connection between a client/server pair and you can guess the ISNs, you can probably inject traffic into the connection and it might be accepted as valid. Actually, if you're really clever you might be able to hijack the connection by killing one peer (using other methods) and then impersonating it to the other one. Certainly, you could send an RST (reset) packet which will kill the connection.

These attacks are made a lot easier because TCP is a windowed protocol. What this means is that at any time the sender send a large chunk of data (say 32K) without receiving any kind of acknowledgement from the receiver. This data would be spread over multiple packets in an unpredictable way and those packets may be reordered so the receiver has to be prepared to handle data with sequence numbers anywhere within that window. This makes the attacker's life easier because he doesn't need to know the ISN exactly or exactly how much data has been sent. Just getting close is good enough to generate a plausible looking RST packet.

As I say, these attacks are very old and the fix is well-known: randomize the ISN. If the ISN is random, then the simple sequence number guessing attack becomes essentially impossible. Getting the randomization right turns out to be somewhat tricky and some implementors have botched it, but basically it's a good defense.

Which brings us to Watson's attack. Remember the TCP window? Well, the faster the network you're transmitting over, the wider the TCP window has to be to get good performance. As a consequence, TCP windows have been creping up over the years and are now fairly large (32K is typical). Mr. Watson's observation is that with windows this wide, the probability of selecting a sequence number in the Window is actually fairly good. If you think of dividing the TCP sequence number space (2^32) into 32K chunks and then systematically trying a sequence number in each of those chunks, you only have to try 65536 sequence numbers before you get one the recipient will accept. This is fairly doable.

So, how important is this attack? Well, you can think of it as having two modes: denial-of-Service (terminating people's connections) and data injection. The data injection mode probably isn't very important. The injection process is sloppy and unpredictable and so you probably can't make targeted changes to someone's connections. More importantly, anyone who cares about connection integrity should be using a security protocol like SSH, SSL/TLS, or IPsec.

The DoS attack is potentially somewhat important. It's probably not particularly useful to kill individual people's connections. It's too difficult and there are a lot of much easier ways to attack individual people. However there has been some concern that this attack would be used to attack the Border Gateway Protocol (BGP) connections that Internet Service Providers use to communicate routing information. If you did enough damage to routing, you could probably screw up the Internet fairly badly.

Even then, though, this attack has to be put into perspective. There are lots of other ways to DoS the Internet (attacking the main DNS server or writing a really bad worm would be a good start). Back in 2003, there was a vulnerability that would let you totally DoS a Cisco router (the most common kind of Internet router) with a single packet. In addition, there are some farily simply countermeasures that are already being put in place. [*]. Truth be told, this probably isn't even the worst serious vulnerability that's been found this month. [*].

At times like this, it's important to remember the sage words of Agent Kay:

There's always an Arquillian Battle Cruiser, or a Korilian Death Ray, or an intergalactic plague that is about to wipe out life on this miserable planet. The only way these people can get on with their happy lives is that they do not know about it.
Posted by ekr at 07:58 AM | Comments (37) | TrackBack

April 21, 2004

Good news for people with asthma

I happened to notice today that the patent on one of the mainstay anti-asthma drugs fluticasone (brand name: Flovent, or Flonase in the nasal version) runs out in May 2004. It's a very popular drug and so I'd expect to see I'm definitely looking forward to saving some money on my meds.
Posted by ekr at 09:16 PM | Comments (48) | TrackBack

Airport fate-sharing

Liliputian Lilith has a sane-seeming set of guidelines for airport personnel, including:
Passenger is entitled to receive boarding pass at this juncture, even if there's a slight risk (due to some strange idea about how long it takes to screen checked baggage and get through security check to gate - definitely less than an hour for both procedures) that checked baggage will not arrive on same flight as passenger. And by the way, it did.

Sadly, I suspect that the reason that they won't issue you a boarding pass when they're not sure you can't get on the flight isn't that they care whether your bags arrive or not. Rather, it's what's called "positive bag matching." The idea is that having your bag on the same flight as you disincentivizes you from having your bag explode--thus killing the unsuspecting passengers on the plane--since if the bag is on the same plane as you, you are likely to be involved in said exploding event, thus vastly decreasing your utility. Of course, the available evidence suggests that at least some terrorists consider the disutility of exploding to be much less than the utility of blowing up a bunch of strangers, so it's not clear how effective this procedure it. What's really entertaining is what happens when someone who's bags are on the plane doesn't board, at which point they have to un-load the entire plane so they can find the non-boarder's bag.

Posted by ekr at 07:21 AM | Comments (32) | TrackBack

April 20, 2004

WEIS 2004 Agenda

The agenda for the Workshop on Economics and Information Security 2004 is up. I'll be presenting my paper "Is finding security holes a good idea?" (draft available here) and it looks like there are some other interesting papers there as well. If you're coming to the conference, drop by and say hello.
Posted by ekr at 09:05 PM | Comments (2) | TrackBack

Important bicycle maintenance tips

If you use Speedplay pedals, wear the rubber cleat covers so that your cleats don't get all scraped up.
New cleatWorn cleat

Because you've ruined the heads of the screws by walking around on them, they're very hard to extract and too damaged to put back in.

The screws that come with your new Speedplay cleat kit are too short to mate with Carnac shoes, thus making it likely you'll have to purchase an entirely new Carnac insert kit to get those 8 screws.

Check your cleats regularly to see that they're not too worn. Otherwise, the exposed cleat spring (see worn pedal above) wears a deep groove in your pedal spindle, thus destroying your $165 pair of pedals (the left and right pedals aren't interchangeable).

When removing your pedals, try to avoid gashing your hand on the chainring. It hurts. Safety tip: put the chain on the ring closest to your hand. That means shifting the bike in between pedals. Consider wearing heavy gloves.

Otherwise, have Band-Aids and Polysporin on hand.

Posted by ekr at 08:02 AM | Comments (3) | TrackBack

April 19, 2004

Civil liberties as an externality

Caught Eric Muller from IsThatLegal? on NPR today talking about the Guantanamo, Hamdi, and Padilla cases. The question on the table is what the government is whether the government is allowed to incarcerate certain offenders without trial or counsel. Anyway, I got to thinking: we all want to be safe from criminals, right? And law enforcement is obviously essential to having that safety. In principle, the more vigorous the law enforcement the more safety we'll have. The problem is that vigorous law enforcement has negative externalities--the easier it is to hold and interrogate suspects, the more likely it is that innocent people will be apprehended by mistake.

In the economics literature, there's a standard technique for dealing with negative externalities: a Pigouvian tax. There's no reason we can't use a similar technique here. We simply charge the authorities to arrest/detain people

This scheme has several benefits:

  1. It compensates the suspect for the inconvenience of being arrested. Remember, they might be innocent.
  2. It incentivizes the authorities to detain people only when they really think it's worthwhile, thus making mass arrests difficult.
  3. It lets the authorities detain someone even with very weak evidence if they're really sure.

One objection to this scheme is that it makes it difficult to arrest people even when the evidence is very strong. If you're a strict utilitarian, that may not be a bad thing, but if you're not it's easily dealt with: have a sliding scale based on the strength of the evidence. The payments starting small in the case where the police have probable cause and increase in size as the grounds for the arrest get weaker. Obviously, we'd still need to assign the size of various payments, but because the gradations are finer, it's much less important exactly where the lines get drawn.

Of course, if you're a civil libertarian, assigning values to rights violations probably makes you fairly uncomfortable, as it means admitting that civil libertarians aren't absolute, but we all know that's true. In fact, the Supreme Court has an intricate set of procedures for deciding when people's rights can be overriden. I'm just proposing a different--and more efficient--way of making that decision.

Thanks to Kevin Dick for suggesting the title of this post.

Posted by ekr at 09:41 PM | Comments (55) | TrackBack

RCS ids and unused variables

Like a lot of other C programers, I like to embed an RCS id in my code. The standard clause is something like this:
static char *RCSSTRING="$Id: ssldecode.c,v 1.9 2002/08/17 01:33:17 ekr Exp $";

This lets you know what revision of a file is represented in a given program. Hypothetically, if a user then has a problem you can ask them to get the RCS ids using strings(1) and then you know exactly what version they're running.

The only problem here is that the variable RCSSTRING isn't used anywhere in the code. It's just hanging out. So, if you tell your compiler you want it to check for unused variables (-Wunused on GCC), you get a complaint like this:

x.c:11: warning: `RCSSTRING' defined but not used

So, you've got three options:

  • Ignore the warning. This is a problem because you'd really like to use the -Werror flag, which causes all warnings to turn into errors, thus making you very careful.
  • Not use -Wunused, which is kind of annoying because it's actually a useful warning.
  • Find some way to suppress the warning for this case only

It turns out that this is a pretty common problem with a common solution, though one that took me about 30 minutes of digging around to find. As a public service I'm describing it here. GCC (does anyone use any other compiler on UNIX?) provides you with an "attribute" that tells the compiler not to emit a warning even though this variable is unused, like so:

static char *RCSSTRING __attribute__((unused)) = "$Id: ...";

Now you know what to do if you're ever in this situation.

Credit: I learned about this trick from reading the tcpdump source code.

Posted by ekr at 10:40 AM | Comments (37) | TrackBack

April 18, 2004

The New A-Team

Movie remakes of old TV shows is all the rage now. I was a big "A-Team" fan as a kid (construction and shooting is a pretty cool combination) and I got to thinking. Here's who would be in my A-Team remake.
Role Original Actor New Actor
Hannibal Smith George Peppard Jack Nicholson
B.A. Baracus Mr. T Ving Rhames
Face Dirk Benedict George Clooney (or Ben Affleck)
Murdock Dwight Schultz Vincent D'Onofrio (or Ben Stiller)

I think I'd pay $9 to see that...

Wendy McKibben suggested Ben Stiller

Posted by ekr at 09:38 AM | Comments (61) | TrackBack

April 17, 2004

Well, that was a waste of two hours

So, I wake up this morning to find that my mailbox is full of spam. What the heck?! I just installed a spam filter! A little grovelling through the procmail logs reveals a bunch of lines like the following:
procmail: Program failure (-25) of "/usr/local/bin/crm"
procmail: Rescue of unfiltered data succeeded 

So, CRM 114 is choking and procmail is helpfully delivering the spammail. But why?

Lacking any actual intuition as to what's going on, I try to replicate it. Unfortunately, CRM 114 works great from the command line, even on all the same messages. Ok, that's not good.

When you don't have any intuition, it's time to get scientific and so I start to dig deeper. Is there some reason why CRM 114 should be returning -25? A quick grep through the source code doesn't reveal any places where that should happen. Maybe procmail's screwing up. Ok... how about we wrap the program and see if it's really returning 25. Yep, it is. Hmm...

As you may have guessed, this investigation went off the rails a while back. A little more research and a little less divide-and conquer would have paid off handsomely. If I'd read the procmail man page, I would have noticed the following:

Program failure (nnn) of "x"
Program that was started by procmail returned nnn instead of EX- IT_SUCCESS (=0); if nnn is negative, then this is the signal the rogram died on.

Unfortunately, I didn't notice this and I also didn't notice that while procmail was reporting -25 my wrapper was reporting 25, which should have been my second clue. Chalk it up to having only 6 hours sleep.

Even in my brain-dead state and having missed these clues, I know how to figure out what's going on: my next move is to put CRM 114 under the debugger and find the exact line of source where it's exiting. (This would have worked great, btw, since I would would have seen the program die with a signal.) But this requires a complete rebuild, etc. etc. so at this point, I do what I should have done in the first place and go to the CRM 114 mailing list archives to see if anyone else has encountered the problem. And lo and behold, one Richard Ellis has had this problem and has read the procmail man page.

So, crm died on a sig 25, which on my Linux box is actually mapped to SIGXFSZ. The meaning of that signal is that CRM exceeded the maximum file size allowed by ulimit for the user it's running under.

Short story, CRM died because it was killed by the kernel with sig 25. Beyond that, there is insufficient information to know why that happened.

Now, Mr. Ellis may not have sufficient information at this point, but I do. I check my ulimit and sure enough it's unlimited. That means that procmail or Postfix has limited it for me. A little grepping through the procmail source reveals [0] that it's not reducing the limit and when you Google for "postix rlimit", things rapidly become clearer. Turns out that Postfix enforces mailbox quotas with the rlimit() call. By default, the size is 50M. This wouldn't be a problem, except that CRM 114 uses very big databases and the rlimit doesn't discriminate. From then on it's easy to fix: just jack up the Postfix mail quota and I'm good to go.

[0] I know this because during my debugging phase I got a single, unrepeatable "File size limit exceeded" error that I failed to connect with this problem (clue #3!) but did go far enough to see if I could find in the procmail source.

Posted by ekr at 09:08 AM | Comments (49) | TrackBack

April 16, 2004

Two cheers for CRM 114

So, after my bogofilter meltdown, I decided that I needed a new spam filtering solution. After a little research I settled on CRM 114, at least in part because it has a cool name.

I'm actually fairly impressed with the performance of CRM 114. It's Train-On-Error (TOE): you only train it with messages that it misclassifies. I ran it for a day with all of the mail coming to my inbox and training the misclassified messages. After the first day I felt comfortable enough to divert the messages classified as spam to a separate mailbox and just periodically check and update. So far it's had very few small negatives (maybe 1%) and maybe 10% false positives.

That said, the installation procedure wasn't enormously fun. What's particularly weird is the combination of two features:

  • When CRM 114 has already classified a message correctly it doesn't train based on it.
  • CRM 114 uses mmaped files for its database. This has the consequence that when you update the file the output of 'ls -la' doesn't change. This is in the FAQ but not quite as obvious as you might like.

This wouldn't be a big deal except that as part of the installation I wrote some XEmacs functions to let me send messages to CRM 114 for training--but the datestamps never changed. It took me a modest amount of debugging to find out that my macros actually did work. [0] Anyone who'd like a copy of the macros should drop me a line.

After all that, though, it seems to work pretty well. Well enough that I've removed the prehistoric SpamAssassin preprocessing stage that I've been using for years (see here for why I'm not running a modern SpamAssassin) and am now using CRM 114 as my only mail filter.

[0] A related problem was that I was picking up the byte-compiled .elc files that didn't contain my changes. Can you tell I don't write a lot of elisp code?

Posted by ekr at 10:47 AM | Comments (28) | TrackBack

Professor Jameson

Brian over at the Brad DeLong's Semi-Daily Journal has an interesting suggestion:
I think wireless internet in lecture halls is great. The professor can stay at home in front of a blue screen and deliver the lecture. The students can choose some virtual representation to be the professor using some crude form of CGI to mimic the professors actual movements.

Think about it: would you rather learn economics from Professor Delong in person or from a CGI Jenna Jameson (controlled by Professor Delong) on your laptop?

Now, that would be a good reason to go to Berkeley...

Posted by ekr at 08:20 AM | Comments (2) | TrackBack

Can we please have our DDT back?

Check out this depressing article in the NYT Magazine about why developing countries with huge malaria problems don't use DDT to control the mosquitos that carry it. The reason, of course, is environmentalism:
But this time around, I was also struck by something that did not occur to me when I first read the book in the early 1980's. In her 297 pages, Rachel Carson never mentioned the fact that by the time she was writing, DDT was responsible for saving tens of millions of lives, perhaps hundreds of millions.

DDT killed bald eagles because of its persistence in the environment. ''Silent Spring'' is now killing African children because of its persistence in the public mind. Public opinion is so firm on DDT that even officials who know it can be employed safely dare not recommend its use. ''The significant issue is whether or not it can be used even in ways that are probably not causing environmental, animal or human damage when there is a general feeling by the public and environmental community that this is a nasty product,'' said David Brandling-Bennett, the former deputy director of P.A.H.O. Anne Peterson, the Usaid official, explained that part of the reason her agency doesn't finance DDT is that doing so would require a battle for public opinion. ''You'd have to explain to everybody why this is really O.K. and safe every time you do it,'' she said -- so you go with the alternative that everyone is comfortable with.

Of course, people have mostly forgotten about malaria because it's been almost totally eradicated in the developed world (thanks to DDT):

Lawrence Barat, the World Bank's adviser on malaria control, said, ''When I tell people I work on malaria, sometimes I get, 'Gee, I didn't know it still existed.'''

One of the most depressing aspects of talking about malaria is that you get to hear the phrase ''the powerful AIDS lobby,'' a term no one but a malariologist would use. AIDS in the third world is still criminally underfinanced, but at least it gets some money and a lot of attention. Malaria gets AIDS's dregs. AIDS was a sudden plague, very visible in its choice of victims, and it has a vocal constituency in rich countries. Even in Africa, malaria gets nowhere near the attention of AIDS. It has always been around, and it kills not middle-class adults but rural 4-year-olds, who don't have much of a lobby.

This is a common lament among malaria researchers. Back around 1990 I spent a few summers in a lab researching p. falciparum (the nastier strain of malaria) and even then there was enormous resentment about how all the research money went into HIV. I can only imagine what it's like now.

That said, there is a principled reason why malaria shouldn't get that much research money: p. falciparum and p. vivax are incredibly complicated organisms and have shown a remarkable ability to develop resistance--chloroquine, the standard drug in developing countries, now has resistance rates on the order of 80%. Why spend a lot of money on researching them when there's something simple, cheap, and effective that we refuse to do?

Posted by ekr at 07:47 AM | Comments (1) | TrackBack

April 15, 2004

Name that algorithm

The other day I noticed a funny thing about the way that cryptographic algorithms are named. Even though algorithm names are often not descriptive, you generally can tell quite a bit about what kind of algorithm you're dealing with from the name, as follows:
  • Symmetric algorithms have cool-sounding names like Lucifer, Rijndael, Twofish, Serpent, MARS, etc.
  • Asymmetric (public-key) algorithms are named after their inventors, e.g., RSA (invented by Rivest, Shamir, and Adelman), ElGamal, Diffie-Hellman, etc.
  • US government algorithms have uninformative generic sounding names like Secure Hash Algorithm, Digital Signature Algorithm, Data Encryption Standard, etc.

Why? I'm not sure. Certainly part of the explanation is that symmetric algorithms are often designed by people in industry and so they naturally try to choose cool names. By contrast, asymmetric algorithms are often designed by people who are more or less mathematicians and so just end up publishing under their own names. And of course, government algorithms are standardized by civil servants, who naturally choose fairly generic names.

Another theory, conjectured by Hovav Shacham, is that a new symmetric algorithms actually has some chance of being use and designers think of adoption as their major goal. Accordingly, they choose names suitable for marketing. By contrast, asymmetric algorithms have a much lower chance of being used and so the designers major incentive is to have them discussed in the academic community. Having their own name be the name of the algorithm naturally increases their name recognition in the field. And, of course, government is concerned with creating the impression that their algorithms are the only choice, which a generic name does quite well.

Posted by ekr at 10:16 PM | Comments (2) | TrackBack

What do tax protesters want?

The other thing I don't get about tax protesters is what they think the endgame is. Great, so you don't think that you're required to pay taxes. But, as your mother would say, "What if everyone did that?" Now, I'm not saying it's not legitimate to want to have a world where there are no taxes. After all, that's the kind of world that David Friedman wants. And of course that means much less government, which David thinks is a good thing. But I don't get the sense that that's what the tax protesters want. They just seem to not like the income tax. But would they really be that much happier if they paid the same amount of money but it was through sales taxes not income tax? I'm skeptical.
Posted by ekr at 11:48 AM | Comments (10) | TrackBack

Hey, it's a bug in the IRS program

Brian Doherty's article in Reason nails the fundamental weirdness of the tax protester movement.
Not merely Protestant, the tax honesty people are strangely reminiscent of fandom -- of the comic book, fantasy, science fiction, role-playing-game variety. They have the same obsession with continuity and coherence within a created fantasy world of words. It's just that, in this case, that world of words isn't a multivolume fantasy epic or a long-running TV series -- it's U.S. law. When these people try to reconcile the definition of income in this subsection of Title 26 of the U.S. Code with the definition in a 1918 Supreme Court case, it's like hearing an argument over the inconsistencies between a supervillain's origin as first presented in a 1965 issue of The Amazing Spider-Man and the explanation given in a 1981 edition of Peter Parker, the Spectacular Spider-Man.

The tax honesty movement's vision of the world is fantastical in another way. It is not merely obsessed with continuity; it is magical in a traditional sense. It's devoted to the belief that the secret forces of the universe can be bound by verbal formulas if delivered with the proper ritual. There are numerous formulae in the tax honesty spellbook, with rival mages defending them. Which spell is best: The summoning of the Sovereign Citizen? The incantation of the Constitutional Definition of Income? The banishing spell of No Proper Delegation?metimes

The tax honesty folks similarly believe that their foe the IRS must also be bound by these grimoires of magic: that without the properly sanctified OMB number an IRS form holds no power, that without uttering the mystic word liable no authority to tax can truly exist.

And always, always, the ultimate incantation, The Question: Where does it say that I owe income taxes? Show me the law!

You often see this kind of attitude with computer programmers and science types, who are used to the notion that things are predictable. Which is actually kind of surprising because one of the first things that any programmer learns is that programs have bugs. If you think of the tax code as a program for the IRS to follow, why should you be surprised that that program has bugs? Just as it's basically impossible to write error free programs, you should expect that it's not possible to write non-self-contradictory laws. This isn't some Godel-type thing, just a statement of human failing.

The difference between software and laws is that laws are executed by people, not computers, and people don't respond to this kind of contradiction by crashing, but by trying to find some way to make the interpretation of the law consistent with it's overall intention. In this case, I think the intention is pretty clear:

Pay your taxes or go to jail.

I would think that by this point that would be pretty clear.

Posted by ekr at 11:32 AM | Comments (47) | TrackBack

April 13, 2004

Improving the arterials

Chief Transportation Official for Palo Alto Joe Kott was kind enough to respond in the comments section to my post on Palo Alto traffic calming (thanks Joe!):
Lots going on with regards to the "making arterials work better front" - alas, not very dramatic or, perhaps more aptly, not melodramatic enough for the local daily press.

The entire City of Palo of Palo Alto traffic signal system (100 plus signals) is bieng upgraded, including new electronics, new controllers, new signal cabinets (in many locations), and fiber optic interconnections (in many locations). A more than $1m project mostly funded through competitively won federal and State transportation grants. This project will be completed by fall of this year. Benefits will include much more reliable traffic signal system operation, creating capability for City of Palo Alto traffic engineers to re-time signals from their offices as needed based on real-time traffic condtions, and provision of the basis for future enhancements, including full traffic signal automation (signal timing changes are made automatically and electronically, next cycle, based on real time traffic conditions)for which grant funding is now being sought. In addition, the City of Palo Alto Transportation Division has just won a regional (Metropolitan Transportation Commission) grant to re-time (which requires extensive data collection as a precondtion and preparation of whole new signal timing plans)all traffic signals on Middlefield, Embarcadero, El Camino Real (in conjunction with Caltrans), Oregon Expressway (in conjunction with Santa Clara County) and Page Mill Road (also in conjunction with Santa Clara County). Again, none of this is perhaps melodramatic enough for a local daily press headline, but all is strategically important for safe and efficient mobility in Palo Alto.

I'm glad to hear that the PA transportation types are worried about this problem and it sure seems like there's a lot of stuff going on. That said, I can't help but notice that all the projects Joe is talking about seem to be centered around control systems as opposed to laying down new asphalt. I only have a passing familiarity with traffic engineering for roads (as opposed to for data networks) but I do know that control systems can sometimes dramatically improve throughput of roadways that you'd think were totally congested. However, I don't know if that's really the case in Palo Alto. Maybe one of my readers would care to comment.

Posted by ekr at 05:55 PM | Comments (15) | TrackBack

Bogofilter goes berserk!

I haven't gotten any mail for the past 12 hours or so. That never happens. My first thought was that my internet connection had crashed, but it's doing just fine. Next question: where did all that mail I must have received go?

It turns out that it was all classified as spam by bogofilter. Something very bad just happened to my installation of bogofilter. A little investigation shows that something has gone wrong with the database of spam/not-spam words. Here's what happens running an innocuous message through 4/9/2004's database:

X-Bogosity: No, tests=bogofilter, spamicity=0.499997, version=0.14.5.4
   int  cnt   prob  spamicity histogram
  0.00   12 0.012854 0.007492 ############
  0.10    2 0.194437 0.027089 ##
  0.20    1 0.253553 0.039617 #
  0.30    2 0.347771 0.073674 ##
  0.40    0 0.000000 0.073674 
  0.50    0 0.000000 0.073674 
  0.60    0 0.000000 0.073674 
  0.70    0 0.000000 0.073674 
  0.80    0 0.000000 0.073674 
  0.90    5 0.981870 0.482533 #####

And then through this morning's database:

X-Bogosity: Yes, tests=bogofilter, spamicity=1.000000, version=0.14.5.4
   int  cnt   prob  spamicity histogram
  0.00    0 0.000000 0.000000 
  0.10    0 0.000000 0.000000 
  0.20    0 0.000000 0.000000 
  0.30    0 0.000000 0.000000 
  0.40    0 0.000000 0.000000 
  0.50    0 0.000000 0.000000 
  0.60    0 0.000000 0.000000 
  0.70    0 0.000000 0.000000 
  0.80    0 0.000000 0.000000 
  0.90   34 0.999730 0.999729 ##################################

Don't ask me what happened.., I didn't do anything unusual. I just shoved all the spam that bogofilter misclassified as good through 'bogofilter -s' as usual. And now it thinks every word is spam. Outstanding!

Posted by ekr at 07:13 AM | Comments (45) | TrackBack

April 12, 2004

Why not expense stock options?

Caught Anna Eshoo on NPR today arguing against expensing stock options. Here's a link to her argument (from her web site) for why expensing options is a bad idea:
"This issue cuts to the heart of job creation, economic growth and competitiveness," Eshoo said. "Broad-based stock option plans for rank-and-file employees are a critical tool in helping small start-ups to mature into medium and large-size companies. Yet if FASB's proposal is put into effect, it will result in the elimination of most broad-based stock option plans, doing away with a powerful tool for attracting talented workers and promoting employee ownership."

"FASB's draft rule ignores the fact that no accurate model for valuing employee stock options has been identified - including the binomial and Black-Scholes models required by FASB's proposal," Eshoo continued. "Neither of these methods was designed for the purpose of valuing employee stock options and FASB has refused to road-test these formulas in a real-world business environment."

Uh... wait a second.

At the moment, investors need to work out for themselves what the impact of options on the company is. Either the market is accurately valuing options or it's not. If options are being valued accurately, then putting them on the balance sheet shouldn't affect a company's stock price at all, since investors will be able to work out what the correct treatment would be. So, in order to believe that expensing options is a bad idea, you must also believe that investors are currently misvaluing options--and you're in favor of keeping it that way. Now, one can argue, Eshoo does, that the tools for computing option values (Black-Scholes and binomial models) aren't accurate, but of course those are precisely the same tools that investors are using to compute option values now!

Eshoo's argument seems particularly disingenuous because she's actually in favor of expensing some options:

Requires companies to expense options granted to the CEO and the next four highest-paid officers. Small businesses are exempted from this obligation and cannot be required to expense options for the three years following an initial public offering.

Call me crazy, but I don't see why Black-Scholes would be lousy for valuing options except when those options are owned by the CEO and the next four highest paid officers.

Posted by ekr at 06:54 PM | Comments (48) | TrackBack

Aaah! More traffic calming!

Ever since Terence moved to San Mateo, I'd kind of forgotten about Palo Alto's incredibly annoying traffic calming program. Until last week, that is, when the City Transportation construction guys erected these enormous speed bumps on Channing. Worse yet, Channing is a road I take on a daily basis. As I pointed out last year the way to stop shortcutting is to make the main arterials faster. There still doesn't seem to be much action on that, though.
Posted by ekr at 07:56 AM | Comments (29) | TrackBack

The Passion Of the Christ--well sort of

I'm a compulsive Fark reader and the other day I ran across a reference to The Passion of The Christ--in Glorious Musashivision. Be warned: this is hilariously funny but also probably offensive to just about everyone.
Posted by ekr at 06:58 AM | Comments (38) | TrackBack

April 11, 2004

A censoring DVD player

Here's a lower-end variant of my content-merging player. ClearPlay makes a DVD player that automatically edits out the naughty bits of the videos it plays.

Unsurprisingly, a bunch of directors are unhappy about it:

Several leading Hollywood figures, however, including Steven Spielberg and Steven Soderbergh, are backing a lawsuit, arguing that the technology will violate the rights of directors who expect their works to be viewed in their entirety, without censorship.

"In the guise of making films 'family-friendly', ClearPlay seeks to make whatever 'edits' they see fit to any material they don't like," said the Directors Guild of America. "By not seeking the consent of the director, whose name on the movie reflects the fact that the film comprises his or her work, or of the studio as copyright holder, they can and do change the very meaning and intent of films."

Economically, the directors aren't really being damaged here. In fact, the market for their product is being increased. Now, I understand that in Europe artists have "moral rights" that give them some control of derived works, but as far as I know that's not the situation in the US. Seeing as the "derived work" here is made solely on the customer's video screen, I'm not real sympathetic to the claim that they're being damaged, even in some fuzzy non-economic fashion.

Posted by ekr at 09:25 AM | Comments (1) | TrackBack

April 10, 2004

What controls the number of terrorist attacks?

As September 11 made fairly clear, the pre-2001 American air travel security apparatus was incredibly lame. But if you do a little research on terrorist incidents in airplanes you discover something interesting [*] [*] In the period 1992-2000, the National Transportation Safety Board lists 0 "Air Carrier Occurrences Involvining Illegal Acts (Sabotage, Suicide or Terrorism)", despite that fact that our security was clearly terrible. And in fact, there have been no terrorism incidents from US airports since September 2001 either.

What conclusion ought we to draw from this? Think of airport security as a detection function that has a probability P of detecting any given terrorist entering a plane. If there are T attempted terrorist attacks in any given year, then we should expect A = (1-P)T actually successful attacks per year. The fact that most years saw zero attacks suggests either that security has been spectacularly good or that that the actual number of attempted terrorist attacks is very low.

So, what happened in 2001? One of two things must have been true: either security in 2001 was much worse than in any other year or that the number of attempted attacks was much higher in 2001. My intuition--and I think yours should be too--is that it's the latter. The fact that airport security is transparently awful--there was next to no bomb scanning in American airports prior to 9/11, for instance--is pretty good evidence that there can't have been a lot of terrorist attempts that we thwarted.

But by the same token, there's no reason to believe that post 9/11 security has gotten much better. Sure, there haven't been any terrorist attacks since then, but what makes you think anyone has tried?

Update: Slightly rescoped the above claim in response to a comment by Bob McGrew

Posted by ekr at 08:01 AM | Comments (40) | TrackBack

April 09, 2004

Seven nasty weapons you can bring on a plane

What's particularly humorous about the new TSA screening process is the obsession with pointy metallic stuff. The same guys who confiscate your nail clippers will cheerfully let you walk right by with a ball point pen, which is much more dangerous. Therefore, as a public service here's an incomplete list the nasty weapons that you can bring on a plane:
Weapon Ingredients Allowed How to use Notes
Ball point pen - Yes Stab Prefer metal bodied pens
Slock 1 tube sock, 6 AA batteries Yes Insert batteries in sock. Swing. Common prison weapon
Ceramic knives - Of course not. But can't be detected by metal detector. Do I really need to explain what to do with a knife? Keep in person, not in carryon, since may be detectable on X-ray
Molotov cocktail Bottle, alcohol or gasoline, rag Mostly. Alcohol and gas are both tacitly allowed Light, throw An old favorite
Concentrated acid - No, but hard to detect Pour, spray, throw, etc. Acid burns are particularly horrifying
Concentrated base Drano No, but hard to detect Pour, spray, etc Sodium hydroxide (the stuff in Drano) reacts with aluminum to emit hydrogen. Bring some aluminum foil. Did I mention that a lot of the metal in planes is aluminum?
Chlorine gas Bleach, Ammonia Probably not but hard to detect Mix. Breathe. Die Use concentrated ingredients if you can get them. May explode.

All of this stuff is easy for ordinary people to get their hands on. There's a whole other category of stuff (explosives, thermite, Sarin, etc.) that is also really hard to detect but a little harder to get your hands on...

Posted by ekr at 07:49 AM | Comments (42) | TrackBack

April 08, 2004

No-fly lists...

Just read Tim Noah's article on the Transportation Security Agency's No-Fly list. It's mostly factual reporting with some sniping at the ACLU's lawsuit opposing the list, but then he pulls out the old "If it saves just one..." card:
Let's suppose just supposethat the No-Fly List has caused only one terrorist not to board an airplane with a sharp tool or explosive shoes. Wouldn't that still be worth these mild inconveniences? Of course it would. I don't mind being the haystack, because Sept. 11 taught me that there are needles out there. By all means, let's find better ways to search for them. But let's not make the perfect be the enemy of the good.

Well, maybe. It's not clear that selective screening is actually that desirable. The problem is that terrorists are adaptive. So, if we always give extra attention to people on the No-Fly list but never or rarely to anyone else, then the terrorists only need to find someone who's not on the list and they've drastically reduced their risk of being caught. (A more thorough writeup of this problem can be found here.) Depending on the number of people being screened and how selective the list is, random screening may actually be better.

More importantly, the very fact that we use high-intensity screening for people we believe to be higher risks suggests (correctly) that the routine screening procedures are pretty useless--at least if your goal is to stop terrorists as opposed to people carrying nail clippers. I'd rather see us focus on fixing that (or, more likely, admitting that the whole job is hopeless) than worrying about who should get the super-duper-extra screening.

Posted by ekr at 08:09 AM | Comments (48) | TrackBack

April 07, 2004

I love these guys

Ed Felten over at Freedom To Tinker points out Ernest Miller's commentary on the appalling mess that is the proposed WIPO treaty on broadcasting:
Here's the most amazing part, from Article 16, Alternative V:
2. In particular, effective legal remedies shall be provided against those who:
...
(iii) participate in the manufacture, importation, sale, or any other act that makes available a device or system capable of decrypting or helping to decrypt an encrypted program-carrying signal.
Every computer is "capable of decrypting or helping to decrypt" such a signal, so this provision, if adopted, would apparently require signatories to the treaty to ban the importation, sale, or distribution of computers.

This sort of thing seems to be a pretty common feature of attempted government regulation of technology. The people writing this stuff seem to think they can draw a bright line between good and bad uses, but that's generally not possible. What typically ends up happening (think DMCA) is that they make just about everything illegal and when challenged claim that of course they would never think of prosecuting good people... until it becomes convenient to do so, that is.

Posted by ekr at 09:07 PM | Comments (3) | TrackBack

Home medical care

You know, for this I think I would go to a doctor.

Link via Kevin Dick and Jennifer Gates

Posted by ekr at 11:14 AM | Comments (5) | TrackBack

April 06, 2004

An additively separable, exogenous, non-pecuniary endowment good

Continuing my browsing through the latest issue of the JPE I ran across this Marriage and Consumption Insurance: What's Love Got to Do with It? by Gregory D. Hess.

You can always tell you're in for some hard-core econ theory when the introduction to the paper contains something like this:

While the desire to offset idiosyncratic labor risk could be a powerful inducement to marry, it is also the case that other issues also matter when it comes to marrying and staying married. I simply term this factor "love." In the model, love is an additively separable, exogenous non-pecuniary endowment good, which two individuals mutually share. It is, for better or for worse, subject to shared fluctuations.

Skipping past all the heavy-duty math, Hess's argument is that there are two reasons to get married:

  1. To hedge economic risk. If you're married to someone with a different job from yours, then it's like having insurance. If your career works out badly, maybe theirs won't and so you won't be so bad off. Of course, maybe your career will go well and theirs won't, but that's the price you pay for insurance.
  2. For some unspecified and economically unmeasurable reason, you actually desire to be with this particular person.

One's total desire to get and stay married is some function of these two factors. Moreover, there's a tradeoff. If a potential marriage is a very good risk hedge, then a couple doesn't need to love each other that much in order to put them over the threshold where it's worth marrying. Conversely, if a couple loves each other a lot, they might get married even if they have almost exactly the same job and thus the marriage is a lousy risk hedge. Given this model, Hess is concerned with which factor is more important. After a lot of modelling, number crunching, etc., He concludes that:

This paper present strong evidence that joint economic characteristics from the beginning of a marriage are significant explanatory factors in a marriage's probability of survival. The evidence uncovered is that more positively correlated incomes between partners and a bigger gap in their income volatiles are associated with marriages of decreased duration, though bigger mean income gaps do not affect a marriage's rudation. This pattern of results is consistent with the view that spouses who are good economic matches for one another are associated with longer-lasting marriages and that initial love is not a reliable substitute for this essential ingredient.

That's good information to have. Potential Mrs. EKRs should can submit their resumes and evidence that our incomes are uncorrelated to ekr@rtfm.com or in the comments section.

Posted by ekr at 06:18 PM | Comments (41) | TrackBack

Tired of labor disputes?

Alan B. Krueger and Alexandre Mas have an interesting interesting paper in the April 2004 Journal of Political Economy. They argue that the high rate of defective Bridgestone/Firestone tires (which you'll recall were blamed for causing Ford Explorer rollovers) was the result of a labor dispute between Bridgestone/Firestone and the United Rubber Workers (one, which it should be noted, Bridgestone ultimately won and the URW went bankrupt).

It turns out that making tires is actually a substantially manual and skilled process:

Franklin (2001, p. 117) describes the plant as "dark, brooding, and smoky in places, a noisy monument to the almost unchanging way most tires are still made: with much reaching and lifting and cutting, a lot of human muscle power, and an endless demand for unflinching robot-like concentration." Particularly in the Decatur plant, tire production was not automated. For example, workers had much discretion over the amount of effort exerted to wrap the steel belts, the condition of the rubber used to produce the tread and sidewall, and the implementation of the vulcanization process.

And of course, in situations where workers have an enormous amount of control over production, they have an enormous ability to impact quality, which is what Krueger and Mas argue happened. The following figure makes their case fairly clearly.

Note that the times of lowest quality coincide with the times of highest labor strife. In particular, the worst period occurs after the strike when the replacement workers and original Union workers were working side-by-side.

Posted by ekr at 08:18 AM | Comments (4) | TrackBack

the Linux iPod, .ogg, etc.

Joel Johnson over at Gizmodo pretty effectively skewers the Linux iPod project:
Great Stallman's ghost! Linux on the iPod, so I can finally...well, play MP3s. Sadly, one of the goals of the project, the ability to use the iPod to playback OGG Vorbis files, has still not been realized. So without being too cruel, this project is essentially useless at the moment, but has the potential to do some interesting things with the firmware in the future (like create a highly-useable interface and a fast song database for quick performance, sort of like, you know, Apple).

I've never really been sympatico with the Open Source (sorry, Free Software) obsession with replacing perfectly good stuff that didn't happen to be Open Source/Free Software with inferior stuff that was Open Source/Free Software. Let me know when the Linux iPod actually does something more interesting than displays it's boot prompt.

Posted by ekr at 07:19 AM | Comments (6) | TrackBack

April 05, 2004

More good news from the forensic scientists...

This article in the New Scientist. This time it's date of death estimation using maggots that's being tested and found wanting:
When Archer put this technique to the test at sites of hypothetical murders, the estimates were usually within 1 degree Celsius of actual temperatures.

However, if the weather had changed markedly between the monitoring period and the period the imaginary body was decomposing, the estimates were out by as much as 8 degrees Celsius. That could lead to errors of several days in the estimated time of death (Journal of Forensic Science, DOI: 10.1520/JFS2003258).

Sheesh. Is DNA the only forensic technique that does have a good scientific basis?

Posted by ekr at 04:40 PM | Comments (1) | TrackBack

Scooby Doo

Just watched Scooby Doo last night. I didn't have really high hopes (and the reviews weren't good) but it turned out to be very funny, at least if you'd watched the TV show as a kid. The tricky part about doing a live action version of a book/cartoon/comic book--especially one that kids grew up on--is producing something that's somehow "the same" as the original while still working in the bigger format.

One way of doing this is try to slavishly follow the original as much as possible. That's pretty clearly what happened with Lord of the Rings and I suspect with X-Men. Another option is to totally reconceptualize it (how many teen high school remakes of Shakespeare have you seen?) The makers of Scooby Doo have clearly watched the original a zillion times and instead of doing a straight remake or a reconceptualization have decided to make a mostly straight remake but bring to the fore all sorts of stuff that was implicit in the original but I at least wasn't sophisticated enough to pick up at the time:

  • Fred is an egomaniacal credit grabber.
  • Daphne is tired of always having to be the damsel in distress.
  • Velma secretly pines for Fred and is annoyed that he "only wants to date swimsuit models" (Fred responds that he thinks "dorky chicks like you are hot").
  • Although we never see Shaggy take a toke, it's time to face the fact that he's a stoner...
  • And then there's what they do to Scrappy Doo...

Now, maybe that stuff was all there in the original (I know that the Shaggy/stoner bit was, but then I never saw it when I was eight). So, watching it on film was a series of "ah ha!" moments as I realized that of course this is what was really going on in the original series.

Definitely a specialized taste, but if that taste is yours...

Posted by ekr at 08:19 AM | Comments (2) | TrackBack

April 04, 2004

Project Pluto

Pretty much all space buffs have heard of the legendary nuclear bomb powered spacecraft Project Orion. The appeal of Orion is obvious: it was simple, clever, and attainable even with 1960s technology. Of course, the fact that involved setting off nuclear explosions behind your spacecraft (and presumably in the atmosphere) tended to give some people pause, but what's a little radiation between friends?

Until last night, however, I hadn't ever heard of Project Pluto. During the late 1950s, the Pentagon became concerned that the Soviets would develop an effective anti-missile defense system and wanted to have a backup plan in case our ICBMs became ineffective. The solution? A nuclear powered hypersonic cruise missile!

Pluto's namesake was Roman mythology's ruler of the underworld -- seemingly an apt inspiration for a locomotive-size missile that would travel at near-treetop level at three times the speed of sound, tossing out hydrogen bombs as it roared overhead. Pluto's designers calculated that its shock wave alone might kill people on the ground. Then there was the problem of fallout. In addition to gamma and neutron radiation from the unshielded reactor, Pluto's nuclear ramjet would spew fission fragments out in its exhaust as it flew by. (One enterprising weaponeer had a plan to turn an obvious peace-time liability into a wartime asset: he suggested flying the radioactive rocket back and forth over the Soviet Union after it had dropped its bombs.)

The project was headed by Ted Merkle (the father of Ralph Merkle of public key cryptography fame). Like Orion, much of the appeal of Project Pluto is that it was technically sweet, high-tech (just designing the reactor was apparently a major challenge) outside the box thinking and inherently insanely destructive. Like Orion, the designers got to the point where it was pretty clear it would work--they had a successful static test of the reactor--but then was inevitably cancelled for reasons that should be obvious. Still a pretty cool idea, though.

Posted by ekr at 09:55 AM | Comments (2) | TrackBack

April 03, 2004

Life before decent medical care

In case you need to be reminded, here's David Landes from The Wealth and Poverty of Nations:
In June of 1836, Nathan Rothschild left London for Frankfurt to attend the wedding of his son Lionel to his niece (Lionel's cousin Charlotte), and to discuss with his brothers the entry of Nathan's children into the family business. Nathan was probably the richest man in the world, at least in liquid assets. He could, needless to say, afford whatever he pleased.

Then fifty-nine years old, Nathan was in good health if somewhat portly, a bundle of energy, untiring in his devotion to work and indomitable in temperament. When he left London, however, he was suffering from an inflammation on his lower back, toward the base of his spine. (A German physician diagnosed it as a boil, but it may have been an abscess.) In spite of medical treatment, this festered and grew painful. No matter: Nathan got up from his sickbed and attended the wedding. Had he been bedridden, the wedding would have been celebrated in the hotel. For all his suffering, Nathan continued to deal with business matters, with his wife taking dictation. Meanwhile, the great Dr. Travers was summoned from London, and when he could not cure the problem, a leading German surgeon was called in, presumably to open and clean the wound. Nothing availed; the poison spread; and on 28 July 1836, Nathan died. We are told that the Rotschild pigeon post took the message back to London: Il est mort.

Nathan Rotschild died probably of staphlococcus or streptococcus septicemia--what used to be called blood poisoning. In the absence of more detailed information, it is hard to say whether the boil (abscess) killed him or secondary contamination from the surgeons' knives. This was before the germ theory existed, hence before any notion of the importance of cleanliness. No bactericides then, much less antibiotics. And so the man who could buy anything died, of a routine infection easily cured today for anyone who could find his way to a doctor or a hospital, even a pharmacy.

Of course, in 100 years we may be reading about how people in the 20th century died of a routine tumor that could be easily cured by anyone with access to a home nanopharmacy.

Posted by ekr at 07:18 PM | Comments (54) | TrackBack

April 02, 2004

Blue Shield thinks that you only need to get laid 6 times a month.

I happened to be browsing Blue Shield's formulary when I noticed this little gem:
Levitra: For the treatment of erectile dysfunction, requires prior authorization. If approved, quantity limit of 6 tablets per 30 days. Not covered for all plans.

I can just picture the Blue Cross meeting where they decided that one...

Posted by ekr at 03:25 PM | Comments (75) | TrackBack

So, what if filesharing doesn't depress sales?

If we buy Oberholzer and Strumpf's argument that filesharing doesn't depress CD sales, then the economic arguments for allowing it start to look overwhelming:
If we are correct in arguing that downloading has little effect on the production of music, then file sharing probably increases aggregate welfare. Shifts from sales to downloads are simply transfers between firms and consumers. And while we have argued that file sharing imposes little dynamic cost in terms of future production, it has considerably increased the consumption of recorded music. File sharing lowers the price and allows an apparently large pool of individuals to enjoy music. The sheer magnitude of this activity, the billions of tracks which are downloaded each year, suggests the added social welfare from file sharing is likely to be quite high.

However, it doesn't appear to change things from a legal perspective at all. Indeed Patel's opinion in the Napster case specifically contemplates this argument and rejects it:

Any potential enhancement of plaintiff's sales due to sampling would not tip the fair use analysis conclusively in favor of defendant. Indeed, courts have rejected the suggestion that a positive impact on sales negates the copyright holder's entitlement to licensing fees or access to derivative Markets. See Ringgold v. Black Entertainment Television, 126 F.3d 70, 81 n.16 (2d Cir. 1997) (nothing that, even if allegedly infringing use of poster in television program increased poster sales, plaintiff retained right to licensing fee); DC Comincs Inc. v. Reel Fantasy, Inc. 696 F.2d 24, 28 (2d Cir. 1982) (stating that speculated increase in plaintiff's comic book sales due to unauthorized use of Batman and Green Arrow figures on advertising flyers did not establish fair use defense as matter of law); MP3.com, 92 F. Supp 2d at 352 (holding that allegedly positive impace on plaintiffs' prior market "in no way frees defendant to usurp a further market that directly derives from reproduction of [the] copyrighted works.")

That's not to say that O&S's result isn't important. While it may not change the legal sitation, it does change the political situation. Much of the moral force for cracking down on P2P comes from the claim that downloading is taking food out of artist's mouths (or at least bling bling from around their necks). If that's not true, then it's a lot harder to argue that filesharing is a big societal problem warranting a harsh response.

Thanks to Hovav Shacham for pointing me to this section of the Napster decision.

Posted by ekr at 06:42 AM | Comments (54) | TrackBack

Does P2P filesharing reduce music sales?

The primary argument against P2P filesharing is that it deprives the copyright owners of revenue in the form of foregone sales. I.e. people download their Britney from Kazaa rather than buying it on CD or from the iTunes music store. Although this argument is intuitively compelling, and it's certainly true that music sales seem to be declining it's a hard problem to study and attempts to demonstrate that that is connected to P2P haven't been particularly convincing.

On Tuesday Oberholzer and Strumpf posted the results of their econometric analysis of the problem and come to a different conclusion. They write:

Downloads have an effect on sales which is statistically indistinguishable from zero, despite rather precise estimates. Moreover, these estimates are of moderate economic significance and are inconsistent with claims that file sharing is the primary reason for the recent decline in music sales.

To understand what Obherholzer and Strumpf did we have to first understand how to think about the problem. If we simplify the problem down to it's most trivial form, we can think of sales of an album as being proportional to some album specific "desirability". So, we might write:
S_i = A*d_i
Where S_i is the sales of album i, A is some proportionality constant, and d_i is the desirability of album I. So, more desirable albums sell more. It's true that we don't know the relationship is linear, but it's good to start with linear models.

Now, it should be clear that the decision of whether to "purchase" a download looks rather similar, just with a different proportionality constant, B.
D_i = B*d_i
This is the treatment assuming that downloading and sales don't interact, i.e. that downloads are not substitutable for sales. However, of course they are. So, let's try writing a combined equation.
S_i = A*d_i + C*D_i = A*d_i + C * B * d_i
Where C is the impact that downloading has on sales (assume that sales has no impact on downloading for the moment). C is the variable we're concerned to estimate. So, imagine that we do the obvious thing and fit a line to (S_i,D_i) over time. What are we going to see? Because A,B and C are assumed constant, what we're basically measuring is the change in desirability of the song. But since both sales and downloading vary with the desirability of the song we can't draw any conclusions about the effect of downloading on sales.

The standard econometric technique at this point is to look for variables (called instruments) that make downloading more or less attractive but don't impact sales otherwise. So, for instance, if one of the major P2P services went down for a month or so, downloading would--no matter how desirable a song is--and we could see if sales go up. If they do, that suggests that downloading is depressing sales. Unfortunately, there's no such natural experiment sitting around for us to analyze, so we have to be a little trickier about it.

O&S use four major instrumental variables:

  • File size--Large files are more difficult to download, but are equally easy to buy on CDs. Thus, we would expect long songs to have relatively fewer downloads by comparison to their sales.
  • Network congestion--Even if no P2P service has gone out completely, large amounts of network congestion would make downloading less convenient and therefore ought to reduce downloading without affecting sales.
  • German vacations--Apparently a lot of American downloading happens from Europe, in particular from Germany and the computers of German teenagers. When German kids are on vacation, their computers are likely to be on and available for filesharing. This increases the supply of files and thus makes downloading more attractive while having no direct effect on CD sales.
  • Aging--Because people's hard drive space is limited, the authors theorize that older tracks get crowded out by newer tracks. This ought to depress downloading of older tracks. So, you can look at the total volume in minutes of newer tracks in a category.

Now, I know that this is unusual looking stuff to CS types. We're used to thinking of variables which are really tightly tied to the effects we think we're measuring. The influence of German teenagers on US file downloading is not the kind of obvious connection that people in CS are comfortable with. However, this is absolutely standard stuff in econometrics--the data you have just doesnt allow you to measure things directly a lot of the time. In fact, a lot of the skill of econometrics is finding clever instruments that allow you to isolate what you're interested in.

I won't describe O&S's analysis, which is quite complicated and requires a pretty good knowledge of econometric statistics to understand. I'm only an amateur econometrician and I've only spent an hour or two looking at the paper, but the results don't look crazy to me. Basically, when the dust settles, O&S weren't able to find any statistically evidence of sales depression due to downloading. Even if we ignore the fact that the estimates aren't statistically significant, they're not really clinically significant either: the estimated shifts are small compared to the size of the recent drop in music sales.

It's important to realize that this study doesn't prove that filesharing doesn't affect downloads. It's very hard to prove a negative. However, given that it's the best data we have, the appropriate stance to take is to say that despite some intuition to the contrary, we don't have any evidence that P2P sharing decreases album sales and that we shouldn't take any actions based on the theory that it does. r

Posted by ekr at 06:11 AM | Comments (39) | TrackBack

April 01, 2004

Free Zyrtec!

Ok, so you're not in favor of making all prescription drugs OTC. But there are certain restricted categories that should be easier to come to consensus on.
  • Non-drowsy antihistamines: Claritin is already OTC. Zyrtec and Allegra are quite old now and everyone treats them more or less interchangeably.
  • Proton-Pump Inhibitors: Again, Prilosec is OTC and it's used almost interchangeably with Prevacid, Protonix, and Nexium. We should be able to OTC all of these.
  • COX-2 Inhibitors: This is a harder call because these are all fairly new. However, the current NSAIDs have enough known problems (mainly GI bleeding) that it may be worth transitioning them into the OTC market. Since they're very widely used we already have a fairly good baseline. Celebrex would probably be the best candidate to start with since both Bextra and Vioxx have had some hazardous side effects noted.
  • Oral Contraceptives: We've got decades of experience with these. The pelvic exam doesn't add much. The should have been OTC years ago.

This would be a good start and could make a real dent in the number of times people had to go to doctors for conditions with basically obvious, routinized treatments.

Posted by ekr at 08:11 AM | Comments (44) | TrackBack

Would liberalizing drug access actually harm people? Let's find out.

The central argument against liberalizing drug access (both in the comments threads below and over at Medrants is that doing so would cause patient harm. This harm ostensibly comes in two forms:
  1. Patients will unwisely take drugs that they shouldn't and harm themselves.
  2. If patients are not required to see doctors to get drugs they will not get proper monitoring and therefore suffer health effects.

Now, at the moment a lot of the argument is theoretical, but then so is the argument for liberalization. But it doesn't have to be, because we have a mechanism for finding out.

What we need is a controlled trial. Let's take a reasonable-sized population (thousands, probably) and offer half of them unrestricted personal-use access to previously prescription drugs (excepting scheduled drugs, antibiotics, and antivirals). We monitor them for a year or two and measure morbidity and mortality.

If such a trial were conducted and it showed no significant difference would the people who are currently opposed to removing the prescription requirement now favor it? If not, why not?

Now, I don't expect that this experiment will actually be conducted--though I think it would be a good one--but that doesn't mean that we can't try to answer this question without it. Are there natural constituencies who have easier access to prescription drugs? Maybe people who live close to Mexico and can drive over the border? Has the existence of Internet pharmacies changed the situation at all? Obviously, an epidemiological study of this kind would be less convincing than a controlled trial, but it would be a good start.

Posted by ekr at 07:56 AM | Comments (56) | TrackBack