inamerrata

Media Watch on Mike Carlton:

We’re pleased to report Sydney radio, 2UE host and Sydney Morning Herald columnist Mike Carlton has also corrected the record after lately being full of praise for the ancient and eloquent Democrat Senator – of course were talking America here – Robert C. Byrd.

In spite of Robert Byrd’s background in the KKK having been well known prior to Mike Carlton’s 24th May article; even including a minor controversy springing from his use of the phrase “white nigger” back in March 2001. Michelle Malkin at the time wrote, among other things:

The ex-Klansman later filibustered the landmark 1964 Civil Rights Act — supported by a majority of those “mean-spirited” Republicans — for more than 14 hours. He also opposed the nominations of the Supreme Court’s two black justices, liberal Thurgood Marshall and conservative Clarence Thomas. In fact, the ex-Klansman had the gall to accuse Justice Thomas of “injecting racism” into the Senate hearings. Meanwhile, author Graham Smith recently discovered another letter Sen. Byrd wrote after he quit the KKK, this time attacking desegregation of the armed forces.

But fair enough — it’s not possible to know everything you’re paid to have an opinion on, and maybe you might miss out on something like this if you’re just going by Senator Byrd’s official site at the Senate, whose “About Me” page is curiously absent any comment on his formative ties to the Klan. So fair enough that before laying into him Media Watch writes to inform him of his mistake, before laying into him too heavily on national television. Mike’s recanting looks something like this:

Plainly, that renders his background considerably less than admirable.

…

But Byrd’s racism, past or present, does not, of itself, invalidate his criticism of the war on Iraq and the neo-conservative Bush Administration.

It just makes it much harder to support, that’s all.

(He’s wrong, actually. A person’s background doesn’t make their arguments any more or less valid. The only question it affects is how safe you are to trust what they say without carefully checking their claims of fact and their conclusions yourself. One would hope that, as a writer on the opinion pages, you’d be doing this sort of groundwork yourself, rather than uncritically repeating the claims of political heavyweights; but surely that would imply you wouldn’t be particularly concerned about their background, admirable or otherwise.)

So, fair enough, right? Find a misstatement, tell them about it, let them correct it, report on what happened. Everything’s great!

Let’s compare this to Media Watch’s treatment of another columnist who’d come under a misapprehension. Tim Blair had the happy fortune of being the subject of Media Watch scrutiny over the origin of the American flag that briefly covered a statue of Saddam in Baghdad, and made international news.

Do we see Media Watch taking the time to inform Tim privately and seeking a response, not merely to information widely known that he may simply have not noticed as above, but to first hand information Media Watch had obtained from people directly involved? Apparently not — instead it’s straight to television with it. In Tim’s blog, he makes the following response:

I WAS WRONG. The flag that was draped over the statue of Saddam did not come from the Pentagon. Media Watch last night presented two impressive sources who contradict widespread international reports – and my own repeated and, as it turns out, incorrect claims – that the flag was sourced from the Pentagon on September 11.

Media Watch’s pleasure at being able to report this correction of the record also seems to have been somewhat more muted.

Hrm, blosxom is affected by timezones — dates are whatever day it would’ve been in the timezone specified by $TZ or /etc/timezone, not necessarily when you wrote it, or where you are. That screws up permalinks, too if you ever change timezones. I wonder if there’s anything sensible you could do about it.

It wouldn’t make sense anywhere but slashdot:

“As reported by NTK, …”

NTK: sarcasm and investigative journalism.

So, this site is now running on a new host. How exciting.

In other news, Apache’s SUexec sucks.

One of the fashionable things about ecash is (or at least was back in the mid 90’s when ecash was fashionable, and pre-September 11) that it can provide anonymity: neither the bank, or the merchant, or the government can track how you spend your money. Lucre provides anonymity to the degree that you can’t do any correlations at all except by traffic analysis. This potentially allows vast amounts of money laundering, and the benefit is somewhat questionable.

It should be trivial to do away the anonymity in lucre by requiring the blinding factor to be 1. Other possibilities are only issuing particular variants of the currency to suspected launderers: so any currency marked with a big “D” that’s spent anywhere is likely to be drug money, and you can see where the money goes, but not where it came from. You can’t do this with a great deal of secrecy from the person you’re trying to spy on, which makes their choice “accept that the bank can see what I’m doing, or don’t use ecash”, but that’s going to happen anyway. To a point, it trades off law enforcements ability to easily monitor criminal activity and trace it, to the ability to stop criminals from being able to accept money. Except that it doesn’t really work that well.

What benefits does anonymity provide? It means you don’t have to keep track of accounts, but I think you have to do that to keep your bank in the black anyway. Since you have to track transactions anyway, it doesn’t give you an excuse to use significantly less storage, either. On the upside, it avoids embarrassment, if you know you’re never going to get a statement, but that can be arranged by never giving out statements. Non-anonymity might be a lie, in that you can do money laundering in someone else’s name anyway, and get the cash yourself with good assurance, without having any of your accounts ever used — but knowing who your launderer is is probably more important anyway (Drew buys a car from Joe, using Laura’s cash – police track down Joe, ask for his receipts, start wondering who this Laura is…)

Probably better to provide something that’s anonymous, then remove the anonymity if it’s not valuable, though — doing it the other way around would probably be hard.

Raph Levien points to a comment by Tim Bray that says, among other things:

Those fables were taken to the UN Security Council and ended doing severe damage to America relationships with pretty well everybody, because pretty well everybody refused to sign up for war on the basis that they were scared of Saddams WMDs, because they weren’t.

Except, they were. No one was arguing that Saddam didn’t have weapons of mass destruction, merely that continuing sanctions and inspections would produce an acceptable result at less expense (in terms of ammunition, or human lives).

But no one with any sense doubted that Iraq had weapons of mass destruction, the question was always how to go about disarmament. If there had been a question, the issue would not have been war or sanctions and inspections, it would have been whether any action was necessary at all. It’s easy to lose sight of the distinction in the emotional response to being lied to and led to a war not of your choosing, but self-deception isn’t healthy.

In a prior entry Tim Bray said:

No, the peace movement’s worst nightmare is that the United States extrapolates from Iraq and decides that unilateral aggression is an easy, rewarding and fun way to solve the world’s problems.

And that was everyone’s worst nightmare: that without a reasonable burden of proof, we’d start getting involved in many wars, and plunge into some sort of a cross between Vietnam and World War II, with the added fun of the other guys having nukes. A quagmire, you might say. As it turns out, we don’t look like risking that at all: we’ve moved beyond Vietnam tactically and strategically, and the Iraq war has had a salutary effect on a number of other potential targets and had the mildly perverse effect of making it less likely for there to be more confrontations.

The administration tried meeting the higher stand of proof (that is, a clear and immediate threat to the US itself, by way of weapons of mass destruction falling into the hands of hypothetical terrorists), and failed. The real question, and the real cause for introspection, is whether we were expecting the right standard of proof in the first place.

There’s an interesting paper on millicents (digital cash worth about a thousandth of a cent) from a guy who seems to be involved in Microsoft’s Penny Black project.

A decade hence, assuming that computers (and their components) continue on the price and performance curves of the last two decades, the minimum transaction grain will be an order of magnitude smaller than it is today. The smallest transactions will be in the sub-penny range. In a decade, we will see pricing by the web page. We can unbundle the newspaper and the magazine, derive revenue from the newspaper morgue, and sell information without restricting consumers to a subscription to a small number of sources.

Interesting thought experiment: imagine an average fee of about .1c/MB ($1/GB) integrated into http. You can probably setup a browser to obey this, so that it just displays “this image too expensive, right click to download anyway” when appropiate. It’s no more expensive than traditional bandwidth charges (10c/MB in Australia), and it sets up an economy so that when you’re slashdotted for writing something way cool you have a chance at making a profit instead of a loss. It potentially gives you a way of rate-limiting users too if you’re overloaded — just raise the prices until you’re not. The numbers might not work out, but in theory should do. Search engines might have problems — they need to scour the web, which would potentially cost a lot of money; but either it already does that, or it’s already subsidised elsewhere, and they already get a bunch of hits. Still, a market should be able to cope with this, possibly by making google.com marginally more expensive to view than other sites.

Note that this doesn’t quite match the email scheme — it presumes downloaders stop paying their ISP per MB, and pay the websites they view instead. Which means the websites pay their ISPs instead, which means fees can be based on outgoing traffic instead of incoming traffic, which in turn puts the burden of paying for the traffic of slammer worms and the like where it belongs. I may be drawing to long a bow at this point.

I’m obsessed with market theory :(

If this theory pans out, then you could make a good argument for allocating the entire social cost of spam to David Chaum’s patent on ecash.

Oh, the double entendres. Anyway, this Infoworld article quotes Ryan Hamlin, the general manager of Microsoft’s antispam technology and strategy:

“It won’t surprise me at all if we spend close to $18 billion a year next year to deal with spam,” he said. This cost includes the price of filtering software and storage hardware and other costs. Loss of productivity is not factored in, Hamlin said.

…

“An occasional spam might show up, but it is kind of noise and you will just delete it. Spam [fighting] will evolve into a measure-countermeasure cycle similar to the antivirus landscape,” he said.

No offense, but the antivirus landscape is abhorrent and a symptom of shoddy security measures in common systems, and $18 billion dollars of market friction isn’t something that should be tolerated. For reference, 35c per year, by the population of America is around $.07 billion USD, or per person, $60 USD versus 22 US cents wasted per year.

Each currency has to have a single dedicated bank to handle transactions. It requires a database containing an entry for every transaction that’s taken place. Each transaction needs to do a single element lookup from that database, and there shouldn’t be any locality of reference. Your storage requirements are O(T), and your transaction overhead is O(lg(T)) secondary storage operations. If each transaction takes 512 bytes of storage, then you can store about two million transactions per gigabyte.

As such, you have to retire a currency on a regular basis — otherwise you need arbitrarily large resources. How regular? If you have 1000 people, getting 1000 emails per day, that probably fills your two million transactions. A month’s email traffic for a small town would thus take up around 30GB. Covering a million people, say Brisbane, for a day hits a terabyte, or 10 100GB hard disks, ignoring redundancy. Coping with everyone in America, say 200 million people, would require 2000 100GB hard disks, per day. Doing that for a month, would take you up to six petabytes. Ouch. At $1/GB, each user costs 0.1c per 1000 transactions. That’s 35c/year per user for storage, which isn’t exactly going to break the bank. It does indicate costs would have to be very carefully managed, and that switching to actual charging (albeit small amounts) would be necessary fairly quickly.

It’s probably necessary to have the storage decentralised, rather than having to query a single, multi-petabyte datastore for every email anyone sends ever. Well, more certainly than probably. That necessitates multiple currencies, and an easy way to convert amongst them. Is it better to just have many floating currencies (which will happen due to competition anyway), or should we have multiple concurrent versions of a currency? ie, AJD.1, AJD.2, AJD.3, which a fixed 1:1 exchange rate? You only need to consult the transaction database to validate a coin, not to generate a new one, so that would make it easy for you to contact s1.ajcurrencyserver to exchange a coin denominated it AJD.1 and to get back a coin denominated in AJD.3. That allows you to do a reasonable degree of load balancing, doesn’t it? As long as the values are very small, it doesn’t let the bank mark the notes in a meaningful way (well, strictly it does, but you should be able to programmatically avoid any problems due to it easily enough). It does mean you have to be willing to accept AJD.* rather than a single currency, which could be awkward. I’m not sure if there are any cases where you care about the race condition where you can check one coin, have it pass, then check the second and have it fail. For email, you just take the coins and drop the mail. For markets, you refund the valid coins and start over.

Conclusions? Per transaction fees are required to pay for storage, as well as other running costs. Multiple currencies are necessary, probably even to the point of splitting a single notional currency at the implementation level, in spite of the complexities that introduces.

Movable bioweapons labs in Iraq have been tested and confirmed.

An obvious problem is how to transition from charging nothing for all your mail to charging something, without having everyone else do the same thing at the same time, or not getting any email ever again. There are a few ways we can ease this in.

The first and biggest problem is mailing lists. Coping with them at all is difficult: if you have a 5c mail going into the list, you somehow have to produce 5000 5c (or 30c) mails going out. Where’s the money come from? Obviously posters can’t afford it. An easy answer is the subscribers — if you subscribe for 500c, you can get 100 5c mails from the mailing list (and thus a full 500c refund!) before your subscription runs out. Coping with the initial subscription is easy too: you send a 5c subscription request, you get a 5c confirmation, and you reply to it.

All that’s well and good, but it doesn’t work unless the list manager is in on our scheme, which is the transition problem. If we’re going to solve it, we have to have an address in “our” control that accepts mails from the mailing list, without coins. It can either then verify them somehow (X-Mailing-List headers, eg) and add them to our inbox, or it can add a coin to them before forwarding them onto our real address. The latter lets you setup a subscription server during the transition that gateways old mailing lists into cashed up mailing lists for individual users. If the mailing list software doesn’t give out lists of subscribers, you can use an address like aj-123456789abcdef@example.com that’s unpredictable and different for each list to avoid spammers and strangers getting you directly. If it does give out lists of subscribers, you need to manage to drop non-list posts to that address somehow, which could get a bit flakey, but should be reliable enough. The other problem is posting to the list — if the list accepts posts from anyone (and is thus a spam trap itself), then you don’t have a problem; but if it’s for subscribers only, you do. You probably have to subscribe twice, and either elect for your real address not to receive mail, or just drop the mails that go there. All of that should be doable.

The next problem is that of strangers emailing you. This is the set of people to whom spammers belong, but it also contains “friends you haven’t met yet”. Obviously the most convenient thing for them to do is to setup ecash themselves, so that needs to be easy. Basically, you pretty much have to bounce them a reply saying “You haven’t paid enough money” and get them to either resend it with a coin, or to go to some website and confirm it. Ideally you want going to the website to be about as easy as it would be if they installed all the software locally. Basically, that website has to store the rejected mails, and allow you to send them on. The URL can contain the Message-ID easily enough, and maybe a nonce. Ways of approving mails could include presenting a coin (if your email software can’t cope), or doing one of the cute problems that aren’t difficult, but require human interaction. You could do this in the same way as subscriptions (ie, put up 500c, for every 100 emails from strangers, and recycle the funds), or you could let people get new coins from the mint when they solve such problems.

Those are really the easy problems, though. The hard one is dealing with friends emailing you. You can’t just tell them to get a new email program, and you can’t expect them to bother solving irritating problems whenever they want to mail you. You also can’t just drop their email. The only way to solve this problem is to hope they change their minds eventually. Work arounds have to involve identifying mails from friends and restricting them less. Possible ways to do so are by special addresses (aj-foo-123456) which suck, matching on From: lines, which spammers can and do fake, or cryptographic signatures, which no one does. Some combination of those would probably work; which means you definitely want your filter to cope well with all of them.

The final issue here is getting coins into the system. One way is letting people buy coins for real money. That seems unlikely to happen, at least early on. Another is for them to do computationally intensive stuff like run a distributed.net client, which seems like a good idea, although working out how many coins for how many blocks might be difficlt. Proving you’re a human is also probably worth a handful of coins. Basically, you need to have enough coins to cope with a few days’ mailing list mails, and enough to attach to all the mails you want to send. That might be, say, 5000c for the lists, and 20c a day for mails, which shouldn’t be too hard to manage. If you equate $1 with 1000c, it’s not particularly expensive to set up, and $0.005 per spam, would net me $350/yr, which ought to be horrendously unaffordable for spammers.

I think what would be nice is having blosxom see the files,

 -rw-r--r--    1 aj       aj            323 May 25 20:29 foo.txt
 -rw-r--r--    1 aj       aj             80 May 26 13:10 foo.txt-1
 -rw-r--r--    1 aj       aj            103 May 26 14:40 foo.txt-2

and magically translate that into an original entry, with two dated updates. You could then mark the files as uneditable once you’ve finished writing them, and not worry about accidently revising history.

UPDATE 2003/08/17:

I’m trying to implement this. Is it working?

UPDATE 2003/08/17:

Why yes, yes it is working! See here.

The biggest difficulty in running an experiment like that is to make sure that it’s possible to get involved in the market. That’s a wildly difficult undertaking: it essentially requires everyone who might ever want to send you email to obtain the tools and understanding to operate in the market. That can be made fairly easy, but it’s a step up on just using regular email. On the other hand, the spam problem has gotten bad enough that people are doing that sort of thing anyway, often in ways that will simply continue the spam arms race in ways we don’t particularly like (eg, spammers faking From: lines to pretend to be from people you know).

Our goal then is to add a pricing system to email, so that you can set a fee for incoming mail. We want this to be as easy as possible, both for senders and receivers, and we want it to be as flexible as possible, so that you can have ways of charging your friends less than strangers, or use the fees you collect from email for other things — most obviously simply reimbursing whoever sent you that gem of an email.

Our first step is to get some functioning electronic money. Paypal and credit cards are an obvious first choice, but both are utterly unacceptable due to high overheads (currency conversion fees, transaction fees, %ge fees, setup fees, etc). They’re also not very efficient at a protocol level — having to receive an email (from paypal, confirming a deposit) is a lot of overhead just to, well, receive an email. Efficient protocols for digital cash have been known for a while, but unfortunately have been mostly unusable and unused due to David Chaum’s patents in the area, and the failure of his company, DigiCash. Ben Laurie’s Lucre, based on an idea by David Wagner seems to be a workable and unpatented alternative.

Workable, in this context, means that it’s possible to have a number that represents a coin, and to be able to “give” that coin to someone else, without having any way of retaining it yourself — people don’t forget their own phone numbers when they give it to someone else, for example. There are two possible tricks here: the easiest is to talk to a “bank” everytime you spend money, and have them record the number you used, and create a new number for the shop keeper to spend later. If you try to give the same number to someone else, the bank will look it up in its list, and say “no”, and that’s that, the trade doesn’t take place, no one’s worse off. This is a nuisance, since you have to ask permission from a bank everytime you want to spend money, both for you and the bank, but it’s workable. The other way to attack the problem is to allow you to use the number twice or more, but to catch you when you do. This can allow you to spend money without talking to the bank immediately, if you’re also willing not to catch crooks immediately. The problem with this, is that if you don’t catch the crooks immediately, you might not ever be able to catch them, and if the crooks can manage to make that fairly likely (by, say, stealing a credit card, withdrawing $5, then spending that $5 a million times in a million places around the world, all in under a couple of minutes), well, you’re probably pretty screwed.

So, we’ll go with Lucre, for the time being.

We’ll make a few simplifying assumptions. First, the currency won’t be backed. That is, it won’t be tied to the dollar, or to gold, or to anything else. You can exchange it for other things freely, but you aren’t guaranteed to get any particular rate. This means that you need to be careful about monetary policy, but it also means you can minimise up-front costs, and give emoney away for free without running any personal risk.

Second, the bank won’t have accounts. Since each coin is useless once its spent (you can’t rely on it since whoever you gave it to could use it first, and they can’t rely on it since you could spend it again), we’ll simply allow you to go to the bank and get a coin changed for another whenever you like — which is to say, every deposit at the bank will also necessitate a withdrawal of the same amount. This doesn’t add any risk or remove any flexibility — it just exchanges account numbers for coin numbers, and removes a layer of state, ie complexity.

Third, we’ll expect there to be many different currencies, all competing. If I can set this up, obviously so can others. Further, competition is a good thing. This doesn’t simplify things technically, but it does simplify things globally — it means you can distribute the load of all the world’s transactions over many currencies, which is a good thing if every transaction in a given currency has to go through a single bank.

Fourth, as much of the cash management as possible will be done using an electronic market. This particularly concerns “buying” electronic cash (trading real money for electronic money, or some other limited resource for electronic money), and converting between different electronic currencies. The theory here is that a single “market” interface should be the standard for as much trade as possible. It will need to provide, in code and reputation, the ability to trade with people you don’t trust remotely — which means binding contracts. There are probably other caveats.

And fifth, we need what I’ll call an $MTP protocol that allows you to send money and mail, and correctly deal with errors throughout the transmission. That’s going to be fairly tricky. In fact, doing Lucre reliably is probably a little tricky, and will require something like “journalling” to ensure system crashes during transactions don’t result in lost money.

And thus ends today’s discussion. Also of interest is Lucrative, who call ecoins “digital bearer instruments”, and expect things to be strongly backed.

Thomas Sowell makes a very convincing argument for markets in his book Basic Economics. In principle, the idea is that markets allow people to redistribute scarce goods (food, labour, materials, land, etc) according to where it’s most useful to society. The more creative you are, the more and better work you do, the more other people trust your judgement, the more say you get, which is to say, the more money you have. It’s not particularly difficult to argue that most of the flaws in a market economy (lack of jobs, lack of investment) are due to misprioritising the desirability of possible outcomes. A possible conclusion is that the market system can’t do away with philanthropy and altruism, but can reduce they’re necessity immensely, and can also reduce government intervention massively.

Anyway, while that’s a possible conclusion, it’s not the one we’re going to work from. What we’re going to work from instead is the theory that free markets are the best possible way of allocating scarce resources. That might or mightn’t be true, but there’s a lot of evidence for it, so it’s worth considering.

Our additional assumption is that the time I’m willing to spend only a limited amount of time reading emails, and that there’s more email that would like me to read it than I have time for. This assumption, and the preceeding theory have an obvious implication: that there should be a market in which sellers (people with email addresses), and buyers (spammers, admirers, confidants, whatever) can negotiate a fair price, and, further, creating such a market is the fairest and most efficient way of dealing with the spam problem.

This is a straightforward conclusion in a mathematical sense. The only way, at least as far as I can see, to disagree with it is to disagree with one or both of the assumptions: that is to say either that free markets are not the most effective way of allocating scarce resources, or that I don’t get too much email. Which should leave us an interesting experiment that has two possible outcomes: either an effective means of reducing spam, or a demonstation that free markets are not effective.

So, let’s just ignore the problem of working out what sort of sliding scale should be used. We’ll propose that (a) it should be non-decreasing, and (b) it’s set by colleges in response to market forces. It can be linear, flat, exponential, polynomial, logarithmic, whatever. But what it does have to be is set, and made public, in advance.

What then does that mean for students trying to enter college? When I was applying for university (straight out of high school) we had to make our applications for courses and universities before we’d finished our exams, let alone had our results, and we had to go on what last year’s cut-offs were. With a setup like the above though, instead of saying “I can probably get into these courses, and these are my preferred three”, you instead have to say “I’d prefer to get into these courses, but, depending on my grades, I can only afford to pay so much”. Effectively, this means bidding on a course, with the result that if you do better, you get in cheaper (yay!), and if you don’t do well enough to get in under your budget, you go elsewhere.

Which sounds basically workable, although the details are likely to be complicated. You can’t really set the fees in advance, since that could result in you having more students than you can handle — and if there’s that much demand, a real market should simply increase the prices. That seems difficult to regulate well, especially if the form of the curve the fees take can fluxuate after the university knows how much everyone’s willing to pay.

Going back to the original point of this, that allows you to give bright but poor minority kids affirmative action scholarships so that they can either get into courses that they’re extremely well qualified for, but could otherwise not afford (and it’s probably in society’s interest to do that for anyone who’s poor and bright), or so that they can get into courses they’re underqualified for, in the same way that scions of industry barons do. Maybe they end up doing poorly (maybe the scions do too), maybe they get extra tutoring (paid for as part of the scholarship), or maybe they excel when they’re finally given the opportunity — the affirmative action goals are met by: not introducing any bias within the college based on skin colour, gender, etc; giving them an extra chance to pull themselves up and act as role models for others; and letting the presumably exclusive majority have a chance at seeing a minority member do well, and losing some prejudices.

Loans, scholarships, rich parents, part-time jobs, and competition amongst schools should make such a thing plausible; encouraging both excellence at the high end (where bright students will be attracted with their scholarships, and rich parents with their oodles of money), and at the low end (where there’re financial incentives to make even small improvements in your grades, rather than just coasting along with a C). Of course, you’ve also got an incentive to threaten and bribe the people giving you the grades so you have less expense down the track; which would probably result in fairly heavy standardised testing.

This also allows the government and colleges to back off from affirmative action if they want — scholarships can equally well be provided by foundations, charities, philanthropists, or collectives.