This page has been migrated from an earlier version of this site. Links and images may be broken.
Here’s a draft of something I’ve been working on over the weekend…
On a hot summer’s day a few years ago I was waiting at a train station in the UK. The train was late. A voice came over the public address system saying “We apologize for the late arrival of the 10:30 to Brighton”. Fair enough. Then ten minutes later, the same voice, in the same intonation, said the same words: “We apologize for the late arrival of the 10:30 train to Brighton”. Ten minutes more, the same thing. And so on for an hour or so until the train finally arrived.
What stayed with me was not the lateness of the train — the day was so hot that tracks were buckling and we knew on setting off that we were in for delays — it was that announcement. The words themselves sounded like an apology, but, just like “your call is important to us”, the message that came through to me was the exact opposite of what the words themselves said.
Literature scholars have a notion of something called “performative language”. The idea is that while most utterances simply talk about the world (“That daisy is yellow”) others actually perform an action. “I resign”, when spoken to your boss, is not just a statement, it is an action that changes your status from employed to unemployed. “I do” when said in the right place and in front of the right audience, actually changes the speaker’s status from unmarried to married.
“We apologize” can be a performative utterance. When offered in the correct way, saying you’re sorry is not just a statement about the world, it is an act: the act of apologizing. But it’s not always a performative utterance. It needs two things to make it so.
First, and most obviously, someone needs to say it. Without an apologizer, there can be no apology. An automated system cannot apologize.
It could be argued that on that railway platform, even though there was no human individual speaking, the one making the apology was the company. The recorded voice actually comes from the railway company, and was simply transmitted by the automatic system, just like a letter carrier delivers a letter. But it turns out there is a problem with this too, and the problem is cost.
Apologizing, as we all know, can be a difficult thing to do, and that difficulty is part and parcel of the act of apologizing. Imagine receiving an apology for a deeply painful betrayal delivered casually, in a way that made it clear it carried no more emotional cost than commenting on the weather. You would understand immediately that it was not an apology at all, even if it contained the words “I apologize”. It is “cheap talk”, and while it could be an attempt to get you to be quiet and leave the offender alone, or to postpone the inevitable recriminations, or one of several other things, cheap talk can’t be an apology.
So when an automated system says “we apologize” then there is, in fact, no apology happening, no matter how often it says it. When the only action it takes to deliver the words is pushing a button somewhere, at the most, then the statement is “cheap talk”: cost-free communication that should carry no weight.
We are getting to understand the business of “speech without a speaker”. When we hear an automated voice saying “Your call is important to us” most of us do receive the real message. We know that if our call was really important then there would be a person there to receive it. The words are not the message; the automated system that delivers it is.
We understand that the letters we get from company CEOs offering us new deals do not, of course, come from the CEO at all. He (or occasionally she) may not even have read it, let alone written it. May not even know of its existence.
“Have a nice day” is a variant on the same theme: the phrase is often said by a real person, so it seems like there is a speaker, but if the hapless individual behind the till is reciting a script, and if delivering that script is a condition of their employment, then they are not really the speaker; they are simply passing on a message from someone else. And that someone else has no interest in the niceness (or otherwise) of your day: it is simply a calculated policy to give the appearance of friendliness.
On occasion, of course, someone behind a till is in a friendly mood, and actually gives the words real meaning. But most of us can tell the difference between such a sincere utterance and someone delivering a programmed script.
But while we are adjusting to the problem of speech without a speaker, we have not really started adjusting to the other changes that cheap communications technology are bringing. We know that our mediums of communication are changing, with new forms of internet-based communications mutating every few years, from e-mail to instant messaging to weblogs and more. What we need to adjust to is the way that the economics of these new media change the meaning of the messages we send and receive.
Many of us do adjust to these changes in meaning in our personal communications. A Christmas letter sent by e-mail to a long list of recipients means less to us than an individually handwritten letter. But our social systems are slower to change. In particular, there are forms of legal communication that have built into them an assumption of cost, but which can now be made very cheaply.
A new communication mutant was described recently by Mark Rasch, former head of the US Justice Department’s computer crime unit, in The Register, an online journal of technology comment. Rasch writes about “robolawyers” or “lawyerbots”: programs designed to search the Internet for suspected violations of a company’s copyright or trademark. A lawyerbot can be designed not only to detect possible violations, perhaps by searching for keywords or combinations of words, it can also be designed to send out legal warnings (Rasch calls them “takedown notices”) to the owners and hosts of the sites involved.
The legality of these automated takedown notices has not been tested. But they don’t have to go to court to be effective. Internet Service Providers and other web site hosts such as eBay don’t like the possibility of lawsuits, and their legal liability for the content of the sites that they host is uncertain, so these notices are often enough to persuade them to take down the problematic material, regardless of whether there is anything wrong with it or not.
Rasch writes about Brian Kopp, who wrote a book based on the popular online game “World of Warcraft” and offered it for sale on the eBay auction site. The book was, by all accounts, in the spirit of those “Dummies” guides that tell you how to use Microsoft Word and other software products – an unauthorized third party guide, but not defamatory or illegal. Once Kopp posted his book, however, eBay began receiving a stream of “takedown notices” telling them that the book violated the rights of Blizzard, the company that owns World of Warcraft, and that they should remove it from their site. eBay responded by not only removing the book, but cancelling Kopp’s account so that he could not auction anything else either.
It is impossible to know for sure whether there were any real people involved at any stages of this campaign against Brian Kopp (who is almost certainly real, although not a very good writer). Rasch notes that Kopp’s eBay auction generated “a slew of takedown notices from various parties. As soon as he reposted the auction, it generated a new takedown notice. Human lawyers are generally not that efficient. So these autonomous agents may in fact be the ones generating these takedown notices.” So did any real person at Blizzard know they were sending takedown notices to Mr. Kopp? Quite possibly not. Had anyone (apart from a computer) “read” his book? Probably not. Did an individual at eBay weigh the evidence and decide what action to take, or does eBay have a program to disable accounts if it receives a sufficient number of complaints? Unclear.
The fact that legal letters rarely go to court, and the fact that they can be generated programmatically, means that automated legal notices are likely to increase. We are already subject to automated legal checks whenever we install new software, in those End User License Agreements that we have to say we’ve read even though we obviously haven’t, and even though they are not designed to be read. Increasingly, those agreements include clauses that let software report back to the company that made it about the use we make of it, or details about our computer. Such information can have legitimate uses, enabling better fixing of software problems for example, but it can also have illegitimate uses.
There is an assumption in our legal system that a legal notice follows some kind of investigation, however cursory, on the part of the party that issues it. That is, it is a document to be taken seriously. But auto-generated and widely distributed legal notices are “cheap talk”, and their increasing ease of generation means that, like the apology at the railway station, they should be treated as meaningless.
It would be easy to dismiss Brian Kopp’s case as absurd if it wasn’t so clearly a harbinger of things to come. As lawyerbots proliferate, companies will find it worthwhile to develop other lawyerbots to handle incoming “letters” and we will have legal communications that nobody writes and nobody reads. If a letter is delivered and nobody reads it, does it carry any meaning? If lawyerbots were just sending meaningless messages back and forth, nothing would be lost except for a bit of Internet bandwidth, but it seems that the legal system has not yet adjusted to this new mutant, and so such communications can be damaging.
Increasing automation means that forms of communications are mutating ever more quickly. Some developments are not difficult to foresee. As with all things computer-related, it is clear that automated communication is going to be more common – a lot more common – in the future than it is now. The near future will bring automated translation software that can provide good enough translation in a wider and wider set of circumstances. It will also bring speech comprehension software, and text-to-speech software of increasing subtlety: no more jerky sentences with those pasted together words – the audio equivalent of the ransom note pasted together from newspapers. Put them all together and programs will soon be carrying out conversations with people around the world, persuading us to part with our money in new and grammatically sophisticated ways, using voices designed to exploit our sympathies and prejudices.
Except they are not voices and not conversations at all, they just sound like they are. We need to be consistently on our guard if we are to avoid becoming confused by these mutant communications. As a first step, we need to acknowledge that machine-delivered apologies are not apologies, no matter how sincere they sound. And we need to decide that lawyerbot-delivered threats and warnings carry as much weight as the cost of delivering them, which is zero.