technical – Page 2

SOPA

I never blogged on the SOPA kerfuffle; it happened while my creative(?) energies were elsewhere.

Looking back, a few minor points emerge:

Some commentators got all excited: “look what we did! What shall we do next?!” “We” meaning right-thinking internet-type people. The answer, obviously, is nothing: this, “we” agreed about, most things, we don’t. I think Wikipedia’s claim: “Although Wikipedia’s articles are neutral, it’s existence is not” was basically justified.

Libertarian commentators had a lot of fun jeering at leftist techies who wanted every aspect of the economy to be regulated by the government except the internet. The criticism is only justified against those who demand that government regulate things but don’t specify exactly how they should regulate them (others can say they’re in favour of regulation, but just want it to be better). But that’s most people. So yeah.

In some ways, it’s a disappointment that SOPA didn’t go through; the circumvention techniques that would have been developed if it had would have been interesting and useful. At the end of the day, the biggest threat to free computing isn’t legislation, it’s that in a stable market, locked-down “appliance” devices are more useful to the non-tinkering user than general-purpose, hackable devices. So far, we tinkerers still have the GP devices, because the locked-down ones go obsolete too quickly even for lay users. I’m not sure whether that situation will persist for the long term: I’ve looked at the question before.

But if the government makes stupid laws that can easily be circumvented using general-purpose devices, the demand for those devices will be helpfully supported.

Note when I talk about circumvention, I’m not talking about copyright infringement. That was not what the argument was about. While I lean toward the view that copyright is necessarily harmful, I’m not certain and it’s not that big a deal. The important argument is all about enforcement costs: given that copyright exists, whose responsibility is it to enforce it. The problem with SOPA was that it would have put crippling copyright enforcement costs on any facilitator of internet communication.

Currently, internet discussion is structured mostly around large service providers — in the case of this blog Google — providing platforms for user content. If those service providers become legally liable for infringing user content, the current structure collapses. The platforms would either have to go offshore, with users relying on the many easy ways of circumventing the SOPA provisions attempting to limit access to offshore infringers, or else evade the enforcers by going distributed, redundant and mobile. What will be to Blogger as Kazaa and then BitTorrent were to Napster? It would have been interesting to find out, and possibly beneficial. There is a lot of marginal censorship that can be applied to easy-target platforms like Blogger or Wikipedia that will not induce sufficient users to create alternatives, as the sheer idiot clumsiness of SOPA would probably have done.

(Note Wikipedia might have been spared, but it would have suffered, because if existing less respectable platforms were removed, their content would migrate to the likes of Wikipedia. If 4chan did not exist, Wikipedia would become 4chan.)

Actually, it’s interesting to think about how to blog over a pure P2P framework. Without comments, you’re publishing a linear collection of documents. (I don’t think you can handle comments — we’d need something more like trackbacks). Posts would need to be cryptographically signed and have unique ids. Serial numbers would be useful so readers would know if they’d missed anything. I wonder if anyone’s worked on it. A sort of bittorrent-meets-git hybrid would be really interesting — search this list of hosts for any git commits signed by any of these keys…

The dance of censorship and evasion is very difficult to predict in detail. I found some time ago that the way to find the text of an in-copyright book is to take a short phrase from it (that isn’t a well known quotation or the title) and google it. That used to work. I wanted some text from Evelyn Waugh’s Decline and Fall the other day, so I did the usual, and got pages and pages of forum posts, containing chunks of the book interspersed with links to pages selling MMO currency and fake LVMH crap. My access to illicit literature was being messed up by someone else’s illicit SEO.

AI, Human Capital, Betterness

Let me just restate the thought experiment I embarked on this week. I am hypothesising that:

“Human-like” artificial intelligence is bounded in capability
The bound is close to the level of current human intelligence
Feedback is necessary to achieving anything useful with human-like intelligence
Allowing human-like intelligence to act on a system always carries risk to that system

Now remember, when I set out I did admit that AI wasn’t a subject I was up to date on or paid much attention to.

On the other hand, I did mention Robin Hanson in my last post. The thing is, I don’t actually read Hanson regularly: I am aware of his attention to systematic errors in human thinking; I quite often read discussions that refer to his articles on the subject, and sometimes follow links and read them. But I was quite unaware of the amount he has written over the last three years on the subject of AI, specifically “whole brain emulations” or Ems.

More importantly, I did actually read, but had forgotten, “The Betterness Explosion“, a piece of Hanson’s, which is very much in line with with my thinking here, as it emphasises that we don’t really know what it means to suggest we should achieve super-human intelligence. I now recall agreeing with this at the time, and although I had forgotten it I suspect it at the very least encouraged my gut-level scepticism towards superhuman AI and the singularity.

In the main, Hanson’s writing on Ems seems to avoid the questions of motivation and integration that I emphasised in part 2. Because the Em’s are actual duplicates of human minds, there is no assumption that they will be tools under our control; from the beginning they will be people with which we will need to negotiate — there is discussion of the viability and morality of their market wages being pushed down to subsistence level.

There is an interesting piece “Ems Freshly Trained” which looks at the duplication question, which might well be a way round the integration issue (as I wrote in part 1, “it might be as hard to produce and identify an artificial genius as a natural one, but then perhaps we could duplicate it”, and the same might go for an AI which is well-integrated into a particular role).

There is also discussion of cities which consist mainly of computer hardware hosting brains. I have my doubts about that: because of the “feedback” assumption at the top, I don’t think any purpose can be served by intelligences that are entirely isolated from the physical world. Not that they have to be directly acting on the physical world — I do precious little of that myself — but they have to be part of a real-world system and receive feedback from that system. That doesn’t rule out billion-mind data centre cities, but the obstacles to integrating that many minds into a system are severe. As per part 2, I do not think the rate of growth of our systems is limited by the availability of intelligences to integrate into them, since there are so many going spare.

Apart from the Hanson posts, I should also have referred to an post I had read by Half Sigma, on Human Capital. I think that post, and the older one linked from it, make the point well that the most valuable (and most renumerated) humans are those who have been succesfully (and expensively) integrated into important systems.

Relevance of AI

I felt a bit bad writing the last post on artificial intelligence: it’s outside my usual area of writing, and as I’d just admitted, there are a number of other points within my area that I haven’t got round to properly putting in order.

However, the questions raised in the AI post aren’t as far from the debates Anomaly UK routinely deals in as I first thought.

Like the previous post, this falls firmly in the category of “speculations”. I’m concerned with telling a consistent story; I’m not even arguing at this stage that what I’m describing is true of the real world today. I’ll worry about that when the story is complete.

Most obviously, the emphasis on error relates directly to the Robin Hanson area of biases and wrongness is human thinking. It’s not surprising that Aretae jumped straight on it. If my hypothesis is correct, it would mean that Aretae’s category of “monkeybrains”, while of central importance, is very badly named: the problems with our brains is not their ape ancestry, but their very purpose: attempting to reach practical conclusions from vastly inadequate data. That is what we do; it is what intelligence is, and the high error rate is not an implementation bug but an essential aspect of the problem.

(I suppose there are real “monkeybrains” issues in that we retain too high an error rate even when there actually is adequate data. But that’s not the normal situation)

The AI discussion relates to another of Aretae’s primary issues: motivation. Motivation is getting an intelligence to do what it ought to be doing, rather than something pointless or counterproductive. When working with human intelligence, it’s the difficult bit. If artificial intelligence is subject to the problems I have suggested, then properly specifying the goals that the AI is to seek will quite likely also turn out to be the difficult bit.

I’m reminded in a vague way of Daniel Dennett’s writings on meaning and intentionality. Dennett’s argument, if I remember it accurately, is that all “meaning” in human intelligence ultimately derives from the externally-imposed “purpose” of evolutionary survival. Evolutionary successful designs behave as if seeking the goal of producing surviving descendants, and seeking this goal implies seeking sub-goals of feeding, defence, reproduction, etc. etc. etc. In humans, this produces an organ that explicitly/symbolically expresses and manipulates subgoals, but that organ’s ultimate goal is implicit in its construction, and not subject to symbolic manipulation.

The hard problem of motivating a human to do something, then, is the problem of getting their brain to treat that something as a subgoal of its non-explicit ultimate goal.

I wonder (in a very handwavy way) whether building an artificial intelligence might involve the same sort of problem of specifying what the ultimate goal actually is, and making the things we want it to do register properly as subgoals.

The next issue is what an increased supply of intelligence would do to the economy. Though an apostate libertarian, I have continued to hold to the Julian Simon line that “Human inventiveness is the Ultimate Resource”. To doubt that AI will have a revolutionarily beneficial effect is to reject Simon’s claim.

Within this hypothesis, the availability of humanlike (but not superhuman) AI is of only marginal benefit, so Simon is wrong. Then, what is the ultimate resource?

Simon is still closer than his opponents; the ultimate resource (that is the minimum resource as per the law of the minimum) is not raw materials or land. If it is not intelligence per se, it is more the capacity to endure that intelligence within the wider system.

I write conventional business software. What is it I spend my time actually doing? The hard bit certainly isn’t getting the computer to do what I want. With modern programming languages and and tools, that’s really easy — once I know what it is I want. There used to be people with the job title “programmer” whose job it was to do that, with separate “analysts” who told them what the computer needed to do, but the programmer was pretty much an obsolete role when I joined the workforce twenty years ago.

Conventional wisdom is that the hard bit is now working out what the computer needs to do — working with users and defining precisely how the computer fits into the wider business process. That certainly is a significant part of my job. But it’s not the hardest or most time-consuming bit.

The biggest part of the job is dealing with errors: testing software before release to try to find them; monitoring it after release to identify them, and repairing the damage they cause. The testing is really hard because the difficult bits of the software interact with multiple outside people and systems, and it’s not possible to fully simulate them. New software can be tested against pale imitations of the real world, and if it’s particularly risky, real users can be reluctantly drafted in to “user acceptance” testing of the software. But all that — simulating the world to test software, having users effectively simulate themselves to test software, and running not-entirely-tested software in the real world with a finger hovering over the kill button — is what takes most of the work.

This factor is brought out more by the improvements I mentioned in the actual writing of software, but it is by no means new. Fred Brooks wrote in The Mythical Man-Month that if writing a program took n days, integrating it into a system would take 3n days, properly productionising it (so that it would run reliably unsupervised) would take 3n days, and these are cumulative, so that a productionised, integrated version of the program would take something like ten times as long as a stand-alone developer-run version to produce.

Adding more intelligences, natural or artificial, to the system is the same sort of problem. Yes, they can add value. But they can do damage also. Testing of them cannot really be done outside the system, it has to be done by the system itself.

If completely independent systems exist, different ideas can be tried out in them. But we don’t want those: we want the benefits of the extra intelligence in our system. A separate “test environment” that doesn’t actually include us is not a very good copy of the “production environment” that does include us.

All this relates to another long-standing issue in our corner of the blogosphere: education, signalling and credentialism. The argument is that the main purpose of higher education is not to improve the abilities of the students, but merely to indicate those students who can first get into and then endure the education system itself. The implication is that there is something very wrong with this. But one way of looking at it is that the major cost is not either producing or preparing intelligent people, but testing and safely integrating them into the system. The signalling in the education system is part of that integration cost.

Back on the Julian Simon question, what that means is that neither population nor raw materials are limiting the growth and advance of civilisation. Rather, civilisation is growing and advancing roughly as fast as it can integrate new members and new ideas. There is no ultimate resource.

It is not an original observation that the things that most hurt our civilisation are self-inflicted. The organisation of mass labour that produced industrialisation also produced the 20th century world wars. The flexible allocation of capital that drove the rapid development of the last quarter century gave us the spectacular misallocations with the results we’re now suffering.

The normal attitude is that these accidents are avoidable; that we can find ways to stop messing up so badly. We can’t. As the external restrictions on our advance recede, we approach the limit where the benefits of increases in the rate of advance are wiped out by more and more damaging mistakes.

Twentieth Century science-fiction writers recognised at least the catastrophic risk aspect of this situation. The concept that the paucity of intelligence in the universe is because it tends to destroy itself is suggested frequently.

SF authors and others emphasised the importance of space travel as a way of diversifying the risk to the species. But even that doesn’t initially provide more than one system into which advances can be integrated; at best it reduces the probability that a catastrophe becomes an extinction event. Even if we did achieve diversity, that wouldn’t help our system to advance faster, unless it encouraged more recklessness — we could take a riskier path, knowing that if we were destroyed other systems could carry on. I’m not sure I want that; it raises the same sort of philosophical questions as duplicating individuals for “backup” purposes. In any case, I don’t think even that recklessness would help: my point is not just that faster development creates catastrophic risk, but that it increases the frequency of more moderate disasters, like the current financial crisis, and so wipes out its own benefits.

Speculations regarding limitations of Artificial Intelligence

An older friend frequently asks me, as a technologist, when computers will have human-like intelligence, and what the social/economic effects of that will be.

I struggle to take the question seriously; AI is something that was dropped as a major research goal around the time I was a student twenty years ago, and it’s not an area I’m well-informed about. As I mentioned in my review of the rebooted “Knight Rider” TV series, a car that could hold up a conversation is a more futuristic idea in 2008 than it was back when David Hasselhof was doing the driving.

And yet for all that, it’s hard to say what’s really wrong with the layman’s view that since computing power is increasing rapidly, it is an inevitability that whatever the human brain can do in the way of information processing, a computer should be able to do, quite possibly within the next few decades.

But what is “human-like intelligence”? It seems to me that it is not all that different from what the likes of Google search or Siri do: absorb vast amounts of associations between data items, without really being systematic about what the associations mean or selective about their quality, and apply some statistical algorithm to the associations to pick the most relevant.

There must be more to it than that; for one thing, trained humans can sort of do actual proper logic, about a billion times less well than this netbook can, and there’s a lot of effectively hand-built (i.e. specifically evolved) functionality in a some selected pattern-recognition areas. But I think the general-purpose associationist mechanism is the most important from the point of view of building artificial intelligence.

If that is true, then a couple of things follow. First, the Google/Siri approach to AI is the correct one, and as it develops we are likely to see it come to achieve something resembling humanlike ability.
But it also suggests that the limitations of human intelligence may not be due to limitations of the human brain, so much as they are due to fundamental limitations in what the association-plus-statistics technique can practically achieve.

Humans can reach conclusions that no logic-based intelligence can get close to, but humans get a lot of stuff wrong nearly all the time. Google Search can do some very impressive things, but it also gets a lot of stuff wrong. That might not change, however much the technology improves.

There are good reasons to suspect that human intelligence is very close to being as good as it can get.
One is that thinking about things longer doesn’t reliably produce better conclusions. That is the point of Malcolm Gladwell’s “Blink” (as far as I understand it; I take Gladwell to be the champion of what Neal Stephenson called “those American books where once you’re heard the title you don’t even need to read it”).

The next, related, reason is that human intelligence doesn’t scale out very well; having more people think about a problem doesn’t reliably give better answers than having just one do it.

Finally, the fact that, in spite of evolutionary pressure, there is enormous variation in the practical usefulness of human intelligences, suggest that making it better is not simply a case of improving the design. If the variation were down to different design, then the better designs would have driven out the worse ones long ago. I think it is far more to do with circumstances, and with the fundamental difficulty of identifying the correct problems to solve.

The major limitation on conventional computing is that it can only do so much per second; only render so many triangles, only price so many positions or simulate so many grid cells. Improving the speed and density of the hardware is pushing back that major limitation.

The major limitation on human intelligence, particularly when it is augmented with computers as it generally is now, is how much it is wrong. Being faster or bigger doesn’t push back the major limitation unless it can make the intelligence wrong less often, and I don’t think it would.

What I’m saying is that the major cost of human intelligence is not in the scarce resources required to execute the decision-making, but the damage caused by all the bad decisions that humans make.

The major real-world expense in obtaining high-quality human decision-makers is identifying which of the massive surplus available are actually any good. Being able to supply vastly bigger numbers of AI candidates would not drive that cost down.

Even the specialisms that humans have might be limited more by the cost they impose on the quality of general decision-making than by the cost of actually implementing the capability.

If that’s the situation, then throwing more computing resources at AI-type activity might not change things that much: computers can be as intelligent as humans, but not more intelligent. That’s not nothing, of course: it opens the door to replacing a lot of human activity with automated activity, with all the economic effects that implies.

There will be limitations in application because if human-like intelligence really is what I think it is, then the goals being sought by an AI are necessarily as vague as everything else: they will be clumps of associations, and the “intelligence” will just do the things that are associated with the goal clump. We won’t be able to “program” it the way we program a logic-based system, just kind of point it in the right direction in the same we we do when we type something into a Google search box.

I don’t know if what I’ve put here is new: I think the view of what the major issue in intelligence is is fairly widespread (“associationism”?), but in all previous discussions I’ve seen or participated in, there’s been an assumption that if in x years from now we will have artificial human-like intelligence, then in 2x years from now, or probably much less, we will have amazing superhuman artificial intelligence. That is what I am now doubting.

With intelligences available “in the lab” we might be able to prepare and direct them more effectively than we do now. But even that’s not obviously helpful: with human education, again, the limitation is not so much how long it takes and how much work it is, rather how sure we are it is actually doing any good at all. We may be able to give an artificial intelligence the equivalent of a hundred years of university education, but is a person with that experience really going to make better decisions? The things we humans work most hard at learning and doing: accumulating raw information and reasoning logically, are the things that computers are already much better than us at. The things that only humans can do are the things we simply don’t know how to do better, even if we were to re-implement on an electronic platform, speeded up, scaled up, scaled out.

Note that all the above is the product of making statistical guesses using masses of ill-understood unreliable associations, and is very likely to be wrong.

(Further thoughts: Relevance of AI)

Freemail

In The Guardian, a journalist tells of her experience of having her email account hacked.

“The realisation dawns that the email account is the nexus of the modern world. It’s connected to just about every part of our daily life, and if something goes wrong, it spreads. But the biggest effect is psychological. On some level, your identity is being held hostage.

“The company that presents itself as the friendly face of the web doesn’t have a single human being to talk to in these circumstances.”

I love free stuff. I use free blog services and free email services, and I see it as a double advantage that, as well as not costing me anything, these services are somewhat at arms length from my identity. Possession of a few keys and passwords are what make me “anomalyuk”, nothing more than that.

My real-world identity is another matter. My personal email accounts, with which I support my personal relationships and business relationships, are provided to me — here’s a novelty — as a paying customer. The providers’ customer services may be good or bad, but at least they exist and I can use them. It makes no difference to a Gmail user how good Google’s customer service is, because Ms Davis and other Gmail users are not Google’s customers at all.

I actually pay a couple of quid a month just for my email service, but that isn’t necessary. Like you, Rowena Davis has an ISP — possibly more than one, if she gets her mobile separate from her home internet. They will provide her an email address, as part of the service she is paying for. They know it belongs to her, because she pays the bill, and if, as the bill-payer, she phones up and needs it reset, they will do it for her. However, for this service which she correctly observes is the nexus of her life, she has chosen to rely instead on a handed-out-on-the-street freebie instead.

I hereby declare that to be a Bad Idea.

Davis’s story links to another recent one, of a 79-year-old charity volunteer who went through the same ordeal. Twice. The police told her: don’t use free email services. Her conclusion at the end of the article: the police need to devote more resources. Not her — she’s sticking with free.

There is one drawback with using your ISP’s email service, which is that you may lose it if you want to change ISPs. As it happens, two generations of free services have come and pretty much gone (remember bigfoot? rocketmail?) in the time I’ve been with my current ISP, but that may be a fluke. And in any case, the old addresses are still supported.

If that concerns you, then do what I do and pay for it. One leading provider charges 69p a month for email hosting, plus £2.99 a year for domain registration — giving you an address that is transferable across providers and that looks more professional than a vodaphone or gmail address. And they have 24×7 telephone support. Alternatively, Yahoo! do an email service for $19.99 a year. Bigfoot, it emerges, are still around, and charge $19.95 a quarter. Is £1 or £3 a month really not worth paying for “the nexus of the modern world”? I should emphasize: it’s not just that paying for the email makes it feasible for the provider to offer you some level of support: the mere fact of there being a payment makes it enormously easier for them to identify you, and therefore to clear up these fraud issues.

The surprising thing is that they’re not marketing this more aggressively. The problems Davies had have been common for a few years: everyone in her position should be paying for decent email, but the providers aren’t advertising on that basis. Google don’t offer a premium service like Yahoo’s, Microsoft charge $9.95 a month, which is a bit steep, and the services just aren’t marketed.

ISPs could offer domain and mail hosting as an extra, but the consumer-oriented ones don’t, or don’t push it.

Possibly the providers are worried about adverse selection: if they advertise on the basis of being able to handle hacking incidents, they’re offering hostages to fortune in terms of the inevitable dissatisfied customers undermining their name with complaints.

As a disinterested (and irresponsible) third party, I will do it for them: Do not use Gmail. Do not use MSN Hotmail, unless you are paying the $9.95 a month for premium (which I don’t recommend, because it’s too much). Use your ISP’s email account if you’re not planning to move or switch in the next five years. Otherwise get a personal domain and get a basic email service from the likes of 1and1, or, if that’s too complicated (and it is a bit complicated), get Yahoo! Plus for $19.95 a year. I’m not recommending these through experience, just through looking for email services that cost a little money and offer telephone support.

If you’re not willing to pay, or you’re not willing to give up Gmail (which, I admit, is a very nicely done service), then remember that you have nobody to whine to if your Gmail is hacked. You have other options, and you have chosen to trust your email to a company you have no commercial relationship with. I have nothing against Google, but if you want a company to have responsibilities towards you, you have to pay them.

Anonymous versus HBGary

I don’t think the HBGary story has had the amount of attention it deserves from the mainstream.

It’s worth reading just as drama: Security researcher takes on the “Anonymous” hacker group, and loses so spectacularly it almost defies description.

It’s important for what it says about any organisation’s IT choices and their security implications. HBGary used Google Apps. Cloud services are enormously convenient, particularly for an organisation that does not really have a physical “home”, but using them means losing perimiter security altogether.

Perimiter security has a bad name, because in the old days it was all there was, and today it is not enough. But the things that are possible even if you try to protect your perimiter are much easier if you don’t even have one.

A basic IT risk assessment question for anybody is, “how much damage can an attacker do with one password?”. With one password, Anonymous downloaded all of HBGary’s corporate email from Google and posted it on the internet. They did more than that — the highlight for security commentators was the social-engineering attack on rootkit.org via a Nokia engineer — but the email was enough by itself, as well as enabling the other attacks. They got the email admin password from an ad-hoc CMS with a SQL-injection vulnerability, as it happens, but if your whole company can be destroyed with one password then you’re doing it wrong. (Damn, it’s so hard to avoid lapsing into dialect on this story).

And the third interesting angle is what is to be found in the data Anonymous posted. The company was proposing to feed fake data to WikiLeaks to discredit it, and to pressure journalists who defended WikiLeaks. There is chatter about government involvement in this, but I haven’t seen that actually substantiated. It may be in there somewhere. The HBGary Federal projects aimed at government clients seem to be standard network monitoring / intrusion detection stuff.

In case anyone gets confused, I’m not here to defend Anonymous, or for that matter to attack them. They exist. If they get caught they’ll get the book thrown at them, which is understandable, but I’m more interested in what the world looks like with them in it. Whereas Assange attempts to define his aims, and appeals for support, Anonymous claim only to be “in it for the lulz”, which is not open to disputation.

Update: Intriguing piece on HBGary government work on rootkits and penetration tools. In principle this should be verifiable from the email dumps, but I haven’t checked.

Major threat to your email

I just came across this story, from a few months ago. I’m surprised it didn’t get more play, because it’s much more serious than the run-of-the-mill software vulnerability story.

PC’s are not secure, and never have been. For most of us, that hasn’t been a big concern. We try to keep viruses and bots off our systems, either by avoiding Windows or by more iffy and difficult methods. But that’s mostly due to a desire to keep our systems running and be good network citizens. But the risk of a personal attack on your system has always been a long shot, because, despite the fact there are many people who could read your email, there’s little reason any of them would want to. The sets of people who know how, and people who would care to, are small enough that their intersection is probably zero.

That calculation has now changed. If there is someone who has a grudge against you, or some other motive to want to read your email or impersonate you, and that person knows how to buy stuff on the internet, you are now at serious risk.

I’ve talked before about how to make your email secure, but it’s difficult to do reliably, and the advice in the article is probably best. If you want to keep stuff secret, don’t put it on a computer, unless you’re an expert.

Time to Tax Email

Prospect Magazine is written and edited by people who don’t know what they’re talking about and don’t care:

“A penny charge for every email would stop spam, and fill the empty public purse” – lead article by Edward Gottesman

If he – or the editor, had read my very brief little primer in email for novices and government ministers he would know that

Email is an addressing system and message format by which messages can be sent between users over the internet.

ISPs provide internet service. Sometimes they also provide web or email services over the internet as an add-on, and sometimes they don’t.

It is quite possible to send and receive email messages without one’s ISP even being aware of the fact. Indeed, most people do. If you have a large site, you probably run your own email servers. You emails go over your ISP’s internet service, but do not use your ISP’s email service, even if it has one.

Conversely, if you use webmail, your email does not reach your network in the form of messages – only web pages. Your messages originate or terminate with your webmail provider, who may well not even be in this country.

Only if you use the old-fashioned POP3+SMTP setup, or your ISP’s webmail service, will your ISP see your email as email. In some cases it might be possible for them, by searching your entire network traffic, to identify and extract email from your network flow. That involves a whole lot of processing that they would otherwise not need to do.

If you use an offshore webmail provider, they can’t even do that, because the traffic between you and the webmail provider is encrypted.

If he had done the smallest amount of research he would have known all that. If he had done, say, a day’s research, he would have already seen the check-the-boxes form objection to stupid spam-fighting schemes that inevitably landed on the prospect discussion blog.

There are of course many other reasons why it’s a moronic idea. But it only needs one.

Retreat into history

The reason I’ve gone very quiet of late is that two weeks ago I visited Bletchley Park, and was so fascinated by the details of the cryptanalysis of Enigma that I’ve spent every spare moment since working out the crib/bombe technique, and implementing software simulations to verify my understanding.

I had what I think was a working bombe simulator by last weekend, but running in ruby on my netbook, it was somewhat slower (for a moderately complex menu) than the 1942 electromechanical version. Not having the resources of a state war machine to draw on, that makes it a bit too time-consuming to actually test the process. Every optimisation I attempted made it slower, so I have resorted to a C++ port of my ruby code, which is not yet complete.

(I am aware that many simulators already exist – the point of my simulator is to demonstrate to myself that I know how it is supposed to work).

There’s loads of important stuff to write about, but I just can’t put this down right now.

Visual Security

The story of Bob Quick’s exposure of secret anti-terrorist documents to photographers outside Number Ten, and his subsequent resignation, is of course highly amusing. It does also highlight some significant issues.

The changes that information technology make to privacy and secrecy are changes what can be done with information that was always available. Most obviously, information once captured can be stored, searched and shared. But also, what was a glimpse can now easily be turned into something that can be analysed at leisure. The implications are not immediately obvious.

In the current case, new technology doesn’t really come into it. People have been taking pictures outside 10 Downing Street with good-quality cameras for a long time. Nonetheless, we now need to be aware that anything exposed to public view is potentially public property.

One example is the “Fake ATM” fraud, where criminals fit an extra magnetic strip reader onto an existing ATM, and also add a video camera to record the user entering their PIN. They then can clone the cards and use the PINs.

A possibility I’ve not heard of, but which occurred to me when I worked in a large shared office building, involves barcodes. The building issued temporary passes to guests which opened the security gates with a barcode. It should not be difficult to take a picture of somebody wearing such a badge, read the barcode from the image, and print a fake temporary pass with the same barcode which would then open the gate. I never got round to trying it, because I couldn’t find free software for reading and printing the barcodes.

This is just the beginning. The ubiquitous security video cameras do not, these days, produce images of sufficient quality to resolve text, barcodes etc. (except of course in CSI and the like, where they can resolve even minute off-screen detail via incidental reflections). But they are getting better. The same goes for cameras in phones, and “toy” concealable cameras. But the high end today of both security video and cellphones are probably about at the level where exposed text can be captured, and it is a matter of only a few years before such image quality becomes the norm.

Confidential documents are often exposed by people reading them while in transit, on trains and planes as well as getting in and out of official cars. Bob Quick got caught out because he was in a place where it was natural for him to be photographed directly with proper cameras. But someone hanging around Canary Wharf underground with a T929 could quite likely grab a fair bit of confidential information surreptitiously.

So if your documents are worth shredding rather than dropping in the bin, they’re worth keeping inside an opaque folder when in any public place.

Update: Via a commenter at Bruce Schneier’s, this has happened before