Effects before causes in the lab?

This is very exciting:

http://bps-research-digest.blogspot.com/2010/11/dramatic-study-shows-participants-are.html

Daryl Bem has taken the unusual, yet elegantly simple, approach of testing a raft of classic psychological phenomena, backwards.

Take priming – the effect whereby a subliminal (i.e. too fast for conscious detection) presentation of a word or concept speeds subsequent reaction times for recognition of a related stimulus. Bem turned this around by having participants categorise pictures as negative or positive and then presenting them subliminally with a negative or positive word. That is, the primes came afterwards. Students were quicker, by an average of 16.5ms, to categorise negative pictures as negative when they were followed by a negative subliminal word (e.g. ‘threatening’), almost as if that word were acting as a prime working backwards in time.

Got that? The experimenters have used the same techniques usually employed to see how various events affect people’s behaviour, but reversed the order of the stimulus and the measurement of the response, and found that the stimulus has the same effect, even if it hasn’t happened yet.

If the experiment has been done correctly, then it confirms what I have long believed. No, not that the structure of space-time is fundamentally different to what we are told. Rather, that the normal scientific techniques used to measure effects and evaluate their significance are no bloody good.

Nobody seems to have picked up on that possibility just yet, but I think the idea will gradually get around.

Libertarian Politics

It’s funny: (h/t Isegoria)

The Country Club Republicans put up most of the money and provided meeting places. Important.

The religeous right provided a lot of work. It was they that walked precincts and they that worked phone banks. Very important.

The libertarians talked. The libertarians also complained. They were always too busy talking and complaining to do any work.

… but I don’t think it represents a personal failing on the part of the libertarians this politician attempted to work with. Rather, it exposes the fundamental flaw with libertarian politics. The other groups were important because they had bought into the idea of politics — they had picked their side and were prepared to work to make it win, effectively obtaining what power they could, and trading it with their allies to get help on the few issues they particularly cared about.

For a libertarian, this is fundamentally illegitimate. Libertarians are not comfortable seeking power outside of the specific policy changes they want to make. That makes them, in political terms, useless.

There isn’t a way around this. For a libertarian to accept that he needs to fully engage in the political process, he has to accept that there is more to politics than policy — that who has power is an important thing in its own right. Once you believe that, you are no longer a libertarian.

Non-violent revolution

In my critcism of the Nobel Peace Prize, I didn’t address the point that Liu Xiaobo is an advocate for non-violent democratic change in China.

That was because it is irrelevant. It is the violence after the government falls that bothers me, not before.

The Tsar of Russia was removed non-violently, by strikes and demonstrations – the more democratic regime that replaced him lasted a few months, a different gang replaced it, their enemies started a civil war… Long story.

The exemplar of the non-violent revolutionary is Gandhi. He succeeds, the British hand over power, there are rival factions and interests sharing it out, a partition results, social unrest – 5 to 10 million dead.

Both those revolutions might nevertheless have been good things; that’s not the point. The point is that either way, the non-violence of the first stage is pretty much insignificant. A non-violent revolutionary is only harmless if he fails.

The Nobel Peace Prize

The Nobel Peace Prize has long been beyond the grasp of rational criticism, but I don’t think this year’s award can be let by with just the usual cynical chuckle.

Timothy Garton-Ash says in the Grauniad CiF that the prize “hits China’s most sensitive nerve”. In fact, the offence that the Chinese government has taken is all the result of a misunderstanding. They really do have difficulty understanding the level of the West’s hypocrisy and stupidity.

To the extent the award means anything at all, it is a declaration of intent, by the Nobel Committee and all those that speak in support of it, to overthrow the government of China and replace it with a Western-style government. Garton-Ash explains that Liu Xiaobo “has consistently advocated nonviolent change in China, always in the direction of more respect for human rights, the rule of law and democracy”. It is possible to advocate respect for human rights and the rule of law within Chinese politics, but to advocate democracy is to advocate the destruction of the Chinese government and its replacement with a Western-style one.

To make such a warlike declaration in the name of peace is, of course, just the usual annual joke.

Therefore, it is reasonable for the Chinese authorities to react to the award as a declaration of outright enmity. Their reaction is, nevertheless, wrong. There are two things they do not understand.

The first is that this declaration is purely ritual. In calling for the overthrow of the PRC, the Western intelligentsia have not the slightest idea of any actual program of action; they are merely showing each other how virtuous they are. It is the equivalent of the prayers for the conversion of England that used to be said by Catholic congregations – a creed that had to be regularly affirmed, without the slightest reflection on its actual meaning.

The second is that, because of the lack of such reflection, the self-declared enemies of China actually have no inkling of what they are actually saying. “Democracy”, in the mouth of someone like Garton-Ash, is just something that goes with human rights and rule of law – it is a minor adornment of a political system, that can be increased here and there without killing millions of people.

In Britain, that is indeed what it is – as democracy crept gradually into the British system over a couple of hundred years, the system absorbed and to a great extent neutralised it, producing a comfortable and moderately stable synthesis. That is not what happens when it is introduced in one go. Then it destroys one regime and produces another, usually very short-lived, replacement. Then there is generally a settling down into some kind of civil war. France is the model, not Britain.

The Garton-Ashes and Nobel Committees do not understand that. The thought never even enters their heads. They probably assume that even the CPC leadership itself really wants democracy, but is just a little too cautious and conservative in bringing it in, and needs to be gently chivied by the likes of Liu Xiaobo.

If the Chinese really understood Western politics, they would ignore it and watch the X-factor like sensible westerners do. But it is out of place for the politicians themselves to criticise the Chinese for taking them at their word.

I don’t say all this to attack the idea of reform in China. While I am no great fan of democracy, and while the Chinese regime does have a fairly decent record over the last couple of decades, I recognise that it is bound to run into serious problems as wealth and economic freedom increase the power of rivals to the present establishment. There are already serious power struggles between central and regional governments. It may well be that political collapse is inevitable, and if so, then a somewhat Western-ish democracy would not be the worst possible outcome. Liu Xiaobo might be the nucleus of a future non-terrible government of China, and the alternative to something worse. It’s hard to say. But these aren’t the terms in which the debate is being carried on.

A Strong Criticism of Monarchy

From the comments to the World Cup piece:

Monarchy is rule by a single individual. It works on this wise. Immediately after his succession, the new monarch enthusiastically attempts to rule the country. For a certain period, shall we call it a year. As there is only so much time between breakfast and supper, this is largely impossible. The next year, he carries on out of a sense of duty. The third year, he announces that he does not want to be bothered with this ruling crap, but if there are any fit women around would you please send them up. Monarchy then gives way to pornocracy: porne is Greek for prostitute.

Previous objections to Monarchic rule which I have rejected rest on the possibility that the monarch might be an idiot or a psychopath. In my estimation, mere idiocy or psychopathy are less damaging to good government than politics is.

The commenter brings up the more fearsome possibility that the monarch could be a normal sane bloke, more concerned about what his girlfriend thinks of him than about whether GDP next year will be 2.5 trillion or 3 trillion.

If decisions simply end up being made by some random attractive woman (or boy) instead of the hereditary ruler, that’s not a problem in itself. But the reason this situation is so much more dangerous than mere insanity is that it produces politics, (meaning a struggle for power), based this time not around arming supporters or controlling journalists, but around forming close personal relationships with the monarch. This was often the main form of “politics” in historical monarchies.

I’m not sure that it is a worse form of politics than exists in a democracy or a military Junta, but my aim in proposing monarchy is to remove politics altogether, which is obviously more difficult than I thought.

Devaluation of Significance

I referred in my last post to a lost writing of mine on the subject of abuse of statistics in economics. I’ve sort of found it – I sent it as an email in response to this blog post by Noel Campbell at Division of Labour. (Read it – it’s short).

He quoted from my response, but I can’t find the actual email I sent him. I do have a draft of it, so it would have been very much like this:

That’s a superb question, and I think the answer will surprise (and disturb) many.

Your paper will include a calculation of significance. This is essentially an estimate of the probability that a correlation as strong as the one you found would exist purely as a result of randomness in the data, even if your theory is false.

This calculation assumes the “proper” sequence of events. You have a theory, and you test the data for a correlation. Since you in fact poked around for correlations, then came up with a theory, the significance calculation is not valid. The true significance depends on the probability that, having found a randomly-caused correlation somewhere, you can then invent a theory to explain it. That probability is very difficult to estimate, but is probably much greater – meaning that the significance of the correlation is much smaller.

It is very counterintuitive that the order of your actions affects the validity of your findings, and indeed it is a close relative of the famous Monty Hall problem – the poster child for counterintuitive probability. When you reveal the correlation that you already knew of, you are revealing no information about the chance of your theory being correct, much as when the quizmaster opens the door that he already knows doesn’t have the car, he reveals no information about the chance that the door you first picked has the car. Conversely if you pick a door and find that it doesn’t have the car, that does change the probability that the first door had it, and if you had no prior knowledge of the data, the correlation does change the probability of your theory being true.

Back to science. As you say, theories aren’t formed in a vacuum, and so there is not such an clear division between the “right” way of doing it and the “wrong” way of doing it. Nobody is completely ignorant of the data when they start to theorize. That is a real problem with nearly all statistics-based results that are published today. They are all presented with significance calculations based on the assumption that the forming of the theory was independent of the data – an assumption that is very unlikely to be completely true. Therefore nearly every significance published is an overestimate.

This was much less of a problem when collecting data and analysing it was difficult and laborious. Now that large data sets fly around the internet, and every researcher has the capability of running analyses at the click of a mouse, it is a problem that has already got out of hand.

I didn’t want to be rude at the time, but I found Campbell’s response shocking. He seemed to fully accept my argument, but wasn’t bothered by the implication that pretty much all published research relying on analysing pre-existing statistics was wrong. Rather, his conclusion was that since everybody else was doing what he was doing, nobody should complain and demand “purity” (his scare quotes). That came to mind particularly reading Bruce Charlton’s discussion of the state of honesty in science.

The story of real science

Bruce Charlton has published what he calls an “mini-e-booklet”: http://thestoryofscience.blogspot.com/

I think he is saying, in greater detail and at much more length, and with the point of view of an insider, what I was saying in the last few days: that science has declined, because science has become an industry which no longer allows for the extraordinary honesty that real science requires.

This is the problem of science today – it has been bloated by decades of exponential growth into a bureaucratically dominated heavy industry soviet factory characterized by vastly inefficient mass production of shoddy goods. And it is trundling along, hour by hour, day by day; masses of people going to work, doing things, saying things, writing things…

Science is hopelessly and utterly un-reformable while it continues to be so big, continues to grow-and-grow, and continues uselessly to churn out ever-more of its sub-standard and unwanted goods.

Switch it off: stop making the defective glasses: now…

There are some very general arguments he makes which I have been meaning to spell out for a while. He suggests that the peak of science was in the mid-20th century, and it was a transitional state.

this transitional state of classic science was an early phase of professional science, which came between what might be called medieval science and modern science (which is not real science at all – but merely a generic bureaucratic organization which happened to have evolved from classic science). But classic science was never a steady state, and never reproduced itself; but was continually evolving by increasing growth, specialization and professionalization/ bureaucratization.

I think such transitional phases occur in different fields quite frequently. Part of my disillusionment with libertarianism is that it is an attempt to recapture a transitional state in government that was never sustainable – the state where a new class is taking over power and opens up freedom for everybody because it has not yet thrown off its self-identification as an underdog that benefits from freedom.

The failure of science is also an aspect of the widely-recognised but ill-understood problem of trying too hard: some things can only be achieved by trying to do something else.

The scientists of the past, like the individuals making up the governments of the past, were privileged. They ruled or researched not in order that they optimise some output, but because they could – they had reached positions of genuine personal responsibility, and had to make their own judgement.

If these “very general arguments” sound rather woolly, do not adjust your set. That’s why I haven’t published on them already – nevertheless, I bring them up now because they’re bugging me and I think Charlton’s writing is relevant to them.

Back to the specifics, Part 3 quotes an earlier post of Charlton’s that chimes very closely with what I was saying yesterday:

Charlton BG. Are you an honest scientist? Truthfulness in science should be an iron law, not a vague aspiration. Medical Hypotheses. 2009; Volume 73: 633-635

Summary

Anyone who has been a scientist for more than a couple of decades will realize that there has been a progressive and pervasive decline in the honesty of scientific communications. Yet real science simply must be an arena where truth is the rule; or else the activity simply stops being science and becomes something else: Zombie science. Although all humans ought to be truthful at all times; science is the one area of social functioning in which truth is the primary value, and truthfulness the core evaluation. Truth-telling and truth-seeking should not, therefore, be regarded as unattainable aspirations for scientists, but as iron laws, continually and universally operative. Yet such is the endemic state of corruption that an insistence on truthfulness in science seems perverse, aggressive, dangerous, or simply utopian.

Indeed.

There are points I disagree with: Charlton tells the orthodox story of Lysenko – he was a gangster, he brought politics and political arguments into science. As I said yesterday, that lacks the understanding that he believed he was not the first to do so, that he believed he was only trying to correct the political influence that had already occurred. We are distracted by the fact that Lysenko’s enemies were not merely removed from influence, but actually imprisoned – that is incidental, just part of the difference between Stalin’s Russia and our world. The dissenting scientist today is as much an enemy of the state as Vavilov was, the only difference is that our establishment is secure enough to leave its enemies at large, while Stalin wasn’t.

The reason I insist on this is that the orthodox story makes the problem seem too easy: don’t allow monsters like Lysenko, keep politicians out of science. It isn’t that easy – the politics that matters is the “office politics” of science itself, not the real politics of the government.

Charlton does not suggest a solution like mine of yesterday – de-emphasising the quest for originality in favour of more checking and reproduction – but it’s clearly a prerequisite for the sort of changes he does advocate. To restore the primacy of truth to science a necessary step would be to ensure that only truth-seekers were recruited to the key scientific positions, and to exclude from leadership those who are untruthful or exhibit insufficient devotion to the pursuit of truth. Obviously, before you can do that you have to have a way to find out who is truthful and who isn’t – you have to check.

Certainly there needs to be a slowing-down of science – Charlton and I are as one on that.

There’s another point that Charlton gets close to: Real achievement in science requires a great deal of luck – the thing you are looking for has to really be there. However, when someone is in a career, it is unjust to value them by whether they are lucky. That is one of main forces that has driven a wedge between the practice of science and any real product – every research project has to produce something publishable (failing incompetence by the scientists), whereas in reality most research of the most valuable kind finds nothing, producing only a few jackpots for the lucky. The only solutions within the structure of science as bureaucracy is to either know what you are going to find in advance (which is useless), or publish results which are in fact devoid of real content, drowning any real results in the noise. This is largely achieved by abuse of statistics – something I thought I’d addressed in relation to economics, but I can’t find. Perhaps I’ll post something later.

 

 

See Also:

Originality and Science

One probably-final point to come out of the Lysenkoism discussion of the previous two posts:

Yesterday I admiringly referred to Richard Feynman’s quote

I’m talking about a specific, extra type of integrity that is not lying, but bending over backwards to show how you’re maybe wrong, [an integrity] that you ought to have when acting as a scientist. And this is our responsibility as scientists, certainly to other scientists

As I said, that is a key part of cutting out the cascade of distrust that can occur when science becomes politically sensitive.

The problem is that that was an easy thing for Feynman to say, because Feynman was a flamboyantly insane genius, and the last thing he ever had to fear was being ignored. The situation is rather different for one graduate student or new PhD among twenty aiming for the same grant-funded research post. In that position, playing down the significance or certainty of one’s own work is a ticket to the dole queue.

And the density of competition is astonishing. There was a piece in Nature a couple of years back, on the limitations of fMRI, that pointed out that from 2007-2008 there have been eight peer-reviewed papers published involving fMRI per day – 19,000 since 1991. “About 43% of papers expore functional localization and/or cognitive anatomy associated with some cognitive task or stimulus”. Thousands upon thousands of papers, each searching for the little piece of originality that will give them importance.

However, this torrent of research demonstrates a solution as well as a problem. I wrote yesterday that “it is … impractical to replicate every experiment, confirm every observation, check every calculation”. Clearly, I was wrong. There is ample manpower in the science industry to double- and triple-check important results, but the system does not value the work highly enough for it to actually get done. Only original work actually merits funding.

That is a widespread problem in non-commercial fields, most obvious in the arts. In commercial arts, most artists make small variations or combinations of existing products, just trying to be a little more attractive and entertainment. The minority who are truly original are highly valued, because they are providing material for the rest to refine or perfect. Indeed, I can think of no other distinction between “high” and “popular” art, but that high art always seeks to be original, and popular art isn’t too bothered. In academic arts, the only valid work is to do something really new. The end result is a product that is always different, but never very good. In science, every new paper is original, but most of them are wrong.

I would assume that in the cases of both art and science, the original assumption was that the market worked well enough to perfect existing work, but that originality required help and subsidy. However, the subsidised sectors at length became isolated from the commercial, to the point that now there is no commercial sector relevant to the academic work being done, and the new stuff is being pumped out into a vacuum.

It seems obvious that it would be beneficial for science to move more slowly and carefully, but the academic system has evolved in a way that does not permit it. It would take a major shakeup to get the science establishment to start to value that caution.

What they want to hear

There was another interesting point in the discussion at Hans von Storch’s that I brought up – an interesting comment by “Toby” on the earlier piece:

Lysenko told the politicians what they wanted to hear – a “short cut” to socialism. Which side of the current “debate” is telling politicians what they want to hear? The ones arguing that money must be spent and sacrifices made? Or the ones advocating that nothing be done?

That is a good question, and is the root of much of the political polarization of climate science.

Toby implies that politicians want to hear that nothing need be done – money need not be spent.

A right-winger – like myself – believes that what politicians want to hear is that their departments and budgets must be enlarged.

As I explained, the distrust of motives is enough by itself – without any actual dishonesty or malpractice – to mess up the scientific process in a field where unequivocal confirmation or rejection of theories is difficult to come by.