Studies Show Most Do Not Understand Statistics

In this election season, repeated citations of polls provide reminders of how little even most educated people understand about statistics. I should like to review a few basic errors that cause most people to overvalue the accuracy of polls and other studies based on statistical samplings and correlations.

Journalistic polls often state a “sampling error” of 3 or 4 percentage points. This sampling error is a measure of the statistical error resulting from taking a sample of several hundred or several thousand random people out of the entire population represented. It does not include other sources of error, such as systemic sampling bias resulting from favoring, say, urban over rural respondents, women over men, etc. Thus the total error of a poll is usually more than the stated sampling error. This is why voter exit polls turn out to be inaccurate more frequently than their sampling error would indicate. If the error were truly 3 percent, we would expect the poll to be accurate within that margin of error two-thirds of the time, following a normal distribution.

Understatement of the error is also common in economics. Recently, former Treasury Secretary Robert Rubin opined that the current financial crisis was a “low probability” event, following conventional economic models. However, as Benoit Mandelbrot has pointed out, conventional economic models of asset valuations substantially underestimate risk, since they assume a normal Gaussian distribution of variations in price when a Cauchy distribution would be more accurate. Higher mathematics aside, we could gather as much when we consider that “low probability” events occur with remarkable regularity and frequency. Rubin’s understatement of error in his economic model leads to a tragic failure to appreciate that there may be systemic reasons for our propensity for bubbles and busts; instead, he regards the crisis as a freak occurrence.

Worse still is when polls are advanced to support claims for which they may have little relevance. Telling us that a majority of economists support Candidate X is not an economic argument for Candidate X, any more than a majority of physicists supporting Candidate X would prove the candidate is good for physics. If anything, it tells us about the political affiliations of economists or physicists, which is sociological data, not a scientific argument. Hard science does not work by taking polls of scientists, but demands that reasons be produced for a position.

Medical studies are often interpreted by journalists to prove causality when they only show statistical correlation. A good rule of thumb is to never assume causality unless a clear aetiology can be shown. Here, common sense may serve as an adequate substitute for mathematical expertise. When the consensus on medical wisdom constantly changes in a matter of decades, we can be sure that the facts were never as firmly established as originally claimed. Medical studies understate their errors by failing to take into account measurement error and systemic error in their statistical analysis. Further, they usually show correlations or “risk factors” without demonstrating causality. For these reasons, the certitude of medical wisdom should be viewed skeptically. Lastly, the claim “there is no evidence that X is dangerous” can simply mean no adequate study of the matter has been done.

In all these cases, a healthy skepticism combined with common sense can guard against most statistical fallacies, even when mathematical sophistication is lacking. Mathematics, after all, is wholly derived from intuitive, rational principles, so it cannot yield absurd results. When a presentation of statistical results seems completely contrary to reality, it is usually a safe inference that there is a wrong assumption underlying the analysis. Even sophisticated statisticians can err, though they calculate impeccably, if they misconstrue the assumptions or conditions of the question they believe they are answering. When studies claiming 90 or 95 percent accuracy prove to be inaccurate more than 10 percent of the time, it doesn’t take a mathematician to realize that there is a lot of overclaiming in the soft sciences.

Update: 29 December 2008

To give a current example of misleading statistics, a new study claims that teens making abstinence pledges are no less likely to have premarital sex than those who do not. If that sounds counterintuitive, it is because it is not true. Pledgers indeed are less likely to fornicate, but the current study decided to control for factors such as conservatism, religion, and attitudes about sex, and compared pledgers and non-pledgers with similar characteristics. Unsurprisingly, this yielded no difference, since the pledge itself does not magically cause abstinence, but rather the underlying attitudes and values are the cause. This is a far cry from showing abstinence programs are ineffective. It would be like saying education is ineffective, but rather it is knowledge that changes behavior. Once again, competent scientists blinded by their biases can make inapt choices of groups to compare, and make interpretations that do not follow.

The New Politics, Same as the Old Politics

Both candidates for the U.S. presidency have portrayed themselves as transcending politics as usual, tacitly admitting the venality of the republic. Nonetheless, the actual conduct of their campaigns has shown that Senators McCain and Obama are creatures of the Washington establishment and Chicago political machine, respectively, and will do little to change their parties’ tactics or policies in anything but the superficial.

Sen. Obama began showing his true colors after the primaries against Hillary Clinton. Posturing as an agent of radical change and appealing to pacifists, he had long advocated for withdrawal from Iraq in 16 months. In his debate with Clinton before the Pennsylvania primary, he answered affirmatively to whether he would give a “rock-hard pledge” to withdraw in that time frame, regardless of what military commanders recommended, “Because the commander-in chief sets the mission.” Once the nomination was sealed, however, Obama backtracked, saying in July he would “refine” his policies based on conditions on the ground as reported by commanders. Hedging his bets, he clarified his clarification, with such nuance that no one can confidently claim he would withdraw sooner than McCain, as both make withdrawal contingent on unpredicable security conditions.

Obama is just another establishment imperialist, who merely disagrees with the strategy, but not the goal of U.S. hegemony. He would withdraw from Iraq only to fortify Afghanistan, and would even invade Pakistan without its consent, effectively embracing the Bush doctrine.

Sen. McCain, it may be argued, began his transformation as early as two years ago, voting in closer alignment with the Bush administration, and passing an opportunity to illegalize waterboarding. He increasingly emphasized his pro-life stance in the primaries, while in the general election he calls himself a maverick, an apode better fitting when given by others. His campaign received a jolt when he chose the unheralded governor of Alaska as his running mate, in what seemed a shameless affirmative action choice designed to garner female votes. This perception has not abated, as Gov. Palin has proven astoundingly inept at addressing basic policy questions intelligently, thoroughly undermining the Republican claim of superior judgment and experience.

Obama, for his part, strengthened his ticket but weakened his credibility as an anti-establishment reformer by selecting the well-entrenched, loquacious politico Joe Biden. Sen. Biden, who not long ago championed the partition of Iraq, brings a dubious sort of foreign policy experience to the table. The selection of Biden makes clear that Obama is just another Democrat, not a post-partisan messiah, as should be evident from his voting record and his proposed domestic policies.

The belief that we have entered a new era of politics is based on platitudinous slogans and a literally skin-deep distinction from previous candidates. If Senator Obama were white, how many people would think he is a different sort of Democrat, on the basis of his policies? If Senator Palin were a man, who in their right mind would consider her (him) as a VP candidate? For all the talk of sexism and racism, it is clear that two of the candidates have benefited from their “underrepresented” status, even if they might pay a penalty in the final election. The important point here is that there has been no sea change in party positions or tactics since the Bush-Kerry campaign of 2004. Some say McCain is not fit to lead a 21st-century America – such a criticism presumes that there is anything really different about 21st-century politics, compared with the late 20th century. We have some new gadgets, better computers and telecommunications; our demons are Muslims instead of Communists; but the same old thought processes, allegiances and tactics persist.

N.B.: As an academic point, McCain was correct in his distinction between “strategy” and “tactics” in the first debate, according to military usage. Strategy involves large-scale force movement and campaign planning, while tactics is combat movement on the small-scale, brigade level. Obama followed a common English usage of these terms, using “strategy” to refer to a grand objective, while “tactics” are the means by which an objective is pursued.

U.S. Imperialism Backfires in the Caucasus

After fifteen years of American imperial encroachment into Eastern Europe and Central Asia, Russia has finally reasserted itself as a power in the region, as its invasion of Georgia takes advantage of the United States’ preoccupation with the Middle East. The U.S., through NATO, has systematically sought to encircle and break Russia as a world power, and the American lust for global hegemony has succeeded only in reviving Russian nationalism, which could threaten the U.S.’ more vital interests in the Middle East.

The Georgian conflict can only be fully understood in a broader geopolitical context, where the United States, through NATO, has attempted to expand its influence all the way up to the borders of Russia. The American public has a short memory, so their leaders can get away with baldfaced lies, as long as they wait several years. For example, President Clinton, following his predecessor, assured the Russians that NATO would not expand into the former Soviet empire, yet in 1998 Poland, Hungary, and the Czech Republic were admitted, followed by Bulgaria, Slovakia, Slovenia, Estonia, Latvia, and Lithuania in 2004, the last three being former Soviet republics. With talk of missile defense systems in Poland and the expansion of NATO into the Ukraine and Georgia, Russia declared its national security was threatened by Western encirclement.

American assurances that NATO expansion is not directed against Russia ring hollow in light of its other false promises. It will be remembered that NATO promised in 1999 to respect Serbian (then Yugoslav) sovereignty over Kosovo, only to completely renege less than a decade later. The Kosovo affair has striking parallels with the Georgia conflict, as it was obvious that the West wanted Kosovo to secede, yet they invaded on the pretext of defending the locals from war crimes, which turn out to have been exaggerated by an order of magnitude. It took extraordinary chutzpah for Western commentators to ridicule Russia’s similar rationalization for invading Kosovo, but fortunately the public has a short memory. Imperialists seek to change facts on the ground through force, and then legitimize these changes after the fact. Imperialist hypocrisy is by no means monopolized by the Russians. The West opposes Russia not on principle, but as rival imperialists. The British Foreign Secretary David Miliband (grandson of a virulently anti-Russian Trotskyite) accuses Russia of having “a nineteenth century approach to politics,” an ironic claim, considering the UK’s participation in the cynical dismemberment of Yugoslavia (in the Bosnian and Kosovo wars), followed by regime changes in Afghanistan and Iraq.

The U.S. clearly tried to play the provocateur in Georgia, as it has been arming and training the Georgian military, which now has rockets and tanks at its disposal to suppress secessionists. A month before the war, over 1,000 U.S. marines were at Vaziani military base, training Georgians in combat maneuvers. Short of a major failure of American intelligence, the U.S. certainly knew of Georgian intentions to invade South Ossetia, but did nothing to prevent it.

Yes, to be clear: Georgia invaded South Ossetia first, not the Russians, contrary to what our beloved “free” press has proclaimed in lockstep. True, there had been shelling of Georgian towns by secessionists, but this had been going on for years, and hardly merited a full invasion. A similar situation has prevailed on Israel’s border with Lebanon. There has always been periodic shelling from militias, so Israel can choose a “retaliatory” invasion any time she pleases. The question, in Georgia’s case, is why President Mikheil Saakasvili chose this particular moment to “retaliate”.

Saakasvili had long promised to restore Georgian rule over the separatist enclaves of Abkhazia and South Ossetia, which have long enjoyed de facto independence and Russian support. South Ossetia declared its autonomy from Georgia in 1990, a year before Boris Yeltsin dissolved the Soviet Union. In the early 1990s, there was a series of ethnic conflicts in which the Georgians claim as many as 300,000 ethnic Georgians were expelled from the disputed territories. Notwithstanding its support of the separatists, Russia has recognized Georgian sovereignty over South Ossetia and Abkhazia through the UN. Of course, the NATO countries had also recognized Serbian sovereignty over Kosovo, so we know how little that means, as imperialists will seek to change conditions on the ground in order to legitmize political change. The Russians were clearly looking for an excuse to declare the independence of these territories, as shown by their long term military mobilization. Undoubtedly, the recognition of Kosovo’s independence and talk of incorporating Georgia into NATO played a role in Russia’s timing.

As for Saakasvili, he was under pressure to secure Georgia’s borders in order to attain NATO membership, so that may have influenced his timetable. The Georgians and the Americans were certainly aware of the Russian military mobilization, yet the invasion proceeded anyway. They seriously expected Russia to stay out of the conflict, tragically underestimating the Russian will. No longer would Russia meekly step aside as the West determined political outcomes. Yeltsin’s failure in Kosovo led to his replacement by strong nationalists who would rebuild the Russian military, which is much more capable than it was in 1999. More importantly, its economic strength is such that it cannot be cowed by threats of isolation, as we shall see shortly. Finally, the Americans overreached.

On 7 August, hours after declaring a unilateral ceasefire, Saakasvili ordered an attack on South Ossetia in response to escalated shelling of Georgian villages by South Ossetians. Georgia invaded South Ossetia on 8 August, killing hundreds, and burning their capital of Tskhinvali to the ground, including houses, the school and the hospital, according to refugees. Civilians were deliberately targeted, giving justification to Russian accusation of war crimes, though not on the scale of thousands killed that the Russians claimed. The Georgian president declared “Georgian military forces completely control all the territory of South Ossetia” except for the northernmost part.

As we know, the Georgians and the Americans miscalculated terribly, as Russia did retaliate, and more importantly, did not back down in the face of international condemnation. The imperialist bluster of the Americans and their allies was exposed as empty rhetoric. The countries in Russia’s “near abroad” know that American aid offers little protection, as Russian troops easily cut of transportation from Georgia’s ports, and came within 40 miles of the capital Tbilisi. The only alternative to such an invasion would be to roll over and submit to U.S. hegemony. Astoundingly, the Americans seriously expected Russia to do the latter, victims once again of their own hubris, as in Iraq.

Defeating Georgia is not a particularly impressive military achievement, yet Russia’s ability to assert itself in defiance of the West signifies an important shift in Eurasian geopolitics. The Americans certainly dare not engage the Russians militarily, and even their threat of economic warfare no longer holds the potency it once did. Stock price falls due to political panics are short-lived, as 9/11 confirmed. At any rate, support for the South Ossetians is sufficiently strong in Russia that they will not be cowed by a mere threat to their pocketbooks. Russia, with its vast natural gas and oil reserves, its space program, and its role as the world’s leading military supplier, is virtually guaranteed an ascendant role in the global economy. Like China, it is increasingly able to demand pragmatic acceptance even from those ideologically opposed to its political regime. The Russians will demand recognition as a regional power, and the West will ultimately oblige, as they sacrificed principle at the Beijing Games, in order to continue to do business. For businesses are ultimately unpatriotic, and in the long run will seek to undo any punitive sanctions.

Whither Russia? She will not join the WTO, but such membership is probably unnecessary, as dozens of bilateral agreements are already in place to facilitate trade, without subjecting Russia to the retaliatory trade wars that have occurred between the U.S. and Europe through WTO arbitration. Nor will Russia have to abandon state agricultural subsidies – she will retain full autonomy over her economy, while WTO nations have sacrificed legislative independence on the altar of neoliberal economics. Excluding Russia from organizations like the WTO will only enhance her status as a world power, as she is unconstrained by the rules that bind others, who nonetheless will continue to do business with Russia out of necessity.

Russia has sent notice to the U.S. that no longer will the Caucasus be its stomping grounds. The U.S. has overreached in its imperialist ambitions and provoked the resurgence of an assertive Russia. American interest in the Caucasus is marginal and largely economic, but antagonizing Russia may compromise cooperation on more important areas of interest such as Iran. In all cases of course, the U.S. is concerned with economics, as business is the only reason for the empire’s existence – we are much more Carthage than Rome.

Don’t think Obama will put an end to imperialism – he’s already on record as saying he would send forces into Pakistan even without permission, and he would not rule out a nuclear first strike against Iran. How he is less of a jingoist than McCain is anyone’s guess. The Democrats are every bit as ruthless imperialists as the Republicans, as their record in Korea, Vietnam, and Yugoslavia shows. Even Carter gave us the Carter Doctrine, the prerogative to pre-emptively attack anyone who attempts to control the Persian Gulf, as if that region’s oil were American property. The U.S. has painted itself into a corner by overextending, yet it will not relinquish its empire until it is torn away piece by piece.