By John Kemp
LONDON, March 27 Fundamental changes in supply
and demand account for less than a third of short-term movements
in commodity futures prices, according to an authoritative new
study by researchers at UNCTAD and the Swiss Federal Institute
Reflexive trading, when prices respond to past price changes
rather than new information about fundamentals, now accounts for
60 to 70 percent of price moves in the main futures contracts,
up from less than 40 percent before 2005.
The authors examined price moves in Brent oil, U.S. crude,
soybeans, sugar, corn and wheat futures using an elaborate
procedure designed to separate movements related to the arrival
of new information from those in which one price change begets
"We find an overall increase in the level of short-term
endogeneity since the mid-2000s to October 2012, with a typical
value nowadays around 0.6-0.7, implying that at least 60 to 70
percent of commodity price changes are now due to self-generated
activities rather than novel information", according to Vladimir
Filimonov, David Bicchetti, Nicolas Maystre and Didier Sornette
in a paper published on March 21.
The paper marks a breakthrough in research into the
formation of commodity futures prices. It brings research on
commodity markets into the academic mainstream, integrating
behavioural and fundamental approaches, and follows pioneering
work in other asset classes by George Soros ("The Alchemy of
Finance" 1987), Sornette ("Why Stock Markets Crash" 2003) and
Robert Shiller ("Irrational Exuberance" 2009).
If their paper is correct, commodity futures markets may
actually have become less efficient at discovering prices in
recent years, not more, as a result of high-frequency trading
and other aspects of financialisation.
Sornette is one of the world's foremost authorities on price
formation and market microstructure, so the findings cannot
easily be dismissed by researchers and policymakers who insist
markets are efficient and all changes reflect fundamentals.
RISE OF THE MACHINES
The authors put forward various possible explanations for
the rising reflexivity in commodity futures markets including
the increased prevalence of algorithmic and high-frequency
trading; more behavioural trading and herding; hedging; margin
calls and stop-loss orders.
Since hedging, margin calls and stop-loss orders have been a
constant feature of commodity markets, the most likely
explanation for rising reflexivity lies with the increase in
computer-driven and behavioural trading.
The authors present some startling numbers to illustrate the
rise of algorithmic and high-frequency trades in commodities.
Using the Thomson Reuters Tick History (TRTH) database
, which contains 2 petabytes of tick-by-tick data on
more than 45 million unique instruments across more than 400
exchanges, the authors show the big increase in the number of
trades since 2005 has been accompanied by a sharp fall in the
number of contracts in each transaction.
There are more transactions, but each one is much smaller
than before. The average volume per transaction fell from
between five and 40 contracts in 2005, depending on the
commodity, to less than three per transaction in 2012 for all.
By 2009, the median volume for all commodities was just one
contract per transaction. In other words, more than half of all
transactions consisted of just one lot.
Larger trades are becoming increasingly rare. In 2005, 10
percent of all Brent trades were for at least 13 contracts. By
2011, the top 10 percent of Brent trades were for just two
contracts or more.
The rising flurry of small transactions is mirrored across
markets and is characteristic of algorithmic and high-frequency
To distinguish between price changes that are initiated by
fundamentals versus other changes, the authors employ a Hawkes
self-excited conditional Poisson model, which has become the
gold standard for modelling discontinuous financial data.
It can be likened to a family tree. Some price moves are
initiated by external events such as new data. These primary
price moves can be described as zero-order, mother or immigrant
events. Other price moves are triggered by previous ones. These
can described as first-order, second-order etc, daughter or
"Every zero order (mother) can trigger one or more
first-order events (daughters), each of whom becoming
mother-event in turn can trigger several daughters (second-order
events or granddaughters) and so on over many generations," the
"First-order, second-order and higher-order events form the
cluster of aftershocks of the main event as a result of the
self-excited (endogenous) generating mechanism in the system."
The researchers compared the number of zero-order events
with those that were apparently first-order or higher
descendants to calculate a branching ratio or reflexivity index.
The higher the index, the greater the proportion of descendant
For a reflexivity index of 0.8, which is where Brent peaked
in late 2008 and 2009, there were five times as many price moves
as the number that would exist if each were reacting to
fundamentals alone. Four out of five trades appeared to react to
previous trades rather than be triggered by external fundamental
The more first- and higher-order price movements there are
compared with zero-order initiating events, the longer it takes
the market to find a new equilibrium, and the less efficiently
"It takes more and more time for the system to adjust to new
immigrants due to the larger and larger number of triggered
descendants and the longer and longer sequences of generations.
This means that the convergence process to any true price
becomes longer and longer, in other words less and less
efficient," the authors write.
"Rather than agreeing rapidly on the 'correct' price after
the arrival of some unanticipated news, the traders take longer,
not knowing on what price to settle."
PRICES AND BUBBLES
Strictly speaking, the research is applicable only to price
movements over short intervals, up to 10 minutes or so. "Our
reflexivity indices are not particularly designed to capture
longer-term herding mechanisms, which are responsible for bubble
formation on times scales of months to years," the authors
However, they go on to observe, "we surprisingly find that
the mechanisms working at longer-term scales sometimes seem to
cascade down to the shorter intervals on which we compute our
"The most remarkable result obtained from the calibration of
the branching ratio is its very large increase during the period
when oil prices started to accelerate (2007 and 2008). The fact
that our methodology identifies a growing reflexivity during the
ascent of the price and, even more so, during its collapse, is
particularly interesting in view of other analyses that
documented strong evidence for the existence of a bubble during
Brent's branching index climbed to almost 0.8 in the run-up
to the price peak in July 2008 and rose even further as prices
collapsed in the final months of the year and into 2009.
Branching indexes for Brent and other commodities show other
localised peaks during periods when cross-asset risk-on/risk-off
trading has been especially pronounced - periods when herding
behaviour has seemed to be maximised and commodity-specific
fundamentals have seemed to be relegated to a secondary role.
The high levels of reflexivity identified in the study
"highlights the failures of the efficient markets hypothesis and
provides evidence that price dynamics are partly driven by
positive feedback mechanisms".
In layman's terms, if their econometrics is correct,
Filimonov, Bicchetti, Maystre and Sornette have proved what
traders have always instinctively understood: prices are driven
by a combination of fundamentals and behavioural factors, in
proportions which vary over time.
For all that this seems like common sense, this view has
proved remarkably controversial, with the debate polarised
between market fundamentalists and those who detect the hand of
speculation behind every rise in commodity prices.
Perhaps this study will move the debate forward towards a
more productive synthesis.
The paper on "Quantification of the High Level Endogeneity
and of Structural Regime Shifts in Commodity Markets" can be
downloaded atDespite the
technical mathematics, the authors explain their tests and
results clearly, and the paper is worth reading in full.