(John Kemp is a Reuters market analyst. The views expressed are his own)
By John Kemp
LONDON, Dec 12 (Reuters) - The inherently cyclical nature of some commodity prices has fascinated economists since the late 19th century but still continues to catch both investors and producers unprepared.
Regular and apparently self-sustaining price cycles were first observed in the price of hogs in the United States and Germany in the final quarter of the 19th and first quarter of the 20th centuries.
Economists in Italy, the Netherlands and the United States separately published theories in 1930 to explain why prices cycled up and down regularly rather than settling at an equilibrium level.
Early theories relied on differences in the responsiveness of supply and demand to a change in prices (price-elasticity) to explain why prices appeared to fluctuate around an equilibrium point.
Cambridge economist Nicholas Kaldor likened the pattern to a spider's web, giving rise to the popular name for cobweb models.
Later refinements by Mordecai Ezekiel, an agricultural economist working for the U.S. government, linked price cycles to delays in adjusting production in response to changes in demand and prices (supply lags) coupled with expectations about future prices based on current and past prices (adaptive expectations).
Ezekiel's paper, entitled simply "The Cobweb Theorem" and published in 1938, remains the clearest and most persuasive explanation of commodity market cycles.
Adaptive expectations models fell out of favour during the rational expectations revolution of the 1970s and 1980s, as they appeared to imply that investors would make systematic forecasting errors.
Economists from the ultra-rationalist University of Chicago subsequently developed a model of commodity cycles consistent with rational forward-looking behaviour.
But the simple adaptive-expectations model, with lags in investment and production, is enough to provide a good description of real world commodity markets.
Backward-looking expectations about prices coupled with the delays in adjusting supply provide a good explanation for the deep and recurrent cycles in the oil industry.
Price expectations do indeed appear to be strongly backward looking in the oil industry. In the first few years of the 21st century, painful memories of the long period of low prices in the 1990s held back plans to expand production even as prices surged.
More recently, the production and investment plans of the major oil companies and U.S. shale drillers appear to have been based on the assumption the period of ultra-high prices experienced since 2011 would be sustained indefinitely.
When prices have been high for some time, it becomes an entrenched assumption that high prices will persist for the foreseeable future, and vice versa. Many oil industry analysts and leaders appear to rationalise from prices to theories, rather than the other way around.
At the same time, changes in investment and production take a long time in the oil industry. It can take a decade or more to train an experienced driller or seismologist, and at least that long to bring many complex offshore oil fields into production.
The availability of oil supplies in the early 2020s depends on investment and hiring decisions being made now, just as the availability of oil supplies in the mid-2000s was heavily influenced by reduced investment and layoffs of skilled personnel a decade earlier in the 1990s.
Owing to the delays in changing expectations and investment, oil supply lagged behind the change in demand and prices during the first phase of the boom (2002-2010), then overshot demand as the cycle started to turn (2011-2014).
Deep price cycles are inherent in capital intensive industries like oil and mining, and exacerbated by the tendency to project fairly recent conditions into the far future.
Yet the desire for some sort of stability remains deeply ingrained among commodity producers and even some consumers.
As a case in point, OPEC's founding statute, from 1960, commits the organisation to "devise ways and means of ensuring the stabilization of prices in international oil markets with a view to eliminating harmful and unnecessary fluctuations."
Senior officials, including Saudi Arabia's oil minister Ali Naimi, sometimes suggest "stable" prices are in the best interest of producers and consumers, as if this were a normal or possible state of affairs.
Even the International Energy Agency, which represents the interests of consumer countries, sometimes seems to subscribe to the idea price stability would be a good idea.
In fact, OPEC has never managed to stabilise prices, and there are good reasons to believe it never will given the characteristics of the oil industry (long investment lead times coupled with extreme uncertainty about future prices).
OPEC's fondness for stable (high) prices is shared by Glencore's chief Ivan Glasenberg. Glasenberg has blamed rival coal and iron ore producers for over-investing during the boom years and creating the conditions for a supply-driven slump.
Glasenberg has repeatedly called for a more "disciplined" approach to investment in future, as if the commodity price cycle was somehow voluntary and driven by ill-considered management decisions. "Capital misallocation, not a lack of demand, remains a key issue for the (mining) sector," Glasenberg told Glencore's investors this week.
"We will continue our disciplined approach to capital allocation based on the supply-demand fundamentals. We don't want to oversupply and cannibalise our own business," he insisted in remarks quoted in the Australian Financial Review ("Glencore's Ivan Glasenberg questions iron ore majors' big bet" Dec 11).
Glasenberg might as well wish for the moon. Deep price cycles are normal in capital-intensive resource industries, and there is nothing he or anyone else can do to eliminate them.
U.S. shale producers, OPEC and the rest of the petroleum industry are currently getting a brutal reminder. Until recently, many seemed convinced oil prices would find a floor close to $100 per barrel.
Continental Resources lifted its hedges at around $85 per barrel on the assumption the market would quickly bounce back. Instead Brent prices have continued to fall to less than $65.
Now most oil industry leaders have been forced to adjust their expectations and investment plans to assume prices will remain lower for longer.
This is almost certainly an over-reaction to recent price moves and will set the stage for the next stage in the cycle when investment and production prove inadequate.
In the meantime, however, new production is still coming onstream as drilling and investment programmes agreed in 2013 or even before are only now coming to completion.
Non-OPEC oil production will continue rising for several months more because of the long delays in the system, even as industry leaders recognise the need to rein it back.
Prices will remain under pressure, even as many analysts, investors and business leaders wonder if they have not already fallen too far.
But commodity markets have always operated that way. Instability and disequilibrium, rather than the opposite, are the norm. (Editing by Michael Urquhart)