LONDON (Reuters) - A near 10 percent drop in international oil prices since last month has taught a painful lesson to many hedge funds and other speculators in energy futures: you can ignore physical markets for only so long.
Hedge funds piled into what seemed a one-way bet in early June, grabbing up oil contracts as the rapid advance of Islamist insurgents in northern Iraq threatened supplies from OPEC's second-largest producer at the same time that peak summer demand was approaching.
The influx of buying helped drive Brent crude oil to a nine-month peak above $115 a barrel, but signs the spike would be short-lived were hiding in plain sight, according to interviews with senior oil traders this week.
While oil future prices rallied, with hedge funds accumulating a record close to 600 million barrels of oil on paper, real physical cargoes of crude were struggling as refiners held back from buying in the face of surprisingly unseasonable weak demand.
"Once the refinery runs didn't materialise and there were some unplaced North Sea cargoes, the market got scared," a senior trading executive at a major bank said.
"Many people stopped out violently, and volumes exploded. This was the sharpest open interest decrease we have had in years - only confirming the magnitude of the pain on the move down."
Between June 19 and early this week, Brent prices tumbled from $115.71 to $104.39 a barrel, racking up searing losses for many investors. To experienced market watchers the warning signs had been clear.
Just prior to the Iraq-led rally in futures, the prices for physical cargoes in the North Sea slipped to multi-year lows, with refiners declining to buy as they wrestled with weak margins for gasoline and diesel in Europe.
North Sea crudes underpin the Brent benchmark used to set prices of more than two-thirds of global oil deals.
Another signal came from Urals, Russia's main export crude, which was experiencing a bout of counter-seasonal weakness, with refiners actually losing money on each barrel they processed for much of June.
The weakness in the North Sea and Russia combined with weak demand for oil from major West African producers Nigeria and Angola, which have lost market share in North America due to the shale oil boom. Many cargoes were left with nowhere to go.
"Some analysts were said the crude market was going to be screamingly tight this summer, but what they were missing is (that) the increase in runs was not happening in Europe but in Russia, Saudi Arabia, and the United States, places with cheap feedstock," said Seth Kleinman, head of energy research at Citi, who previously worked for Glencore.
"This tightened crude a bit but swamped the market with gasoline and diesel, crushing margins and slashing demand from European refineries."
Hedge funds and many other traders were also betting that prices would remain stronger this summer than later in the year, a market structure known in the industry as backwardation.
But heavy selling in the past two weeks, which accelerated after Libya said a nearly year-long port blockade was coming to an end, drove prices for delivery next month to a sharp discount to later contracts, a structure known as contango.
That change in market structure in particular may have caught out many players, traders said.
On Tuesday, volumes in Brent leapt to a record high. Contracts equal to almost 1.5 billion barrels of oil - more than two weeks of global demand - changed hands in just one session as traders adjusted so-called spread positions.
"The physical traders largely called the move in spreads correctly, while the funds were long flat price because of Iraq while also betting the backwardation would continue because of North Sea summer maintenance," one long-time physical trader said.
"When the move back to contango came, it was fast and violent."
To illustrate the extent of the consensus in the market before the latest rout, 22 out of 24 analysts who forecast quarterly averages for the Brent price in latest monthly poll by Reuters expected higher prices in the third quarter than in the last three months of this year.
Only one, Portugal's Banco BPI, said prices would be slightly lower in the third quarter, while ABN Amro said they would be flat near $100 a barrel.
Five forecasters, including banks Credit Suisse and Natixis, saw third-quarter prices averaging more than $4 a barrel above those for the last three months of 2014.
By Friday Brent prices had recovered to just below $108 a barrel as the downing of a Malaysian passenger plane over Ukraine intensified the stand-off between Russia and Western powers. Physical cargoes have also strengthened.
Additional reporting by Simon Falush in London; editing by Jane Baird