* BTG Pactual closes $5 bln GEMM fund
* Fund returned 28 pct last year
* Co-CIO eyes EU convergence trade, Latin American bonds
By Tommy Wilkes
LONDON, Jan 31 (Reuters) - The hedge fund arm of Brazilian bank BTG Pactual has shut its top-performing fund to new cash, in the latest sign managers believe markets may be too tough for trading huge amounts of money.
BTG's Global Emerging Markets and Macro (GEMM) Fund, based out of London and New York, has told investors it will no longer accept more money into its flagship fund, and has turned away about $500 million since late last year.
"2013 is the year when we need to prove we can drive with a better car (before we take on more assets)," Antoine Estier, GEMM's co-chief investment officer said in an interview, referring to the $5 billion fund's rapid growth in assets since its 2009 launch.
BTG joins several macro funds - which bet on shifts in the global economy - that have closed to new money, or even returned capital to clients in the past two years. Louis Bacon's Moore Capital joined the likes of Brevan Howard when he said in August he would return billions from his flagship fund.
A big cut in bank trading since the financial crisis and lower investor appetite for risk has reduced liquidity, leaving more hedge funds chasing the same trades and making it tougher for those running a lot of money to get in and out of positions.
Some research into returns has indicated smaller, more nimble funds tend to outperform larger rivals.
London-based Steve Jacobs, chief executive of BTG's asset management business, said he may reopen GEMM if performance held up but that it remained shut for an "undefined period."
Unlike many of its rivals, however, GEMM closed after racking up big gains. Last year the fund yielded 28 percent, making it one of the best-performing larger funds in the world.
Estier said he had this month exited one of last year's best-performing trades - betting on a recovery in securities tied to the U.S. mortgage market - but kept some Greek sovereign bond holdings, another big earner for the fund last year.
EUROPE, LATAM OPPORTUNITIES
Formed after the takeover of UBS Pactual by BTG in 2009, BTG Pactual has grown rapidly under Brazilian billionaire Andre Esteves to become Latin America's largest investment bank.
As well as a money management business in Brazil, BTG runs $9 billion in global hedge fund assets, including a U.S. distressed Mortgage Fund and a Global Equity Opportunities Fund, which opened to outside investors last year.
Estier, a former head of emerging markets at UBS, is quick to point out the cultural differences between BTG and its rivals. While some managers regularly take money away from underperformers to give to so-called star traders, BTG has opted for a flatter, more collegial structure.
"We are the opposite to a Brevan or a Moore. You will never hear us say all the money is made by 'these guys'," he said at BTG's London offices, which sit above the well-known luxury car showroom, Jack Barclay, in the upmarket district of Mayfair.
GEMM'S 55-odd investment team - 20 of whom are partners - is grouped into 13 different trading themes, with each trader managing their individual positions and risk.
Estier said 2013 is not looking like as "exciting" as 2012, but he has still spotted some opportunities.
GEMM is betting that the yields on Spanish and Italian bank and sovereign bonds will continue to fall as the macro climate improves and confidence returns.
"They say 100 billion (euros) has gone back into Europe. But 4 trillion (euros) left it. I think these convergence trades are just at the beginning," Estier said.
After making money trading in bond markets judged by most to be only for the brave, including Greece, Ivory Coast and the debt of Kazakh bank BTA, the fund is not backing away from more bold bets.
Estier is flying out to Venezuela and Argentina with an eye on their sovereign debt, although he is steering clear of the latter while a U.S court rules on the stand-off between Buenos Aires and some of its investors.
He is also readying for a windfall from bonds bought last year in Ukraine, a country locked in talks with the IMF about a programme to shore up its economy, now that their inclusion in a key bond index should drive up demand - and their price. (Reporting by Tommy Wilkes; Editing by Helen Massy-Beresford)