* Bank says 21 pct of investors reject pay policy
* Remuneration is "wildly out of control" - shareholder
* Pay chief says HSBC pays less than rivals, must protect business
* Chairman Flint is underpaid - pay chief Robertson
* CEO says no plans to exit large chunks of investment banking (Writes through with comments, details on pay vote)
By Steve Slater and Matt Scuffham
LONDON, May 23 (Reuters) - More than a fifth of HSBC's shareholders opposed the bank's pay policy on Friday in the latest show of anger that banks have not reined in bonuses enough in the wake of the financial crisis.
About 21 percent of investors who voted opposed HSBC's vote on its pay policy for the next three years, not enough to block its plans but representing significant opposition.
Pay has got "wildly out of control", said John Farmer, a private shareholder, at the bank's annual meeting on Friday. "You are not as a bank delivering an impressive return. Please would you go away and rethink the issue totally."
He rejected the bank's claim that staff would leave if they were not paid as much as at rivals.
"If these people want to walk away, let them, and find someone else who will do the job for them," he said to applause from other investors.
Bankers' pay is a politically sensitive issue due to public anger over bonuses paid to those seen by some as partly to blame for the 2008-09 financial crisis.
HSBC, Europe's biggest bank, defended its pay and changes to structure that mean more pay is now in shares and deferred for five years, and can be clawed back if problems are spotted at a later date.
"We look very carefully outside at what's being paid. We pay way under what the American banks pay ... we have to be careful not to destroy the business from which you get profits and dividends," said Simon Robertson, chairman of the bank's remuneration committee.
The bank said 16 percent of shareholders opposed its remuneration package for executives, in a separate vote. That was less than a 41 percent opposition at rival Standard Chartered and 24 percent at Barclays, but was up from 11 percent at last year's vote and its highest in any year other than 2011, when there was 19 percent opposition.
HSBC attempted to take the sting out of pay criticism last week by saying it would cap any bonus paid to its chairman Douglas Flint at 1 million pounds ($1.7 million), after criticism from investors about a plan to pay him a bonus of up to 2.25 million pounds. The bank said any payment made to Flint would be a "one-off" to reflect his work on regulatory reforms.
"I actually think Douglas (Flint) is underpaid for what he does, and he does an extremely good job," Robertson said.
Flint declined to comment on his own pay, but said after the meeting some investors were concerned about pay levels and others did not like the structure, and said the bank will discuss grievances with them in more detail.
"The board understands it absolutely. How can you not understand it," he said when asked if banks did not understand public anger at pay levels.
HSBC's proposal to pay staff bonuses of up to 200 percent of salaries passed easily, with 98 percent approval. Shareholder assent for this was needed to meet new EU bonus rules.
HSBC Chief Executive Stuart Gulliver said earlier that the company had no plans to follow rivals by dramatically reshaping its investment bank.
Barclays this month said it will shrink its business as some areas struggle to make profits due to an increase in regulations and a fall in revenues, while Deutsche Bank asked shareholders for billions of euros to grow its investment bank in the United States.
HSBC's Global Banking and Markets (GBM) investment bank arm had increased its revenue in recent years and taken market share from rivals that were shrinking, Gulliver said.
"We have a very different FICC (fixed income, currencies and commodities) model. I don't see any need to exit large chunks of the business. We will continue to invest in the organic growth of GBM," he said.
Gulliver also pledged to increase dividend payouts despite uncertainties over future capital requirements for banks demanded by regulators and said the bank was making good progress on a turnaround plan.
He has axed more than 40,000 jobs and sold or closed 68 businesses over the past three years to cut costs, but has not yet reached its cost-efficiency and profitability targets.
$1 = 0.5938 British Pounds Editing by Sophie Walker and Pravin Char