(Updates with detail on search ads, recasts lead)
SEATTLE/SAN FRANCISCO, June 12 (Reuters) - Microsoft's plan to establish a strong footing in online advertising suffered a big blow on Thursday as merger talks with Yahoo finally, formally failed and Yahoo said it would let Google sell search ads on its site.
Separate statements from Microsoft (MSFT.O) and Yahoo (YHOO.O) signaled a real rift between the two after their agonizing on-again, off-again talks, and Yahoo shares fell 10 percent as final hopes of a full or partial acquisition faded.
Microsoft shares rose more than 4 percent as investors showed relief that the company would not be paying too high a price for a deal they considered risky -- even though its biggest rivals on the Web aimed to work together.
Yahoo said it had agreed to let Google (GOOG.O) put search ads -- advertisements placed next to search results -- on its site in what it called an $800 million annual revenue opportunity that would boost cash flow by $250 million to $450 million in the first 12 months.
"Google has made an enormous gain strategically. This move might well have shut Microsoft out of the online space altogether," said Sanford Bernstein analyst Jeffrey Lindsay.
Google and Yahoo, No. 1 and No. 2 in search, will pit ads against each other in auctions for the ad that pays the most.
"Yahoo is being a reseller of Google whenever it makes sense, and that is likely to be a lot of the time, given how much more effective Google Web search ads have proven to be," Global Crown Capital analyst Martin Pyykkonen said.
The process is nonexclusive, meaning others could join in the bidding to place ads, a factor that could make a deal easier to pass regulatory approval. The companies agreed to wait 3-1/2 months for regulatory approval and to offer a way to end it if Yahoo is taken over.
But the prospect of combining the top search ad vendors in one system immediately raised fears. Sen. Herb Kohl, a Wisconsin Democrat and chairman of a U.S. Senate antitrust subcommittee, said lawmakers would "closely examine" the plan.
Google Chief Executive Eric Schmidt likened the deal to ones in other industries where rivals find ways to cooperate even as they compete.
"The decision of showing ads is a Yahoo decision, not a Google decision," he said.
Yahoo rejected Microsoft's latest proposal, which sources briefed on the subject said included an offer to buy 16 percent of Yahoo for $35 per share, plus to buy its search business.
Yahoo simply said that an alternative Microsoft proposal to buy only its search business did not fit into its plan to grow search and display advertising.
Microsoft's offer for a minority stake was at a premium per share to its early May offer to buy the entire company for $47.5 billion, or $33 per share.
Microsoft, which said it was still open to an alternative deal, had hoped a Yahoo deal would accelerate its ability to capitalize on Web advertising growth and compete with Google, which is increasingly fighting for the same Internet audience.
Yahoo said on Thursday that Microsoft had made it clear in a meeting on June 8 that it was no longer interested in buying the company outright, even at the $33 per share price Microsoft had most recently proposed.
That may not appease Yahoo shareholders -- including billionaire Carl Icahn -- who have been pressuring Yahoo to reach a deal with Microsoft. Icahn has called for Chief Executive Jerry Yang to be ousted.
Analysts said they did not expect that Yahoo and Microsoft would try another round of negotiations.
"It certainly seems to be the end," said Derek Brown, an analyst at Cantor Fitzgerald. "In their most recent discussions, they were talking about totally separate visions of both a deal and the future."
Microsoft is expected soon to be on the prowl for other acquisition targets because it has not given up its goal for online advertising.
"Microsoft will keep trying," said Morningstar analyst Toan Tran. "Yahoo is one of the most popular sites on the Web, and there is no one else with as much traffic. AOL may be one option, and it may not be as expensive."
Icahn, who has waged a proxy battle to remove Yahoo's board at its Aug. 1 annual meeting, had urged Yahoo to secure a higher price from Microsoft. Icahn has said a partnership with Google should only be a second choice.
Icahn could not be reached for comment.
Yahoo shares sank as low as $22.50 on news of the talks failing and expectations of the Google deal. It was their lowest level since Jan. 31, the day before Microsoft announced its offer for the company.
Yahoo shares fell $2.63 to $23.52 on Nasdaq. Google shares finished up $7.75 at $552.95, and Microsoft closed up $1.12 at $28.24. (Additional reporting by Michele Gershberg in New York and Eric Auchard in San Francisco; Editing by Gary Hill/Will Waterman)