Aveo Oncology said its lead experimental drug was unlikely to succeed in a mid-stage colon cancer study that was testing if the drug was superior to an approved treatment.
Data from an interim analysis of the trial was being reviewed to determine the next steps, the company said in a statement.
Aveo shares fell 2 percent in trading before the bell.
The news comes as a big blow to Aveo, which has seen a string of failures associated with the drug, tivozanib.
The U.S. Food and Drug Administration had denied approval for tivozanib's use in renal cell carcinoma in June, citing inconsistent patient survival data.
Earlier in June, Aveo had cut about 62 percent of its workforce to focus on developing tivozanib for the treatment of colon and breast cancers, as it anticipated a rejection for the use of the drug in kidney cancer treatment.
Aveo's partner, Astellas Pharma Inc, was testing the drug in combination with standard chemotherapy against cancer drug bevacizumab with standard chemotherapy, for use in previously untreated patients with advanced colorectal cancer.
Colorectal cancer, also known as colon cancer, rectal cancer or bowel cancer, is the second leading cause of cancer death in men and women combined in the United States, according to the American Cancer Society.
Shares of Cambridge, Massachusetts-based Aveo closed at $1.88 on Thursday on the Nasdaq.
(Reporting by Natalie Grover in Bangalore; Editing by Kirti Pandey)