The Hsiung Feng III missile hit waters off the islands of Penghu, in the middle of the Taiwan Strait, around 8:40 am (0040 GMT), causing no injuries, the navy said.
The navy launched an investigation to determine what caused the incident.
"We've found that the crew did not follow standard operating procedure," navy command headquarters Chief of Staff Vice Admiral Mei Chia-shu told a news conference that was broadcast live nationwide.
The 500-ton patrol boat, called Chinchiang, was undergoing a drill inspection at Zuoying military base in southern Kaohsiung City when the missile was launched by mistake, Mei said.
The missile travelled for more than two minutes before falling into the waters near Penghu, Mei said.
"It did not cross the middle line of the Taiwan Strait," Mei said.
The Hsiung Feng III reportedly has a range of about 300 kilometres, the state-run Central News Agency reported.
Both Beijing and Taipei respect the virtual middle line in the Taiwan Strait, and the two sides do not cross the line with their respective warplanes or ships. The narrowest part of the strait is 130 kilometres (81 miles) wide.
- DPA