The Southeast Asian nation announced in March it was joining the world's only permanent war crimes tribunal after a reform-minded government came to power last year.
But the move sparked an outcry from opposition politicians and the powerful sultan of southern Johor state, who were furious the country's revered Islamic royals would not be exempted from potential prosecution by the court.
Malaysia's decision not to join the ICC is a severe blow to the tribunal in The Hague, which has been under pressure following high-profile acquittals, while Burundi became the first country to quit the court in 2017.
"We have made a decision that we will not recognise the Statute of Rome," Prime Minister Mahathir Mohamad told a nationally televised press conference, referring to the ICC's founding treaty.
"(Opposition politicians) have been able to create confusion in the minds of the people, that this law negates the rights of the Malays and the rights of the rulers."
Ethnic Malay Muslims are the country's majority group, while the rulers refers to the country's Islamic royal households.
Nine of Malaysia's states are ruled by Islamic royalty and the households take it in turns to provide the national monarch, in a unique system.
While they do not have any formal powers, the Islamic rulers are greatly respected by Malays, and criticising them is strictly forbidden.
The new government has been attacked for not doing enough to protect Malays, who have traditionally enjoyed privileges such as preference for civil service jobs and discounts on buying property.
Malaysia had already signed the Rome Statute but still had to ratify it to formally become a member of the court.
It is the government's latest policy flip-flop. It comes fives months after they did a U-turn on signing a UN treaty against racial discrimination after Malays expressed concerns it could erode their privileged position.