In February, the IOC opened an investigation into the International Amateur Boxing Association (AIBA) and said concerns remained over possible match-fixing at Rio 2016.
Amateur boxing's under-fire rulers handed over a crucial report on internal reforms to the IOC last month.
"This report shows some progress and a good will but still lacks execution and substance in some areas," IOC president Thomas Bach said.
"Our concerns on governance, financial and sporting integrity are continuing and we think that we now need to see actions," he added after a meeting of the IOC's executive committee in Lausanne.
"Therefore we retain our right to exclude boxing from Tokyo 2020."
Boxing had also faced being axed from next October's Youth Olympic Games in Buenos Aires, but Bach said the sport would remain part of the programme "under one condition".
"We want to see the refereeing system and the appointment of judges. We want to see this approved by international and independent expertise," Bach explained.
The IOC gave AIBA an extension until July 11 to take further action and present additional information ahead of the next executive board meeting, scheduled for July 17-18.
Bach also said in February the IOC was "extremely worried about the governance of AIBA", which has as its interim president Gafur Rakhimov, a controversial Uzbekistan businessman.
But he said Thursday: "This is not a personal matter, as I explained there are serious factual issues which have to be addressed and that AIBA has to take into account. This can not be reduced to a personal matter."