"Considering the future of the JOC, I think the most appropriate thing is to leave things to a new leader from the next generation so that they can host the Olympics and carve a new era," Tsunekazu Takeda told a meeting of the Japanese Olympic Committee (JOC) in Tokyo.
He said his decision had "nothing to do with the investigation".
The 71-year-old said he would also be stepping down from the International Olympic Committee.
Takeda has come under pressure not to seek an additional term when his current mandate ends in June after it was revealed that French authorities believe they have evidence of corruption in the awarding of the 2020 Games to Tokyo.
Current JOC rules require candidates for the top post to be 70 or younger at the time of their candidacy, but there had been rumours the committee was considering a change to allow Takeda to stay on through 2020.
In recent months however, the investigation into Takeda has become an unwelcome distraction for organisers of the Games, with less than 500 days until the opening ceremony.
Asked why he did not step down immediately, Takeda said: "I really feel sorry for causing this sensation, but I think I'm responsible for fulfilling my chairmanship until my term ends."
- 'I did not do anything wrong' -
Takeda has denied any involvement in corruption and again protested his innocence on Tuesday.
"As I have said, I did not do anything wrong. I will continue to make an effort to prove my innocence," he told journalists.
In January, it was revealed that Takeda was facing a judicial process in France over two payments totalling 2.8 million Singapore dollars (US$2.1 million at today's exchange rate).
He has been placed "mis en examen", a legal step that roughly translates as being charged.
It does not automatically trigger a trial but means that prosecutors believe there is strong or corroborated evidence of wrongdoing.
Takeda has said he was "never involved in any decision-making process" over the payments and that the money was legitimate reimbursement for "a consultancy contract signed through appropriate approval procedures."
The French investigation, launched in 2016, centres around two payments made to Singapore-based firm "Black Tidings", a firm linked to Papa Massata Diack.
Diack is the son of the former head of the International Association of Athletics Federations (IAAF), Lamine Diack, and is also embroiled in a bribery scandal.
He is suspected of having received several million euros in bribes, either for sponsorship contracts or to favour the Rio de Janeiro and Tokyo Olympic bids.
He has been on Interpol's most-wanted list since 2015, but the Senegalese government has refused to extradite him to France despite pleas from IOC chief Thomas Bach.
The payments, labelled "Tokyo 2020 Olympic Game Bid" and paid from a Japanese bank account, were made before and after the IOC voted for Tokyo to host the Games.
- 'No means of knowing' -
The case is being investigated in France because of suspicions the funds may have been laundered there.
According to documents seen by AFP, Takeda told French investigators that Black Tidings had been recommended by the giant Japanese advertising firm Dentsu.
Dentsu is the IAAF's main marketing partner, and Massata Diack previously oversaw the IAAF's marketing strategy.
A former Olympic showjumper, Takeda spearheaded Japan's campaign to win the Summer Games for a second time. His father, Tsuneyoshi, a grandson of Emperor Meiji, was also a former JOC head.
Yasuhiro Yamashita, a JOC board member and an Olympic heavyweight judo champion in Los Angeles in 1984, is among the candidates to replace Takeda, according to local media.
In a statement when news of the indictment broke, Tokyo 2020 said it had "no means of knowing the bid committee's activities", which occurred before the organising committee was set up.
"We believe that the Games were awarded to Tokyo because the city presented the best bid," the statement added.