Kim boarded his olive green armoured train at the Dong Dang border station in Vietnam before it rolled northward toward China en route to Pyongyang, kicking off a marathon 4,000-kilometre (2,500-mile) journey expected to take two and a half days.
Earlier Kim made a highly unusual stop at the stark concrete monument where the body of Vietnam's independence hero is on display.
On historic North Korean anniversaries Kim regularly pays tribute -- with the "humblest reverence", according to the official KCNA news agency -- to his predecessors, his father Kim Jong Il and grandfather Kim Il Sung, at the sprawling memorial palace on the outskirts of Pyongyang where their preserved remains lie in state.
But he is not known to have previously done anything similar for a foreign leader.
The North Korean adjusted the ribbons on a large wreath emblazoned with his name and the message "Cherishing the memory of President Ho Chi Minh" before bowing his head for no less than 48 seconds.
Kim has visited China, the North's key diplomatic protector and main provider of trade and aid, four times but there have been no reports in either country's state media of him going to Mao Zedong's mausoleum in Beijing.
But Kim's grandfather was a close friend of Ho Chi Minh and supplied him with fighter pilots and psychological warfare specialists during his war against the US-backed regime in South Vietnam.
Pictures of the two together are displayed on the noticeboard outside the Vietnamese embassy in Pyongyang.
- Single-string guitar -
Kim's trip to Vietnam was the first by a North Korean leader since 1964, when Kim Il Sung also travelled by rail for his journey to the southeast Asian nation.
Kim boarded his own train Saturday and headed towards China, though his route was not known and it was not clear whether he would be stopping to meet President Xi Jinping along the journey.
A smiling Kim waved to crowds at the station and clasped his hands in the air as he was seen off by white-uniformed soldiers and Vietnamese officials.
"I really admire Chairman Kim Jong Un and I see him as a friendly person," said local resident Nguyen Thuy Chi at the station.
On Friday Kim met Vietnam's top leaders and reportedly tried his hand at playing a dan bau -- a traditional single-string guitar.
Pictures in the Vietnamese press showed the grinning leader with the instrument as a crowd of Vietnamese guests -- including President Nguyen Phu Trong -- cheered him on.
Kim and Trump's much-hyped second summit finished abruptly with a signing ceremony scrapped after the pair failed to reach an agreement on walking back North Korea's nuclear programme in exchange for sanctions relief.
Both sides said they were open to further talks, though a third summit has not been scheduled.