The second meeting between the two leaders was supposed to build on their historic first summit in Singapore but they failed to bridge their differences and did not sign a joint statement as initially scheduled.
"Sometimes you have to walk and this was just one of those times," Trump told reporters.
Trump said that the sanctions imposed on Pyongyang over its nuclear programme had been the sticking point.
"Basically they wanted the sanctions lifted in their entirety and we couldn't do that," he said.
But he insisted he was "optimistic that the progress we made" before and at the summit left them "in position to have a really good outcome" in the future.
"I'd much rather do it right than do it fast," he added.
The outcome fell far short of the pre-meeting expectations and hopes, after critics said their initial historic meeting in Singapore was more style over substance.
"This is a major failure," tweeted Joe Cirincione, president of the Ploughshares Fund peace foundation.
It showed the limit of summitry, he added, with "not enough time or staff" to work out a deal.
In the original White House programme, a "Joint Agreement Signing Ceremony" had been scheduled in Hanoi as well as a working lunch for the two leaders.
In the event, both men left the summit venue without signing anything and Trump moved up his news conference by two hours.
Ankit Panda, from the Federation of American Scientists, warned on Twitter that the White House's expectation of further talks "does not have to be a perception shared in North Korea. Kim may have left irate, for all we know. He may have no intention of continuing this."
Trump flew around the world for the meeting and Kim undertook a mammoth two-and-a-half-day trek through China in his olive green train, travelling 4,000 kilometres (2,500 miles).
At first the smiles and bonhomie from Singapore ran on into their second date in Hanoi as Trump touted the "special relationship" between the two, although concrete statements were vague.
The US president frequently dangled the prospect of a brighter economic future for a nuclear-free North Korea, at one point saying there was "AWESOME" potential.
From the outset, he had appeared to downplay expectations of an immediate breakthrough in nuclear talks, saying he was in "no rush" to clinch a rapid deal and was content if a pause in missile testing continued.
But Harry Kazianis, Director of Korean Studies at the Centre for the National Interest, said that no agreement was better than a bad one.
There would be "nothing worse than signing a deal just to get something", he told AFP.
"The challenge is North Korea's nuclear weapons are already a reality," he added. "Getting a deal that does little to nothing to remove that threat would be far worse than a flawed deal."
- 'Rocket man' -
In Singapore the two signed a vague document in which Kim pledged to "work toward complete denuclearisation of the Korean peninsula".
Progress subsequently stalled with the two sides disagreeing on what that means, as the North sought relief from sanctions and Washington pressed for concrete steps towards it giving up its weapons.
As in Singapore, the two men put on a show of bonhomie in Vietnam, appearing to share jokes in front of reporters.
Looking relaxed but appearing to say little, they indulged in a poolside stroll Thursday around the gardens of the luxury Metropole Hotel, a colonial-era building that has played host to stars ranging from Charlie Chaplin to Brad Pitt.
It echoed a garden walkabout in Singapore, where the two men bonded over Trump's hulking car -- the "Beast" -- with the US president allowing the younger man a glimpse inside.
It was a far cry from the height of missile-testing tensions in 2017 when Trump slammed Kim as "rocket man" and the younger man branded the American president a "mentally deranged US dotard".
In apparently unprecedented scenes, Kim answered unscripted questions from foreign reporters, saying that he would welcome the establishment of a US liaison office in Pyongyang, which would be a step on the way to diplomatic normalisation.
Before the summit, there was talk that there could be a political declaration ending the 1950-53 Korean War which finished technically with an armistice rather than a peace treaty.
There were also hopes Kim could pledge to destroy North Korea's decades-old Yongbyon nuclear complex, which has long been at the heart of Pyongyang's atomic development but remains shrouded in secrecy -- and North Korea has promised to mothball it twice before.
A patent distraction from the summit was a scandal back home in Washington with Trump's former personal lawyer and fixer Michael Cohen calling him a "racist" and a "conman" during a congressional hearing.
The president already showed he had the testimony at the back of his mind when he tweeted about it before his first meeting with Kim, saying Cohen -- who has been sentenced to three years in jail -- was "lying in order to reduce his prison time".