The turtle, known as "Cu Rua," or "Great Grandfather," had lived in Hanoi's Hoan Kiem Lake since the 19th century, and news of its death on Tuesday inspired expressions of grief and the burning of incense.
Cu Rua was around 120 years old, said Ha Dinh Duc, a biology professor who treated the giant turtle in recent years, although some believed Cu Rua was more than 700 years old.
And while the turtle was called "grandfather," Cu Rua was actually female.
Only three other members of the same species are thought to remain in the world, one in another Vietnamese lake and two in China.
The species was believed to have been brought to the lake centuries ago.
Cu Rua was popularly regarded as a link to a mythic "genius" turtle that gave ancient emperor Le Loi a magical sword that enabled him to vanquish Vietnam's enemies.
In recent decades, Vietnamese considered any sighting of Cu Rua an omen of good luck.
But when the turtle started surfacing frequently five years ago, Duc and others recognised that Cu Rua was ailing amid the growing pollution and the invasion of small, non-native turtles that further damaged its habitat.
The public outcry prompted official efforts to clean the lake and nurse Cu Rua back to health.
Cu Rua's body, fished from the lake after being discovered on Tuesday evening, will be preserved at the Vietnam National Museum, news reports said.