By Agence France-Presse
National broadcaster ABC published a leaked photo of the 2007 incident and cited a defence source as saying it was more a "twisted joke" than evidence of genuine neo-Nazism.
Prime Minister Malcolm Turnbull said displaying a Nazi flag was "absolutely wrong".
"Completely and utterly unacceptable," he told reporters. "It was reported (to officials) in 2007, that incident, and the flag obviously was removed and the personnel involved were disciplined," he told reporters.
"But the incident, it was wrong ... it was absolutely wrong, and their commanders took action at the time."
The defence department said in a statement to AFP that "neither the flag nor its use are in line with defence values".
"The flag was briefly raised above an Australian Army vehicle in Afghanistan in 2007. The commander took immediate action to have the offensive flag taken down," it said, adding that it was later destroyed and those involved cautioned.
"Additionally, steps were taken to reinforce education and training for all personnel who witnessed the flag."
Australian forces have been active in Afghanistan in various roles since 2001, when the conflict there began. Around 900 were in the country in 2007, predominately in Uruzgan and Kandahar provinces.
The photograph has emerged as the actions of Australia's special forces in Afghanistan come under intense scrutiny.
Earlier this month, elite troops were accused of committing war crimes in a secret defence report obtained by local media that described a "complete lack of accountability" from top military brass.
The investigation, reported by Fairfax Media, quoted special forces insiders saying some Australian troops engaged in the "unsanctioned and illegal application of violence on operations" with "a disregard for human dignity".
In response, Australia's former spy chief David Irvine was appointed last week to conduct a third inquiry into the nation's special forces in two years.
Some 300 Australian defence personnel remain in Afghanistan.