The men were found in a small room crammed with fermented fish at the CP Foodtech factory in Phu Yen province. The source said investigators were looking into the cause of death.
"We believe they died because they inhaled toxic gas inside the room that was full of fermented fish for fish sauce production," he told AFP, speaking on condition of anonymity.
He did not say what materials are used in the fermentation process at the Thai-owned factory.
One of the victims entered the storage room on a routine check and never returned, prompting his colleagues to search for him, but they were all later found dead, according to Tuoi Tre newspaper.
Industrial accidents are common in rapidly developing Vietnam, where labour laws and safety regulations are often flouted.