Television images showed buildings swaying and burst pipes spewing water after the quake, which struck at the height of rush hour in the city of around two million people.
However, there was no large-scale destruction and no tsunami warning issued after the earthquake, although commuters were stranded and tens of thousands were left without power.
Among the casualties was a nine-year-old girl who died in the city of Takatsuki, north of Osaka city, reportedly trapped by a collapsed wall following the 5.3-magnitude quake.
Public broadcaster NHK said an 80-year-old man had also been killed by a collapsing wall and another man trapped under a bookcase in his home.
In addition to the three deaths, the government's Fire and Disaster Management Agency said at least 91 people were injured.
Prime Minister Shinzo Abe told reporters the government was "working united, with its first priority on saving people's lives".
And government spokesman Yoshihide Suga cautioned "there is a possibility that strong aftershocks will happen".
"Large-scale quakes are likely to happen in the next two to three days," he told reporters.
Japan sits on the so-called Pacific "Ring of Fire" where a large proportion of the world's earthquakes and volcanic eruptions are recorded.
On March 11, 2011, a devastating magnitude 9.0 quake struck under the Pacific Ocean, and the resulting tsunami caused widespread damage and claimed thousands of lives.
It also sent three reactors into meltdown at the Fukushima nuclear plant, causing Japan's worst postwar disaster and the most serious nuclear accident since Chernobyl in 1986.
- 'Strong vertical jolt'-
Local residents described the moment the quake jolted the highly urbanised area at 8:00 am, when platforms would have been heaving with passengers waiting to board their commuter trains.
"The floor moved violently. It was a strong vertical jolt. Nearly all of the dishes fell and shattered on the floor," said Kaori Iwakiri, a 50-year-old nurse in Moriguchi, just north of Osaka city.
"My parents suffered a blackout and they have no water. I plan to take water to them now."
Despite its relatively low magnitude, the quake caused quite a shake, registering a lower six on the Japanese experiential scale of up to seven, meaning it is hard to stay standing.
Eiji Shibuya, 52, said the tremor reminded him of the devastating 1995 Kobe earthquake, which killed nearly 6,500 people.
"I was stunned. I couldn't do anything," he told AFP from Itami, a city in eastern Osaka region.
"I was worried about my son as he had just left for his high school. I was relieved when I confirmed he was safe."
NHK showed footage of firefighters tackling a blaze that ripped through a home north of Osaka.
A number of train services were suspended, including the "shinkansen" bullet train, as multiple smaller aftershocks followed the quake.
The Nuclear Regulation Authority said it had detected no problems at its local atomic power plants, but some companies, including Honda, said they had suspended operations at local plants.
Kansai Electric said on its website that around 170,000 homes in the Osaka region were briefly without power.
"There are fears that the risk of house collapses and landslides has increased in the areas shaken strongly," said Toshiyuki Matsumori, in charge of monitoring quakes at the meteorological agency.//AFP