By Agence France-Presse
The shooter was himself a doctor and former employee of the hospital, according to Mayor Bill de Blasio and police chief James O'Neill, who ruled out a connection to terrorism.
He entered the building in a white coat with an automatic weapon concealed under his shirt, officials said.
The incident occurred at the 1000-bed Bronx-Lebanon hospital just before 3 pm (1900 GMT) as many patients were receiving consultations.
Police quickly cordoned off the area and encircled the hospital, placing officers on rooftops as they conducted a floor-by-floor search for the assailant.
The shooter, whose identity has not been disclosed, was eventually found dead on the 17th floor of the building after apparently killing himself, according to O'Neill.
The dead doctor, a woman whose identity has also not yet been disclosed, was found nearby.
Speaking at a press briefing, De Blasio said: "Thank God this is not an act of terrorism. It is an isolated incident."
Several witnesses spoke of the panic that spread during the attack.
Garry Trimbie told AFP he got a phone call from his fiancee, a hospital employee, at around 3:15 pm.
"She was crying, she said somebody started shooting and then the employees ran and barricaded themselves in the room," he said.
A pregnant woman who was undergoing a consultation on the 14th floor praised the way staff responded.
"I did as I was told and I'm safe and sound. They told us to go inside a room and hide. We were barricaded inside a room. The hospital staff was on top of everything. They were in control," Tamara told AFP, giving her first name only.
"I learned a long time ago: when you're pregnant, never panic," she added.
Nurse Patricia Phipps said the shooting did not come as a shock given the borough it took place in, The Bronx, which is known for its high crime-rate.
She said: "The neighborhood is sketchy. So you're not surprised sometimes these things happen when you work at hospitals like this. Things happen. That's just living in New York. I just said to myself: it's my turn."
The latest incident could also re-ignite debate over gun control laws in the United States, which sees more mass shootings than any other country.
Mayor De Blasio briefly touched on the topic in his press briefing, calling the attack "something we have seen around the country."
"Every time these things happen, it's another reminder that more needs to be done for those people that have access to these weapons...we can't continue not to do something about it," state legislator Marcos Crespo told news channel New York 1.
The last mass shooting occurred on June 14 near Washington DC when a man opened fire on a baseball team comprised of Republican congressmen, wounding four people including the chamber's third most senior member, Steve Scalise.
The gunman, James Hodgkinson, was shot dead by police.