In one of the most dramatic matches in Champions League history, Son scored twice in the space of three minutes early in the first half to give Tottenham a 2-1 lead in the second leg at the Etihad Stadium.
City hit back to take a 4-2 lead that would have sent them through to the semi-finals.
But Fernando Llorente bundled in Tottenham's decisive late third goal -- awarded after a VAR review.
That was enough for Tottenham, whose 4-3 defeat secured an away goals triumph after they won the first leg 1-0.
Tottenham still needed another VAR assist before they could celebrate a first Champions League semi-final berth as Raheem Sterling's goal in the final seconds was ruled out for offside against Sergio Aguero.
In total there were five goals in the first 21 minutes -- the fastest five goals scored in Champions League history.
It was all too much for South Korea forward Son, whose side will face Dutch club Ajax in the last four.
"I have never seen something like this, It was a very tough game, a very crazy game. But we are very proud of our team-mates. It was madness," Son said.
"Sometimes you are annoyed with VAR but today it is 'thank you and good decision'.
"We fought together for 90 minutes. We showed unbelievable character and fight."
Mauricio Pochettino's men were 12 minutes away from being eliminated in the group stages before a late fightback against PSV Eindhoven.
Now they are in their first European semi-final since the 1984 UEFA Cup.
Tottenham playmaker Christian Eriksen admitted he was relieved after City's late goal was ruled out as he had given the ball away in the build-up.
"I think I must be one of the luckiest guys on the planet today. With the last one they scored, I thought it was over. Thank God the ref gave offside," Eriksen said.
"The whole game was a roller coaster, it was goals, goals, goals, goals and drama everywhere.
"It was a fun game to play but I'm lucky we're on the good side."
Injured Tottenham striker Harry Kane, sidelined by ankle ligament damage, joined in the celebrations from home.
The England captain tweeted a video of himself shouting "let's go, come on" in front of a television showing Tottenham's celebrations.
He accompanied the video with the caption: "Omggg! I love you boys!!!!"
Guardiola has failed to reach the Champions League semi-finals in any of his three seasons as City manager, extending his long wait to win the tournament for the third time -- with his last triumph coming as Barcelona boss in 2011.
Here AFP Sport looks at Guardiola's frustrating exits from Europe's elite club competition:
2016-17: Eliminated by Monaco on away goals in last 16, won first leg at home 5-3, lost second leg away 3-1.
Guardiola's first Champions League campaign with City came to a surprising end against French side Monaco, featuring a young Kylian Mbappe.
Monaco scored three times at the Etihad Stadium in the first leg, but City hit four in the second half to seemingly take control of the tie.
However, in the second leg hapless City couldn't prevent Tiemoue Bakayoko's late goal handing Guardiola his first Champions League last 16 exit as a manager.
2017-18: Eliminated by Liverpool 5-1 on aggregate in quarter-finals, lost first leg away 3-0, lost second leg at home 2-1.
Traumatised by the vicious reception from a huge mob of Liverpool fans who attacked the City team bus with missiles outside Anfield, Guardiola's team crumbled in the first leg.
Goals from Mohamed Salah, Alex Oxlade-Chamberlain and Sadio Mane put Liverpool three up by the 31st minute.
In the return in Manchester, City scored early on through Gabriel Jesus, but never built up enough momentum for a comeback as Salah and Roberto Firmino struck for the Reds after half-time,
2018-19: Eliminated by Tottenham on away goals in quarter-finals, lost first leg away 1-0, won second leg at home 4-3.
Handed a golden opportunity to take the lead in the first leg at the Tottenham Hotspur Stadium when Danny Rose's handball conceded a first half penalty, City blew it as Sergio Aguero's spot-kick was saved by Hugo Lloris.
Even an injury to Tottenham striker Harry Kane couldn't inspire Guardiola's side, who played a cautious game and were beaten by Son Heung-min's late goal.
Roaring into a 4-2 lead after trailing 2-1 in the second leg, City were heading for the semi-finals until disaster struck in the 73rd minute when Fernando Llorente bundled home despite a VAR review of a potential handball by the Tottenham striker.
In an incredible finish, Guardiola was punching the air in delight when Raheem Sterling fired what looked to be a last-gasp winner, only for VAR to rule it out for offside.