By Agence France-Presse
After the 64-year-old oilman's nomination was confirmed by the Senate, he headed to the White House, where Trump formally named the political newcomer Washington's next top diplomat.
"This is a man that is respected all over the world before he even begins," Trump said, thanking Tillerson. "He left a very good job for this, I want to tell you."
Also in the Oval Office as Tillerson was sworn in was Trump's chief strategist Steve Bannon, the nationalist former boss of right-wing news platform Breitbart.
Bannon is emerging as a key foreign policy figure and was reportedly involved in drafting the visa and refugee ban that triggered global protests.
Tillerson thanked the president and vowed to serve him and the American people at all times. He is due to meet State Department staff at his new headquarters on Thursday.
He takes over an agency already rattled by top-level resignations and by Trump's order suspending refugee arrivals and visas for citizens from some Muslim countries.
On Tuesday, around 1,000 officials from US missions across the globe signed and submitted a protest memo, decrying Trump's ban as an affront to American values.
The State Department "dissent channel" has existed since the Vietnam War to allow diplomats to question official policy, but the scale of the protest was unprecedented.
One official, speaking to AFP condition of anonymity, warned a bureaucratic "insurgency" against Trump is under way. Others spoke of colleagues bursting into tears.
Tillerson has not made his views known on Trump's executive order, but the White House's response was uncompromising.
Trump's spokesman Sean Spicer said the travel restrictions would help weed out extremists and keep America safe.
The dissident diplomats, Spicer declared, "should either get with the program or they can go."
His threat outraged many at the State Department, who argue that their in-house experts should have been consulted before the order was signed.
Thomas Countryman, a 35-year career diplomat who retired on Monday as assistant secretary of state, urged Tillerson to protect his new employees from retribution.
"If you don't trust professional public servants to help you on foreign policy then by definition you end up with an amateur foreign policy," he told MSNBC.
Tillerson comes into office after a decade as Exxon's CEO and trails an impressive reputation as a manager of a large international organization.
ExxonMobil's global empire has been compared to a quasi-state with its own policy.
It is the world's largest publicly-traded energy firm -- a $350 billion corporation with revenue last year of $226 billion and more than 70,000 employees.
The US State Department has an annual budget of only $65 billion and employs 13,000 diplomats, 11,000 civil servants and 45,000 local staff at 270 missions worldwide.
At Exxon, Tillerson was the ultimate insider. He joined the firm in 1975 as a young engineer straight out of college and worked his way up the ranks to the top.
At the State Department, by contrast, he will come in as an outsider with no political or diplomatic experience.
ExxonMobil has operations in dozens of countries, under multi-year contracts with a variety of governments, and Tillerson was at the heart of many of the deals.
As the director of Exxon Neftegas, a Bahamas-registered consortium set up to exploit offshore oil and gas in the Russian far east, he built close ties to the Kremlin.
So close was he to President Vladimir Putin that in 2013, Tillerson received Russia's "Order of Friendship."
This has raised eyebrows in Washington, where lawmakers from both sides of the aisle have questioned why, under Tillerson, Exxon lobbied against sanctions on Russia.
Trump has said he too hopes to form a friendship with Russia, and US allies are concerned that he and Tillerson might halt measures taken to punish the Kremlin's intervention in Ukraine.
This led to some combative moments in Tillerson's confirmation hearing, but the Senate voted by 56 votes to 43 to approve him.