They face extradition to the US following a three-year FBI investigation
The charges reportedly include wire fraud, racketeering and money laundering.
More than a dozen policemen arrived at a five-star hotel in the Swiss capital, while the leaders were gathering for their annual meeting.
One officer reportedly carried bags containing medication and money.
Journalist Michael S. Schmidt tweeted: "FIFA execs not being led out in handcuffs. Very peaceful. Hotel staff meanwhile is freaking out."
The organisation has been plagued by allegations of bribery for decades.
A law enforcement official was quoted as saying: "We're struck by just how long this went on for and how it touched nearly every part of what FIFA did.
"It just seemed to permeate every element of the federation and was just their way of doing business.
"It seems like this corruption was institutionalised."
FIFA is holding a presidential election on Friday, when Sepp Blatter has been widely tipped to stay in the job.
Blatter is not understood to be one of the officials charged.
The US attorney general Loretta Lynch is due to hold a news conference later today.