Following the unanimous vote, Prime Minister Shinzo Abe voiced "sincere regret" and said the government "apologised wholeheartedly" over the notorious policy.
Some 16,500 mentally disabled people were sterilised without their consent under the law that remained in force until 1996, according to health ministry data.
Each victim will receive 3.2 million yen ($29,000) under the measures passed on Wednesday -- an amount derided by campaigners as "failing to meet the seriousness" of the damage suffered.
The issue hit the headlines last year after a Japanese woman, now in her 60s, sued the government over a sterilisation operation carried out in 1972 after she was diagnosed with a mental disability.
Lawyers and campaigners have long criticised the government and parliament for failing to compensate victims long after the eugenics law was abandoned in 1996.
About 20 victims have so far filed lawsuits across the country seeking compensation of up to 38 million yen.
The first verdict over the issue will be announced on May 28, and plaintiffs' lawyers have vowed to seek compensation they say matches the gravity of the harm suffered.
"It is understandable that lawmakers have been hurrying to enact the law to pay one-off compensation to ageing victims," lawyers said in a statement before the legislation was passed.
But without sufficient compensation, it is not a "true solution to the issue," they charged.
Tokyo has pledged to pay the compensation "swiftly" but the government will likely continue to battle in court against victims claiming more.
Germany and Sweden had similar eugenics laws and governments there have also apologised and paid compensation to the victims.
Under Japan's law, some leprosy patients were also forced into abortions under policies that forbade them from having children.
In 2005, a Japanese court for the first time ordered the state to pay damages to a former leprosy sufferer affected by this law.