The fighter jet crashed some 250 kilometers (156 miles) east-southeast of Okinawa's capital city Naha at around 11:45 am (0245 GMT), said Osamu Kosakai, spokesman for the ministry's Okinawa defense bureau.
Its two crew members ejected and were rescued by a US military helicopter, he told AFP, adding the accident was "not life-threatening".
The fighter jet crashed "due to engine trouble," he said.
Japan's coastguard "dispatched an aircraft to the waters to see if there is any debris or floating oil", a coastguard spokesman told AFP.
Okinawa accounts for less than one percent of Japan's total land area, but hosts more than half of the approximately 47,000 American military personnel stationed in Japan.
For decades, Okinawa residents have asked for some of the bases to be moved, with resentment growing after a string of accidents and crimes committed by US military personnel and base workers.