Two elderly passengers became the first people from aboard a cruise ship moored near Tokyo to die of the coronavirus, the Japanese government said on Thursday, as hundreds more passengers disembarked after two weeks’ quarantine.

The 621 coronavirus cases aboard the Diamond Princess cruise liner are by far the largest cluster of infection outside China. The ship has been held since February 3 with initially 3,700 people on board.

The two patients who died, an 87-year-old man and an 84-year-old woman, had both tested positive for the virus although the woman’s cause of death was listed as pneumonia, the health ministry said.

Two government officials who had worked on the ship were infected, it added, bringing the number of infected officials to five. Public broadcaster NHK reported that 27 people from the ship were in serious condition.

The quarantine operation has sparked criticism of Japan’s authorities just months before Tokyo is due to host the Summer Olympics.

Chief cabinet secretary Yoshihide Suga defended Japan’s efforts.

He told a news conference that after measures were put in place to isolate passengers on February 5, the number of new infections fell.

Japan’s National Institute of Infectious Diseases (NIID) said in a report issued on Wednesday that no new cases of the onset of the COVID-19 disease from the cruise ship were reported on February 16-17 and only one crew member case on February 15.

In a move to reassure the public, the health ministry also issued a statement in both English and Japanese that said all passengers had been required to stay in their cabins since February 5.

Critics have noted that the day before that order, as passengers were being screened, shipboard events continued, including dances and quiz games.

About 1,000 Japanese released from the ship after testing negative for the virus were permitted to go straight home this week. Other countries are flying their citizens home but subjecting them to two more weeks of quarantine on arrival. “We believe the isolation was effective,” Suga, the chief cabinet minister, said.

Those who have shared a room with infected people are being kept on board under further quarantine.

Around 600 people are expected to disembark on Thursday, 500 of whom will return to their homes in Japan, according to the health ministry. On Wednesday, 800 people left the ship including foreigners who left on evacuation flights.

“We are asking people to keep an eye on their temperature at home,” a health ministry official told Reuters. The government handed out pamphlets with advice on the disease.

Some experts, however, worry returnees could infect others. Findings published in the New England Journal of Medicine on Wednesday suggest the virus may be spread more easily than previously thought, including by carriers who have no symptoms.

The health ministry official said the US had taken the decision to risk bringing home infected passengers, and it was up to each country to quarantine people entering their ports as appropriate. “Our stance is that Japan as the local authority has already quarantined these people for two weeks,” the official said.