MILAN - An overcrowded ship carrying up to 600 people trying to flee Libya sank just outside the port of Tripoli, the U.N. refugee agency said Monday, citing witness accounts.

Aid officials were still trying to confirm the fate of those people after the vessel broke apart Friday in the Mediterranean Sea off the coast of Libya, UNHCR spokeswoman Laura Boldrini said.

Witnesses who left the Libyan capital on another boat shortly afterward reported seeing remnants of the sunken ship and the bodies of some passengers floating in the sea, she told The Associated Press.

Other witnesses saw passengers swimming to shore but it was unclear how many survived, according to the International Organization for Migration.

Its staff on the tiny Italian island of Lampedusa interviewed a Somali woman who said she lost her four-month-old baby in the sinking. The woman swam to shore and managed to board another boat heading to Italy, the IOM said in a statement Monday.

A Libyan government spokesman in Tripoli had no comment Monday evening when asked for information about the sunken refugee ship.

At least three other boats that left Libya in late March have disappeared, with hundreds feared dead, Boldrini said.

The number of people fleeing North Africa has soared since mid-January, after Tunisia overthrew its longtime president Ben Ali and set off a series of uprisings in Egypt and Libya. Some 25,000 people, mostly Tunisians, have flooded Lampedusa, which is right off the North African coast.

Since fighting began in Libya in mid-February, the IOM estimates that another 10,000 people have reached Lampedusa or the neighboring island of Linosa from Libya, including almost 2,000 who arrived on five boats last weekend.

Many of those fleeing Libya are foreign workers from sub-Sahara Africa, who in the first weeks of the war were mistaken for mercenaries funded by Libyan leader Moammar Gadhafi and attacked by Libyan rebels.

The witnesses told aid workers that some would-be passengers in Tripoli became scared Friday and changed their minds about boarding the second boat but were forced onto the ship by Libyan soldiers.

"This is the first time that IOM has been told of migrants being forced by Libyan officials to get on a boat," IOM group said. In the past, some have reported being robbed by officials and soldiers of their savings and possessions like mobile phones.

Boldrini said the deaths and disappearances among people trying to cross the Mediterranean to flee unrest and repressive regimes in Africa is increasing as smugglers begin to use bigger and less seaworthy boats.

"Most of the boats, if not all of the boats, are unseaworthy," said IOM spokeswoman Jemini Pandya. "In addition, the boats are way overladen."

Witnesses of Friday's sinking said the Libyans "continued putting people onto the boat, way more than they should have," Pandya said.

In other missing boat cases, Boldrini said the UNHCR told the Italian Coast Guard that two boats carrying 120 and 360 migrants respectively left Libya in late March but never arrived in Italy. Their relatives fear they are dead since they have lost contact with them.

In a third case, relayed by an Eritrean priest in Rome to Boldrini, a boat with more than 70 people on board ran into trouble in the Mediterranean and only a handful survived, making their way back to Libya.

London's Guardian newspaper reported Monday that 61 African migrants died of hunger and thirst in March on that third ship after being ignored by a NATO warship and helicopter.

NATO denied the accusation, saying the Italian ship Garibaldi was 100 nautical miles out to sea while the refugee ship is believed to have floundered much closer to shore.

"Therefore, any claims that a NATO aircraft carrier spotted and then ignored the vessel in distress are wrong," the military alliance said.

Aid officials said it was impossible to know how many people have drowned this year while trying to reach Europe.

"There's been no way of charting for sure how many boats have left, how many people never made it. Some of them we will never know about," said Pandya.