The quake had a magnitude of 7.0, according to the U.S. Geological Survey, and was centered just 10 miles from Port-au-Prince.
The center was also relatively shallow, less than 10 miles below ground, increasing the risk of damage.
Karel Zelenka, a Catholic Relief Services representative in Port-au-Prince, told U.S. colleagues before phone service failed that "there must be thousands of people dead," a spokeswoman for the aid group told The Associated Press.
"He reported that it was just total disaster and chaos, that there were clouds of dust surrounding Port-au-Prince," Sara Fajardo said from the group's offices in Maryland.
State Department Asst. Secretary for Public Affairs P.J. Crowley told reporters that embassy staff "reported structures down. They reported a lot of walls down. They did see a number of bodies in the street and on the sidewalk that had been hit by debris. So clearly, there's going to be serious loss of life in this."
Early reports said a hospital in nearby Petionville had collapsed, and a videographer for The Associated Press said he could hear people screaming for help.
A Reuters reporter, quoting workers for a U.S. charity, Food for the Poor, said there were more houses destroyed than standing in Delmas Road, a major thoroughfare in Port-au-Prince. An employee of Food for the Poor said a five-story building had collapsed.
There were other reports of damage, but they could not be immediately confirmed.
"I couldn't even stand up, that's how bad it was," said Valerie Moliere, a 15-year-old resident of Port-au-Prince, reached by ABC News. "There's a lot of people in the street everywhere. Some are wounded."
"I just heard that right next to my neighborhood there's this pharmacy and this school that broke down and many people died," she said.
"I don't know how powerful it was. But from what I felt, it was very powerful," said Carole Bastin, another resident reached by ABC News. "And it lasted like ... I don't know maybe, it was quite long because I could not walk, I was trying to leave the house, the building, I could not because everything was shaking around me. All the file cabinets were opening and all the things falling apart."
Haiti's ambassador to the U.S., Raymond Joseph, said he spoke with President Rene Preval, who told him that buildings were collapsing around him as he drove his car.
"The president is fine and [the first lady] is fine," he said. "However, the palace building has been damaged."
The ambassador added that Haiti endured four hurricanes in 2008: "For this to come and hit us is like a ouble whammy."
The earthquake was felt by people in Jamaica; Guantanamo Bay, Cuba; and Santo Domingo in the Dominican Republic.
The original earthquake was quickly followed by two strong aftershocks. The USGS said they had initial magnitudes of 5.9 and 5.5, and more were likely.