Castro also acknowledged that Cuba may be in the wrong on this issue.
"We're human beings," he said.
White House Press Secretary Robert Gibbs said the Obama aides were "particularly struck by that."
Obama's position on the embargo has evolved over recent years.
When he was running for Senate in Illinois five years ago, Obama called for the end of the embargo.
"I think it's time for us to end the embargo in Cuba," he said in January 2004 at Southern Illinois University. "The Cuban embargo has failed to provide the source of raising standards of living and it has squeezed the innocent in Cuba."
But last year while campaigning for the presidency in Miami -- home to many Cuban-American exiles who fiercely oppose any relationship with Cuba -- Obama argued that the embargo gives the United States leverage and said he would maintain it if elected.
Now, the Obama administration is sending strong signals that it wants to change its policies toward Cuba but expects cooperation.
As one White House official said today, "This is not a one-way street; this is a very busy two-way thoroughfare. And the steps that can be taken by the U.S. can also be matched or met by steps taken by Cuba."
One potential thorn in Obama's side is a carryover from the Bush years -- Venezuelan president Hugo Chavez, who has threatened to boycott the final summit declaration as a protest of American power and Cuba's exclusion from the summit. Cuba is not invited because it does not have a democratically elected government.
Before the last Summit of the Americas, in Argentina in 2005, Chavez grabbed the spotlight with his anti-American rhetoric and played a role in derailing negotiations at the summit.
Chavez revved up a crowd of 10,000 anti-Bush demonstrators at a rally at a soccer stadium near the summit site, slamming the administration's policies on trade and Cuba, among other issues.
But Latin America experts think it will be hard for Chavez to keep the focus on himself this year.
"President Obama is nothing like the kind of easy target for anti-American rhetoric that President Bush was," Sweig said. "For domestic political reasons, he may feel that he needs to continue this anti-American rhetoric that he has mastered so well. On the other hand, I think he sees Obama as an appealing figure and is trying to figure out exactly how to relate to this man."
Obama is not expected to meet one-on-one with Chavez, but they will participate in summit sessions together.
White House Press Secretary Robert Gibbs said Wednesday that Obama will not avoid Chavez or other leaders who have been critical of the United States, but he did not say if Obama would sit down one-on-one with the Venezuelan president.
"If we didn't sit in the same room with people that were critical of this country, we'd probably be sitting in a room all alone. We certainly wouldn't have gone to Europe," Gibbs said. "But the fact that some people have critical things to say, it hasn't and won't deter the president of the United States from looking for areas of common interests."