In an interview Tuesday with the Miami-based Spanish-language channel WJAN, President Obama said that Venezuelan President Hugo Chavez "has not had a serious national security impact" on the United States despite documented ties to Iran.

"We're always concerned about Iran engaging in destabilizing activity around the globe. But overall my sense is that what Mr. Chávez has done over the last several years has not had a serious national security impact on us," Obama said.

"We have to vigilant. My main concern when it comes to Venezuela is having the Venezuelan people have a voice in their affairs, and that you end up ultimately having fair and free elections, which we don't always see."

The comment has drawn fire from conservatives, including presumptive GOP presidential nominee Mitt Romney, who have been eager to portray Obama as negligent in addressing the Iranian threat.

"I was stunned by his comments," Romney told Fox News' Neil Cavuto this afternoon. "This is Hugo Chavez who has invited Iran in, who has invited Hezbollah. Hezbollah, being a surrogate and a proxy for Iran that would have access to weapons that could be used against us.

Get more pure politics at ABC News.com/Politics and a lighter take on the news at OTUSNews.com

"The idea that this nation, this president, doesn't pose a national security threat is simply naive and an extraordinary admission on the part of this president to be completely out of touch with what is happening in Latin America," Romney said. "This is a very misguided and misdirected thought on the part of our president."

The Republican National Committee and Florida Republican Sen. Marco Rubio have also been fanning the flames of controversy over Obama's remark, circulating to the press his comment on Chavez, along with a rebuke.

Rubio, a potential Romney vice presidential pick, accused Obama of "living under a rock" when it comes to the threat from Chavez.

"Even Obama's own State Department belatedly but rightly expelled Chavez's consul general in Miami for her ties to a plan to wage cyber-attacks on the U.S.," he said in a statement.

White House spokesman Jay Carney today deflected questions on the Chavez remark to the State Department.

But Obama campaign spokesman Ben LaBolt defended the president's handling of relations with Venezuela, saying Chavez has been "increasingly marginalized and his influence has waned."

"People like Hugo Chavez want attention - and that's exactly what Mitt Romney and his supporters gave him today. Governor Romney is only playing into the hands of Chavez by acting like he's ten feet tall," LaBolt said.

"President Obama has refused to be distracted by the outdated rhetoric of people like Hugo Chavez and instead has focused on restoring our nation's standing in Latin America, strengthening our partnerships in the region, and standing up for democratic values in Venezuela," he said.