The Rev. C.L. Bryant, a former president of NAACP's Garland, Texas, chapter who is now a leading Tea Party activist said the idea that the Tea Party is racist or is trying to instigate a racist climate is "simply a lie."
"I have seen posters ... where every president from Reagan to Obama has been called a fascist," Bryant, who serves as a contributor to FreedomWorks, which organizes Tea Party groups, told ABC News. "Why is it that just because we have a black president, we are hyper-sensitive to posters at rallies?"
The NAACP wants to "create a climate where they can say that those on the right are in fact racist and those on the left are their saviors," he added. "This is very much what the liberal agenda is about."
Dale Robertson, a Tea Party activist who runs TeaParty.org and has himself been at the center of a race-related controversy, said the NAACP is merely pandering to the Democratic party.
"I find that the NAACP should be standing against the new Black Panther and their stance and yet instead of doing the right thing, they're doing the wrong thing by attacking people who feel government should be held accountable," Robertson said.
The Tea Party movement's popularity has been boosted by the likes of Paul and former Alaska Gov. Sarah Palin, but it has yet to find a unified voice or widespread support among Americans.
Twenty-seven percent of Americans support the Tea Party, according to a May ABC News/Washington Post poll, but nearly as many Americans oppose the movement as those who support it.
Among registered voters, 15 percent said they would be more likely to support a candidate for Congress who's associated with the Tea Party movement, but 24 percent say they'd be more apt to oppose such a candidate.
The poll also found that 57 percent of people who opposed the Tea Party suspected its members of racial prejudice specifically against Obama. Only 10 percent of Tea Party supporters expressed such a sentiment.
The NAACP, on the other hand, is trying to reenergize momentum within its own ranks. The organization received a big boost from the first lady, who was the keynote speaker at this year's convention.
The first lady's speech focused on childhood obesity and her "Let's Move" initiative designed to promote healthy living and eating for children.
NAACP leaders have individually taken on the Tea Party in the past, but the organization is now trying to build a bigger momentum against the Tea Party, which has emerged as a strong grassroots, albeit fragmented, force across the country.
"We have to close the enthusiasm gap," NAACP president Ben Jealous said in an interview with the Associated Press Friday. "The danger of the Tea Party is that people see them and think about periods in history when groups like them were much more powerful than they are now, and so a lot of what we spend energy doing is explaining to people what reality is, and that the reality is that the majority from 2008 still exists."
ABC News' Gary Langer contributed to this report.