AP Photo|alangraysonemails.tumblr.com

Rep. Alan Grayson, D-Fla., is defying criticism from Republicans and fellow Democrats for comparing the Tea Party to the KKK, saying he is "calling them out on their hate" and that it "needs to be done."

Grayson ramped up his attack on the Tea Party after he was rebuked by Democratic Congressional Campaign Committee Chairman Steve Israel as well as from the GOP.

But Grayson was unapologetic.

"I'm calling them out for their hate," Grayson said. "That's not wrong. That needs to be done. It's the only way to end it."

He also went a step further Wednesday, attacking the Tea Party for being engaged in hate speech, prejudice, discrimination, racism and even warned that they could resort to violence.

In an interview, in the congressman's home district, with Orlando ABC affiliate WFTV, Grayson said, "I think the Tea Party should expel those members who engage in hate speech…. So many members of the Tea Party have engaged in hate speech against the president, against the first lady, against numerous members of the Congress and against me. And we could give you examples of that."

The WFTV reporter then pointed out to Grayson, that the imagery in the email could be offensive, but the congressman said, "No, that's the truth, and sometimes the truth hurts."

Grayson cited former President Harry Truman. "Do you remember what President Truman said? I don't give them hell. I just tell the truth and they think it's hell."

Taylor Budowich, executive director of the Tea Party Express, accused Grayson of engaging in "hate speech."

"Congressman Alan Grayson has lost it," Budowich told ABC News. "For years now, he has been widely recognized as a nut, but this most recent hate speech is frightening, to say the least."

He said that Grayson's controversial comments contrast with President Obama's calls for "civility and bipartanship."

"There needs to be an adult in the room that stands up for civility and tells these guys to knock it off. I would hope that would be President Obama. Unfortunately, he has been MIA, while his party's disgusting rhetoric spins out-of-control," Budowich said.

The email, sent out by Grayson's reelection campaign Monday included a graphic image of two Klansmen standing near a giant burning cross. The cross is used to help spell out tea party in burning flames. Clicking on the image takes viewers directly to an ActBlue webpage, where they can donate directly to Alan Grayson for Congress, the congressman's 2014 re-election campaign.

"No, it doesn't go too far because the essence of both unfortunately is an undue openness to racism, to bigotry, to prejudice," Grayson said in regards to the email. "In fact generations ago, the KKK was the home of that sentiment and all too often today we see the Tea Party being the home of that sentiment."

The National Republican Congressional Committee condemned Grayson for his remarks and called on the DCCC to do the same.

"There's no excuse for the hateful words and imagery used by Congressman Grayson," NRCC spokesman Matt Gorman said in a statement. "House Democrats should swiftly and strongly condemn him and return the money he has raised for them. This hate-filled rhetoric has Americans fed up with Washington."

Democratic Congressional Campaign Committee Chairman Steve Israel told ABC News he disagrees with Rep. Alan Grayson's recent comments about the Tea Party.

WFTV also asked the often controversial congressman if the comparison between the Tea Party and KKK was unfair since the KKK would often use violence and intimidation to project their message, and that the Tea Party has not acted violently. But, according to Grayson, the right wing political group is not far from using violent measures.

"Not yet," he said. "You may recall that there were people that brought automatic weapons to president's speeches. What were they trying to convey? They were members of the Tea Party."

"I stand by the unfortunate truth that the Tea Party has been all too willing to open itself and be a home to bigots, and prejudice now, for years. That's the unfortunate truth we all have to live with. I stand by that."