The Minnesota Democrat said the hearing scapegoats and stereotypes Muslims, and will increase suspicion of the Muslim community, in turn making all Americans less safe.
"Violent extremism is a serious concern to all Americans and is the legitimate business of this committee," said Ellison, who testified at the House Committee on Homeland Security hearing today, but the approach "is contrary to the best of American values and threatens our security."
King's hearing, which he promised will be first in a series on the topic, has touched a nerve.
Democrats say the GOP leader is intentionally isolating Muslims, and is using religion to divide Americans.
Some members of the Muslim community fear that King is on a witch hunt, holding hearings that recall the days of Joseph McCarthy's anti-Communism crusade.
Many are concerned that it could "stoke a climate of distrust and fear in the Muslim community," Rep. Bennie Thompson, D-Miss., said. "I cannot help but wonder how propaganda about this hearing's focus on the American Muslim community will be used by those who seek to inspire a new generation of suicide bombers."
Members of the Muslim community believe they are being unfairly maligned because of the actions of a few. And Muslim leaders have pointed out that a number of terrorism suspects were arrested based on tips from their community.
Some say they are besieged and point to last year's stabbing of a New York Muslim cab driver and the controversy surrounding the scheduled burning of the Koran as recent examples of intolerance.
But King has said the goal of the House Committee on Homeland Security is not to target all Muslims but to deal with the reality of terrorism.
There's "nothing radical or un-American" about this hearing, King said in a staunch defense of his hearing.
"I remain convinced that these hearings must go forward and they will," King said in his opening statement. "To back down would be a craven surrender to political correctness and an abdication of what I believe to be the main responsibility of this committee; to protect America from a terrorist attack."
King said that as the 10-year anniversary of the 9/11 terrorist attacks approaches, Congress "must not let the memories of that tragic day to fade away.
"We must be fully aware that homegrown radicalization is part of al Qaeda's strategy to continue attacking the United States," King added. "Al Qaeda is actively targeting the American Muslim community for recruitment."
Melvin Bledsoe, whose son is charged with killing an Army private at a recruiting station in Arkansas, testified that his young son was converted and brainwashed by Islamic extremists.
"He was with true evil doers," he said. "We must stop these extremists from raping the minds of American citizens."
The hearing, filled with protesters and supporters, was often contentious as Democrats pounced on the panelists and challenged the Republicans' assertions.
"Muslims are here cooperating. They are doing things that this hearing suggests that they do not do," said Rep. Sheila Jackson Lee, D-Texas. "This thing is playing into Al Qaeda right now, around the world."
Hearing Not First of Its Kind
Today's House hearing on "The Extent of Radicalization in the American Muslim Community and That Community's Response" has created a firestorm of criticism by civil rights groups, but it is not the first or the fifth or even the 10th hearing in Congress to tackle the issue of violent Islamic extremism.
Independent Democrat Sen. Joe Lieberman, who is himself a defense hawk, chaired a series of 14 hearings on "Violent Islamic Extremism" from his perch as chairman of the Senate Homeland Security Committee. So Did Sen. Susan Collins, R-Maine, and Rep. Jane Harmon, D-Calif., who held six similar hearings.
Lieberman's hearings spanned from September 2006 to February 2011.
Connecticut's Lieberman says his hearings were different in that they examined the ideology of Islamic terrorism rather than spotlighting the Muslim community, but he called King's hearings "important."
"The problem has gotten worse, and thus there is more awareness of it. Chairman King's focus also appears to be on the responsibility of the Muslim American community for dealing with the threat of homegrown radicalization, whereas our focus was on the ideology that spawns Islamist terrorism," Lieberman said in a statement.
"But the questions Chairman King is raising are important ones. Our government needs a more comprehensive approach to combating and preventing homegrown radicalization. I have been saying that for years," he added. "Law enforcement, intelligence, and local police departments do an increasingly good job. But it's clear that if we're really going to prevent the radicalization of Muslim Americans, people within Muslim American communities must be alert to signs that somebody is beginning to turn in a radical direction and then work with others in the community and law enforcement to stop that person from carrying out an attack."
What's different this time though, civil rights groups say, is King's rhetoric. They also point to the title and witness list of the hearings, saying they more specifically target the U.S. Muslim community rather than the threat of extremism itself.
"The committee should rely on facts and scientifically rigorous analysis, not biased opinions or unsupported theories positing a discernable 'radicalization' process that are belied by available evidence," said a letter sent to King by 40 organizations opposing the hearing. "A fact-based approach enhanced with scientifically rigorous analysis will likely be more successful at providing a clear picture of the threats we face."
A coalition of religious leaders opposing the hearing said today the rationale for the hearings is also false.
"The rationale for these hearings rests on an assertion that the American Muslim community has failed to support American law enforcement in its efforts against terrorism. This assertion is false," a group of religious leaders said in a statement. "A study by Duke University indicated that 'the largest single source of initial information' in helping prevent terrorist attacks were members of the Muslim community. In fact, the American Muslim community is deeply engaged in preventing terrorism."
ABC News' John Parkinson, Pierre Thomas and Lisa Jones contributed to this report.