Eric Wilson of the Kentucky 9/12 Project said the group is "considering" a lawsuit, working with the ACLJ and will be "part of the batch" when ACLJ files. It got IRS approval as a tax-exempt organization in March 2013 after first applying in December 2010.
Toby Marie Walker, president of the Waco Tea Party, also is working with the ACLJ. Although she is not completely certain, she expects her organization will be "most likely part of the ACLJ" group that files suit. Her group was approved for tax-exempt status in March 2013 after initially filing in July 2010.
Carole Waddell, the treasurer of the Waco Tea Party, said "we felt it was intimidation and harassment just because it tied up a lot of our time and kept us from doing other activities that we would have normally been doing in an election year last year.
"It was a hassle, it was expensive, it was time-consuming and it was pretty frightening," Waddell said of the application process.
Adrienne King, who heads up the Honolulu Tea Party, said it has agreed to be a plaintiff in the ACLJ case. Its tax-exempt status was approved in March 2012, after it first applied in December 2011.
Margie Drescher, the executive director of the OKC Tea Party in Oklahoma City, said it was part of the group that will file with the ACLJ. Its tax-exempt status was approved in July 2012. The group first applied in October 2010.
Not all of the groups planning on turning to litigation will be part of the ACLJ lawsuit.
Julia Hodges, who heads up the Mississippi Tea Party is "talking about" filing suit, but said the group was "not close to deciding."
Mary Beth Hutchins with Cause of Action is in the same boat, telling ABC News it was "not currently filing any lawsuits, but [its leaders] are considering the possibility of filing in the future."
The Tea Party Patriots, a national tea party group that represents smaller tea party groups all over the country, wrote a letter to the IRS Thursday saying it was "presently preparing lawsuits to be filed against the IRS on behalf of those organizations." It wrote it was advising the IRS of the development in order to make sure relevant documents were not destroyed, saying it wanted "to place a litigation hold on all documents, correspondence and other materials related to the 'targeting,' including all electronic information, emails (whether sent or received on government or personal email accounts or devices), correspondence, etc., all phone records, notes, and papers of the IRS employees who have been involved, at all levels within the Service."
Logan Churchwell, spokesperson for True the Vote, a group that says it was also targeted by the IRS, said the group will file a federal lawsuit on Tuesday, although that date may change.
Catherine Englebrecht, the president of True the Vote, said in an interview that she was told during the IRS application process by an analyst in Cincinnati: "I'm just following directions and the directions are coming from Washington."
Englebrecht said her group's application for tax-exempt status has been pending for three years, but during that time she was personally audited and even visited by agents from the Bureau of the Alcohol, Tobacco and Firearms.
"Initially, we were filing [a lawsuit] because we had not yet received any kind of determination," Englebrecht said. "Now, with the IRS' admission that certain groups were singled out, we want to fully explore every opportunity to make sure that what we suffered through the last three years is duly noted. ... I think it begins to point to a pattern of behavior that is very troubling."
True the Vote added that it was concerned that other Tea Party groups around the country were asked about their relationship with them in questionnaires.
True the Vote came under serious scrutiny during the election for its work monitoring polling places, with critics saying it was trying to suppress Democratic and minority voters.
Cleta Mitchell, an attorney with Foley & Lardner, LLP which is working with many of the groups that claimed unfair scrutiny, including True the Vote, said there are "hoards of lawyers reviewing the case law and statutes as we speak" -- but, "trying to find out the basis for damages" isn't simple and there isn't "always a remedy in the courthouse."
"It's not as simple as it sounds," Mitchell told ABC News.