The last-minute whisper campaign started after President Trump announced Friday that he had already chosen his top five candidates and planned to announce his nominee on July 9.
Kavanaugh, a D.C. circuit judge and George W. Bush administration alumnus, and Barrett, a seventh circuit court judge and staunch anti-abortion rights conservative, are considered the two front-runners at this point in the selection process, according to sources close to the process.
"The pro-lifers aren't thrilled [about Kavanaugh]," said one senior White House official who has been contacted by one of the groups. "It will matter to the president if the pro-lifers object."
Another White House official said March for Life, which holds an annual march opposing abortion rights that Vice President Mike Pence spoke at, has been in contact with the White House regarding their concerns about Kavanaugh.
"They don't believe he has the backbone to stand up for conservative issues," a source close to the selection process said.
There are questions over Kavanaugh's past rulings and arguments, and how they might translate into making the tough decision to overturn Roe v. Wade.
In 2011, Kavanaugh argued that it was "legally premature" to consider the constitutionality of the Affordable Healthcare Act. And in 2015, he ruled that bulk collection of metadata is consistent with the Fourth Amendment, an issue libertarians like Sen. Rand Paul of Kentucky, a key vote in the process, have been staunchly against.
Another key vote in the Senate, Republican Susan Collins of Maine, told ABC News that she would not support a candidate that would overturn the historic ruling on abortion.
"I would not support a nominee who would demonstrate hostility to Roe v. Wade," Collins said.
So far none of the anti-abortion rights groups have publicly come out against Kavanaugh.
March for Life also did not comment on its lobbying effort, but their President of Government Affairs Tom McClusky said in a statement, "Our impression is that everyone on the short list believes that the Supreme Court should enforce rights that are based on the text and history of the Constitution, and leave the rest to elected legislators, rather than imposing their own policy preferences under the guise of constitutional enforcement."
Sources say that Kavanaugh is the top choice for White House Counsel Don McGahn, who is leading the selection process and is also a Bush administration alumnus. Another White House official cautioned that McGahn has not been pushing his preference to the president.
Other sources have cited Kavanaugh's tenure in the Bush White House as a factor that could hurt his chances, since Trump has made his distaste for Bush known.
At the same time, some point to Kavanaugh's argument in the Minnesota Law Review that a president shouldn't be investigated or deal with lawsuits while they're in office as something that might work in his favor. It's a position that could be helpful if Special Counsel Robert Mueller's investigation ends up in the Supreme Court.
The White House declined to comment.