"There's no way they're ever going to be supporting someone who is seen as thwarting this opportunity," Stern told ABC after the meeting. "It is hard to imagine any union supporting a candidate in the Democratic Party for the U.S. Senate who doesn't have strong positions on both healthcare and Employee Free Choice."
Specter supported the measure -- which opponents call "card-check" and denounce as a job-killer -- until this year, when he announced that he would oppose it.
Even if he gets through the Democratic Primary, he may wind up facing a Republican all-star in the general election. Ridge, a former governor who remains popular in the Keystone State, is considering a Senate bid.
The former Homeland Security secretary would have to face Toomey, a former House member, in a tough primary. Still, the more moderate Ridge -- like Specter, he supports abortion rights -- is facing national and state-level pressure to take on a senator now viewed as a turncoat in his old party.
A Quinnipiac University poll released the week showed Specter only narrowly leading Ridge in a hypothetical match-up, 46-43.
"There's a lot of pressure on him right now. He's being beseeched by Republicans -- and not just moderates," said Terry Madonna, a professor of public affairs at Franklin & Marshall College in Pennsylvania. "There is growing alarm in the Republican Party that if Toomey's the candidate, he will lose."
Toomey's campaign announced Tuesday that he'd brought in more than $500,000 in less than three weeks in the race. Toomey or whoever secures the GOP nomination can expect a surge of national dollars; already, the Republican National Committee and other party groups have launched fundraising pitches based on Specter's flip.
The more immediate concern for Specter remains the Democratic primary. The state and national party establishment -- including President Obama, Vice President Joe Biden, Gov. Ed Rendell, D-Pa., and Sen. Robert P Casey Jr., D-Pa. -- have already lined up firmly behind Specter, but that doesn't mean there won't be other challengers.
Sestak, 57, a second-term House member and former Navy admiral, may not be swayed by the array of powerful Democrats urging him not to run, said Larry Ceisler, a veteran Pennsylvania Democratic consultant.
"He's not beholden to the Democratic establishment, and he's not beholden to President Obama," Ceisler said. "I never thought they could clear the field for Arlen Specter."
That's Sestak's attitude as well.
"I honestly believe that leadership matters," he told ABC. "He admits that he failed to shape the Republican Party. But then he left it. We don't do that in the military."
Some are skeptical of organized labor's threats regarding Specter. Labor unions have generally supported Specter in the past, and leaders may not want to pick a fight with Democrats including Obama, Biden, Rendell, and Casey, Madonna said.
Indeed, the SEIU's Stern said that what will matter is Specter's vote on whatever the Employee Free Choice Act looks like in its final form.
"No one's going to get exactly what they want," he said. "So the question is, where is his flexibility, on not letting his idea of perfection stand in the way of progress."
Looming over all of the positioning is Specter himself, a crafty politician who has learned to win under changing political circumstances and demographics.
"Running against Specter," said Ceisler, "is like getting your teeth pulled without the Novocain."
ABC News' Jake Tapper contributed to this report.