"The state police jurisdiction stops at the state border, so that's why we had to leave the state," said Sen. Jon Erpenbach. "It's not like we wanted to do this."
"We feel that by delaying the vote for awhile, the people of the state will have more opportunity to talk about this issue," said Sen. Jim Holperin, another Democrat.
Gov. Walker, who unveiled the budget bill only last Saturday, has been calling upon the Democrats to return and end their "theatrics."
"I think it's time for them to come home and do their job," he said.
Walker, faced with a $3.6 billion deficit, denied he is trying to bust the unions.
"The bottom line is we're broke," the governor said. "We can't negotiate for something we don't have the ability to give on."
But Miller said Walker is offering tax breaks elsewhere -- such as at an earlier legislative session where "he created $140 million in tax breaks for corporations in Wisconsin."
Wisconsin has a $137 million shortfall this year and $3.6 billion over the next two years.
Miller said he is trying to protect people's jobs.
"This governor has proposed in a mere four days to strip away people's rights on the pretext of a budget crisis," Miller said. "It's a crisis entirely of his own making. He's using this as a way to strip away longstanding of Wisconsin history of how we work with employees."
The governor is trying to take away collective bargaining rights, which he says is necessary to avoid furloughs, and has said by doing so he will guarantee that no one will be laid off. He wants public workers to contribute more to their health care and pension plans.
"The workers of Wisconsin have always been willing to work with the governor when they face a crisis, whether it's a national disaster or a fiscal crisis," Miller said. "They did it at the last session -- they took furloughs and a 3 percent pay cut. They would do it again. The really insulting thing is that the governor never, ever asked them.
"He just introduced this law last week and he expected it to be passed yesterday [Thursday]," he added. "It's unacceptable. It's not the way we do things in Wisconsin, it's not the American way.
"We had a much bigger deficit at the end of the Bush recession that hit our state and every state like a thunderclap," Miller said. "It was a $6.6 billion deficit. And we solved that. The employees stepped up to it, they took a furlough, a 3 percent pay cut, which helped."
The historic standoff will continue until at least one Democrat returns to the state Senate.
"I think we're a little kabuki theatre, at this point," said Ken Goldstein, a political science professor University of Wisconsin-Madison. "Things are going to come to and a conclusion in the next couple days."
Once the Democrats come back, they are likely to lose the vote, in spite of all the protestors.
Other governors facing similar budget crises are watching Wisconsin carefully.
More than 40 states are facing a combined projected shortfall of $125 billion for the fiscal year of 2012. The hardest hit are California, facing a $25.5 billion gap, Texas at $13 billion, Illinois at $15 billion, New York at $9 billion and New Jersey at $10.5 billion.
CLICK HERE for more of George Stephanopoulos' interview with Sen. Mark Miller.