Following every move were the "God Squad," Jeffs' secret police tasked with keeping an eye on the media.
"They know where we are and they just follow us," Steed said.
In the desert sits a newly-built multi-million dollar home, the product of another one of Jeffs' prison edicts.
"They said if we build it then it would melt the bars or whatever in his jail and he would be released," Steed said.
But Jeffs' influence extends far beyond the daily lives of his followers.
When ABC News visited city hall and the police department, officials declined to answer any questions about Jeffs.
This year, the Department of Justice filed a lawsuit against town officials, accusing them of acting as an arm of the FLDS.
In response, the Mohave County Sheriff's Office has stepped up patrols in Colorado City, but it's proven to be a challenge to police a town that views law enforcement as religious persecution.
"They won't talk to you because of who you are and who you represent," said Sgt. Mike Hoggard of the Mohave County Sheriff's Office. "It's troubling."
It was this tyranny of control that caused Willie Steed to question the values of the FLDS when he made the decision to leave.
"The church can just totally kill a family. In just the matter of three days, two days, an hour. And they can break the spirit of their people," he said. "And as you've seen coming out to this place ... they have nothing left, they have no hope, and they can see no future."
Watch the full story on "20/20: Breaking Polygamy" Friday at 10 p.m. ET