Penny was arrested by the U.S. Marshals service in Tennessee on a warrant issued by the state of Texas. He was indicted by a grand jury on a charge of tampering with evidence, a felony, on Sept. 28.
He allegedly removed documents from the Karolyi Ranch, the U.S. gymnastics' training facility in Huntsville, Texas, related to Nassar's activity at the gym.
"The indictment further alleges that the removal of the documents was done for the purpose of impairing the ongoing investigation by destroying or hiding the documents," the U.S. Marshals said in a press release. He allegedly ordered the documents be sent to him at USA Gymnastics headquarters in Indianapolis. Those documents have never been recovered, authorities said.
Penny was taken into custody at a cabin in rural Gatlinburg, Tennessee, by the U.S. Marshals and Smoky Mountains Fugitive Task Force, according to authorities. He is awaiting extradition to Texas.
He faces up to 10 years in prison, if convicted.
The Karolyi Ranch was operated by Bela Karolyi and wife Marta Karolyi, the two people tasked with selecting and developing athletes for the U.S. national team. It served as the national team's training facility from 2001 until earlier this year. Nassar worked at the facility and several of his accusers said they were abused by him at the facility.
Penny resigned as USA Gymnastics president in March 2017 amid allegations of sexual abuse against Nassar. Nassar pleaded guilty to seven counts of child molestation, but more than 130 women and girls, including Olympic gold medalists Simone Biles, Gabby Douglas and Aly Raisman, have accused him of assaulting them. Many of those accusers testified at a hearing in January when he was sentenced to up to 175 years in prison. He pleaded guilty in two other cases — one pertaining to assault and another possession of child pornography — as well.
Rhonda Faehn, Penny's former deputy, testified to the Senate in June that she brought up allegations of misconduct to him and she was told to keep quiet. Penny, who appeared before the same committee, chose to plead the Fifth and not testify.
In a statement, USA Gymnastics said that it had "learned of the charges filed against its former CEO Steve Penny, who resigned in March 2017. We support law enforcement's efforts and have fully cooperated with the investigations by the Texas Rangers, Congress and others, and will continue to do so to help the survivors and our community heal from this tragedy."
USA Gymnastics has been in near-constant turmoil since the Indianapolis Star broke the story in September 2016 of Nassar's abuse of gymnasts under his care, both with the U.S. national team, where he was the team doctor, and at Michigan State University, where he was a faculty member. Nassar had been dismissed from his job with the U.S. national team in 2015, but only said it was done due to "athlete concerns."
After Penny's departure in March 2017, he was replaced by Kerry Perry in November 2017. But Perry's tenure was marked by consistent criticism over the handling of the Nassar scandal as well. She resigned less than a year after being named as Penny's replacement.
Perry was criticized by stars, like Raisman, over her lack of transparency about how the organization planned to protect athletes going forward. She resigned days after she hired Mary Lee Tracy, in part to replace the Karolyis, as the sport's top coach. The hiring of Tracy received condemnation from athletes because she supported Nassar in the immediate wake of accusations against him.
But the neverending parade of presidents didn't end with Perry's ouster.
Just this week, former Congresswoman Mary Bono, who was tasked with replacing Perry as interim president of USA Gymnastics, was forced out for a pair of scandals -- less than a week since her hiring. A day after her hiring, Biles criticized Bono, a Republican, for a photo posted on Twitter in which she was shown blacking out a Nike logo over the company's hiring of Colin Kaepernick as spokesperson. She publicly apologized for the photo.
A few days later, Raisman joined calls for Bono's ouster after discovering that Bono worked for the same law firm that worked with USA Gymnastics during the Nassar scandal and crafted excuses for his absences.
Bono was not directly involved in the law firm's representation of USA Gymnastics, but she stepped down nonetheless on Tuesday.
Penny is one of at least three people arrested in connection with the handling of the Nassar scandal. Debbie Van Horn, a trainer who worked with Nassar, was arrested last month in Walker County, Texas. She has denied charges, according to her lawyer, who spoke to the Houston Chronicle.
William Strampel, the former dean of Michigan State University's osteopathic medical school and boss of Larry Nassar while he was a sports doctor there, was charged March 27 with neglect of duty and criminal sexual conduct. His case will go to trial at a yet-to-be-scheduled date.
ABC News' Ali Rogin contributed to this report.