In some cases, victims say, the Peace Corps has ignored safety concerns and later tried to blame the women who were raped for bringing on the attacks.
"I have two daughters now and I would never ever let them join the Peace Corps," said Adrianna Ault Nolan of New York, who was raped while serving in Haiti.
She is one of six rape and sexual assault victims who agreed to tell their stories, in hopes the Peace Corps will do a better job of volunteer training and victim counseling. The report will be broadcast Friday night on 20/20.
In the most brutal attack, Jess Smochek, 29, of Pennsylvania was gang raped in Bangladesh in 2004 by a group of young men after she says Peace Corps officials in the country ignored her pleas to re-locate her.
"They all took turns raping me," she told ABC News. "They raped me with their bodies,. They raped me with foreign objects."
Smochek says the group began to stalk her and tried to kiss her and touch her from the very first day she arrived at the city where she was assigned.
"Every day we felt unsafe. And we reported everything, we just kept reporting," she said in an interview with five other former volunteers who also were rape or sexual assault victims.
She says the gang rape took place just hours after a Peace Corps safety official filed a report with the local police but again ignored her pleas for re-assignment.
She says the young men knew she had complained to the police.
"They slammed me against the wall and just started threatening me, they're calling me a filthy American whore," she said. "'We told you to stop going to the police. And now we have to kill you,'" she said.
"I was in so much pain that I just told them, 'Just kill me. Please. Just do it.'" Smochek was left unconscious in a back alley.
She says the Peace Corps immediately began to cover up what happened to her, fearful, she says, of offending officials in Bangladesh.
"When the decision was made that I was to go to Washington, D.C., I was told to tell volunteers that I was having my wisdom teeth out," Smochek says.
Peace Corps deputy director Carrie Hessler-Radelet said she was unaware of the gang rape of Jess Smochek because she was only recently appointed.
She denied the Peace Corps has attempted to cover up or keep quiet the large number of rapes and sexual assaults.
"This is the first I've heard of any report of that nature," she said.
The country director for the Peace Corps in Bangladesh at the time, Silas Kenala, told ABC News that because he no longer is a Peace Corp employee he cannot speak about the case. "All I can tell you is that I did what was required to be done according to Peace Corps procedures," Kenala told ABC News.
The Peace Corps pulled all of its volunteers out of Bangladesh in 2006, citing possible "terrorism" issues.
Between 2000 and 2009, Peace Corps figures show there were 221 rapes or attempted rapes, 147 major sexual attacks and 719 other sexual assaults—defined as unwanted or forced kissing, fondling or groping.
According to the figures, there is a yearly average of 22 rapes. There were 15 in the year for which the figures are most recently available, 2009
Peace Corps officials say the number of rapes has gradually declined over the decade.
But some victims say the Peace Corps has continued to treat them in an insensitive way.
"There isn't a point person or an advocate or someone who is managing the case," said Casey Frazee of Cincinnati, Ohio who was sexually assaulted in South Africa in 2009. She has established a support group and website for other Peace Corps victims, First Response Action.
"No one is really looking at this because there's this over-idealized picture of the JFK Peace Corps," said Frazee.
Maggie Young of New Mexico says she decided to tell the story of her rape in South Africa in 2008 because she feels more should be known about the problem.
"It's nice to know that there are people out there who care about hearing our stories," she said of the ABC News report.
In the case of the gang rape victim, Jess Smochek, she says she was made to feel the attack was her fault because she had been walking alone shortly after 5 p.m.
"I had to list all the things that I had done wrong to cause this to happen to me," she says her counselor told her.
Other women said they were told that by having a drink or two they had invited the attacks.
"I was a risky person, and that I had in some way put myself in that situation," said Adrianna Ault Nolan, who was raped in 1998 but says she still carries the mental scars of the incident.
"I still blame myself," says Christina Holsomback of Alabama, who was raped in the country of Georgia in 2008. "Maybe I should not have had a drink or maybe we shouldn't have gone to dinner."
Victims say Peace Corps counselors are doing great damage. "Just because someone has a drink doesn't give anyone the right to violate them," said Casey Frazee. "And they need to stop telling survivors that."
"When bad things happen, you say to yourself, 'How did I bring this upon myself?' and I think, unfortunately, Peace Corps is hoping you'll think in that direction, too," said Adrianna Ault Nolan.
The Peace Corps says it is not the agency's policy to blame victims of rape and sexual assault.
"There are volunteers who have been assaulted or raped who actually do speak kindly of Peace Corps," said Hessler-Radelet.
CLICK HERE to follow the ABC News Investigative Team and Brian Ross on Facebook and join in on the discussion. Shown video of the victims' interviews, Hessler-Radelet said, "I'm deeply grieved by that. It's terrible."
She said she was surprised to hear the stories because the Peace Corps has a Special Services office designed to help victims and has recently created a task force to re-examine agency policies for dealing with sexual assault victims.
"We have a huge commitment to these women," said Hessler-Radelet. "I wasn't around before but I can control the future."
Many of the women interviewed for the ABC News 20/20 investigation said the Peace Corps needs to provide more than the limited counseling they received after they were attacked, and help to cut through bureaucratic red tape to get further treatment.
"It was too hard to navigate by myself and so I just gave up," said Jill Hoxmeier, now studying public health in New Mexico, who was sexually assaulted in the South American country of Guyana in 2007.
"If that's the case, if they need help cutting through red tape, then Peace Corps should be helping them," said Hessler-Radelet.
The victim of the gang rape, Jess Smochek, says she was given only limited counseling sessions and a one-way ticket home and still suffers from after-effects to this day, more than six years later.
"My life has changed forever," she told ABC News.
Angela M. Hill, Hanna Siegel and Chris James contributed to this report.