Silverman said that if the 9/11 Memorial foundation allows all other religious memorials of equal size and prominence to be displayed in the museum, the group would "happily, happily, drop the case."
"It's an all or nothing deal. They can remove the cross, or they can let everybody else in. Either way is legal and we would drop the case," Silverman said.
The man who found the cross, Frank Silecchia, said he came across it on the morning of Sept. 13, 2011, after digging three bodies out from the rubble of the collapsed Twin Towers.
"I was overwhelmed with the image of my faith… It brought me to tears and to my knees," said Silecchia, who was working with the New York City Fire Department that day.
Silecchia believes that the World Trade Center cross is "not just a symbol of faith, but also a symbol of our freedom." He also views the cross as a natural artifact from the Ground Zero site.
"When the dust and smoke diminished, that's what was left. It's not something that we created," Silecchia said.
But Jane Everhart the communications director for the New York chapter of American Atheists who said she was traumatized by the events of 9/11, said that she would not be able to visit the Memorial Museum as long as the cross is included in exhibit.
Everhart was due to start jury duty of Sept. 11, 2001, and was in the World Trade Center area on that morning. "I saw people jumping or falling from the Towers, scores of them... I was horrified," she told ABCNews.com.
"The museum should remember everybody who died or suffered, not just the Christians," she said. "America is a melting pot."