Gordon said that Vincent Astor may have been threatened by Tony's role in Brooke's life.
"I think Vincent Astor felt very threatened by anyone being close to Brooke," she said. "That he really wanted her only for himself."
Vincent Astor died in 1959, leaving Brooke a $120 million fortune; half in a trust for her personally and half in the Vincent Astor Foundation, which Brooke would run.
"Brooke, in a sense, came to life after her husband had died," ABC News' Barbara Walters said. "He left her all this money for a foundation and, you know, who was more popular than someone who's willing to give money to charity?"
Brooke Astor took control of the foundation in 1961. She focused on giving to New York City-based charities, giving back to the place where the Astor's made their fortune, funding major institutions such as the Metropolitan Museum of Art, the New York City Zoo and the New York Public Library.
"It wasn't just that Brooke gave $10 million to the library or $20 million," friend and film producer John Hart said. "She got Bill Blass to give 10 million. You know, she got Annette de la Renta -- a host of people to follow in her footsteps."
Biographer Gordon said, "The foundation gave her a sense of identity. It gave her a purpose in life. It made her feel like she was doing something more than wearing pretty dresses and going out on the town."
In a 1993 ABC News interview with Walters, Astor highlighted her charity as her life's defining work.
"When you get right down to it, the thing about my life that has been marvelous has been the foundation," she said.
Her work continued the Astors' historic legacy, as a family that left an indelible mark on New York -- from charitable donations, to landmarks like the historic Astoria Hotel, built by John Jacob Astor, which later merged with the Waldorf Hotel, the St. Regis Hotel and more.
Brooke Astor gave to smaller hand-picked charities. Linda Gillies, who was director of the now-defunct Astor Foundation, said Astor was deeply involved.
"She wanted to see the people who ran the project," Gillies said. "And she wanted to see the people who benefited from the project. She didn't care about the trustees or local politicians. She wanted to see the people who did the work. We never give to anything we haven't investigated thoroughly ..."
There was also a personal dimension to her charity work and her character: After meeting a homeless woman in a shelter, Astor was so touched by her story that she gave her one of her own coats.
"She [the homeless woman] was left with these enormous debts and absolutely not one penny, so she was out on the streets," Brooke Astor said once during an interview for a documentary about her. "I felt terribly, terribly sorry for her ... and I thought how lucky I am."
Gillies said, "She empathized with people. That was one of her wonderful qualities. That immediately she understood them and communicated with them and sometimes it ... it was emotional."
But the emotional bond she had with philanthropy seemed often lacking when it came to her own son. And, in 1997, when she decided she was too old to run the Vincent Astor Foundation, she decided to close it rather than hand it off to her son, Tony Marshall.