"They all became dear friends, which is a great thing to have happen," she said.
The 92-year-old said Sinatra was one of her greatest friends and "was kind of just the most amazing person in my life." She met the iconic singer when she was stuck in a marriage she thought she would never get out of. Sinatra then called and asked her out to dinner.
"I said 'Yes' and that was it," she said.
She also mentioned dating Errol Flynn and that it "was very brief."
She said she was "star struck" when she was 18, adding that the men were "like gods to me" when she was younger.
As for Hughes, the airplane mogul, "he was the best."
Vanderbilt and Cooper talk about this and more in their new book.
"I hope people will be able to speak in a totally free way with people they love," she said about the new memoir. "Really be able to have a conversation and tell them how you feel."
"The Rainbow Comes and Goes" is out now.