Mary Johnston, 95, who lived in Rhode Island at the time, said, "We weren't prepared. We didn't know 'til people practically blew into the ocean!"
All agreed the media coverage and official warnings were the biggest difference between 1938 and Irene.
Milton Miller said they could tone it down a bit nowadays.
"It's good to prepare, but the way a lot of these weathermen express themselves, you'd think the hurricane was right on your doorstep!" he said, adding, "Well, young reporters want to make a name for themselves."
Despite their traumatic memories of 1938, or perhaps because of them, these survivors did not seem bothered by Irene.
"I don't think this one's going to amount to anything," said Mary Johnston.
"Frankly, I don't think it's going to bother us too much," said Patricia Shuttleworth, noting her location -- Greenport, N.Y. -- is inland.
Miller said: "It don't bother me none. … I just take it with a grain of salt. And at my age, what difference does it make?"
Watch full coverage of Hurricane Irene on "Nightline" tonight at 11:35 p.m. ET.