Ken Brown said he saw an emergency flare Sunday night and immediately began heading toward it. He first found a group of people huddled together in a life raft who told him to help the others.
"The guy was clinging to the Leviathan, he didn't say nothing. He was so lifeless. Trying to pull him in was the hardest thing," Brown said.
The Leviathan II sank off Vancouver Island on Sunday, killing five British nationals.
Brown said he found two women sharing a life jacket as they floated and held on to each other. He also found a man clinging to the sunken boat. Brown said the man had a fishing line wrapped around his left, which he cut off once he brought him on board his boat.
The volunteer mariner said he found several others clinging to nearby rocks. He added none of the survivors appeared to know what happened, only saying the boat suddenly sank.
Jamie's Whaling Station, which operated the Leviathan II, said during a news conference Monday official had not yet found a cause for the accident.