"I mean, if a girl shouts something like, ‘Oi, love, fancy a shag?’ to me as I walk past, I do sometimes wonder how she’d feel if a builder said that to her. Although, of course, I wouldn’t feel physically threatened, as she might," he told the paper.
But the actor, 32, clarified that he only gets mad at the attention if the person doing the catcalling ignores the person he's with.
"I’ve heard some things in my time, I have to say. I’d best not say what. I don’t mind it -- not unless I’m with my girlfriend and someone is being complimentary to me in order to disrespect her. People who don’t respect other people’s feelings really get my goat," he added.
Cavill has been candid about his past and how fellow children picked on him in school, even calling him "Fat Cavill." Now, he's the object of many fans' affections.
"It does help certainly to have that little bit of stigma in the back of your head that you’re never quite good enough, so you keep on working harder," he said, adding that he even "worked a little harder on this one. I got a little bigger."
"When you see Ben on set and everything and he’s in good shape, it makes you want to go back to the gym that morning and work a little harder," he said.
But that doesn't mean Cavill's above trash-talking the Dark Knight.
"If there were really a fight, I mean come on, let’s just think about that for a second [with a look of who would obviously win]," he said. "I’m not saying that out loud, I’m merely inferring it. "