Living without that connection can be a challenge in a world fixated on sex.
"What I mind is when the idea gets enforced that people need sex. That without sex, you're somehow broken. And of course, we can be happy without sex," Jay said.
But Joy Davidson, a certified sex therapist, believes Jay and his fellow asexuals may be shortchanging themselves with the asexual label.
"Sex is a fabulous, enormously pleasurable aspect of life. And your saying you don't miss it is like someone in a sense who's color blind saying, 'I don't miss color.' Of course, you don't miss what you've never had," Davidson said.
Davidson cited a litany of factors that may be at the root of an asexual life.
"There may be something, maybe something physiological, endocrine, maybe something that has to do with trauma, or abuse, or repression, or severe religiosity, that has predisposed you to shutting down the possibility of being sexually engaged," she said.
But the asexuals Chang spoke with for "20/20" said they're perfectly happy as they are. They said they're used to people questioning their identity. They've even questioned it themselves. For years Victoria Glancy, who wasn't attracted to men, assumed she was a lesbian.
"I have slept with a couple of women, and it was just sort of, 'Oh, well, OK, we've done that now, so we don't have to ever do it again, right,'" she said.
Glancy ultimately decided that she's asexual -- a label Davidson has called problematic.
"You might as well label yourself not curious, unadventurous, narrow-minded, blind to possibilities. That's what happens when you label yourself as sexually neutered," Davidson said.
Jay said his group is not trying to paint anyone into a corner. "The thing about the asexual community is that we're not a place people come to to stop exploring themselves. We don't want to slap a label on people and then have that confine them," he said.
It's true that even the most vocal asexuals can have a change of heart. To their astonishment, Victoria Glancy and her fiancé, Karl Hodgetts, recently discovered that their passions under the covers had been ignited.
Glancy said she and Hodgetts had been getting very close to having sex recently. And Hodgetts is open to that possibility. "I just feel completely comfortable with Victoria. I don't think I've ever felt so comfortable with someone. And so I'm willing to, you know, try things," he said.
So are Glancy and Hodgetts just ammunition for critics who say asexuals simply haven't found the right partners for themselves?
That doesn't matter to Glancy, because, she said, there's a possibility she won't want to have sex — or have it more than once.
"I could have sex with Karl one time and go, 'Oh, OK, saw what that was like,' and then not like it," she said.
At first, Glancy said, she felt a bit uncomfortable acknowledging her sexual attraction to her fiancé.
"When we were first kind of fooling around, and I'm like, 'Oh, I'm not supposed to like this.' But then that's stupid. I mean, you're just are who you are," she told Chang.
Although they hadn't yet had sex when Chang spoke with them, Glancy and Hodgetts removed themselves from the asexuality Web site. For the remaining members, giving voice to asexuality is still a proud mission.
Chang asked Jay: "If I had a pill that I could give you to make you sexual, would you want it?"
"I would not take a pill to become a sexual person. I'm having way too much fun as an asexual person," he said.