Currently, nearly 50% of Japanese adults are not sexually active. This has been reported in international news such as the Washington Post, the Telegraph, the Guardian, the Daily Mail, and Japan's own English-language news source, the Japan Times.
Anime characters are a reflection of the real situation. Not all of the adults who are not currently sexually active are virgins (for example, some married couples stop having sex after birthing the number of children they want), but many are.
Japan is a dont-ask,-don't-tell society: what you want to do in the privacy of your own bedroom (watch hentai, use body pillow dolls, any sort of fetishes, etc.) is not considered any big deal, and mentioning that you felt up a stranger's breast in a hostess club can be done in front of your colleagues without anyone raising an eyebrow, so it is not a "prudish" society at all. However, the Japanese overwhelmingly want you to keep your sexuality and sexual activities to yourself: do whatever you like and that will generally be respected, but PDA (public display of affection) or coming out and sharing about it to others is not socially respected.