One wonderful thing about now is the way that people can choose what's comfortable for them without the extreme social penalizing I knew as a young adult.
I swim in colder places--ocean, a chilly pool, and used to envy the little kids that would come in with warmer swimming gear. One day I decided to emulate them, and now I swim in the ocean in fleece-lined tights, a tissue-weight wool shirt, my swimsuit over these, and a high-tech sport shirt over that. It helps a lot in terms of staying warmer, but it's also more comfortable in terms of not having my body exposed. If it keeps me active and comfortable, then what could be wrong with that? It tripled the time I could stay in the ocean before hypothermia started to set in.
Incidentally, since I started being the `weirdo' with more clothes on in the pool, I noticed others coming in wearing leggings, a shirt, or a short-sleeved/shorts version of a wetsuit.

Life is too short not to do the things that make us happy.