This is not unusual. I’ve had similar statements too.
A similar effect is that some people will think you’re dressed well when you’re not.
You’re putting out a chemical signal of health that people mistakenly attribute to their visual perception of your physique.
We explored this effect a bit in The mistaken attribution of attraction to visual stimulus .