My first thought is that with less body fat, you have less insulation and therefore feel colder.