There could be a lot of factors: your body size, muscle to fat ratio and body comp, total exercise in a day, the actual caloric amount of the food you are eatin

There could be a lot of factors: your body size, muscle to fat ratio and body comp, total exercise in a day, the actual caloric amount of the food you are eating (you can eat a lot of low calorie food compared to a single high calorie item for instance), mitochondrial function in your muscles and your ability to clear lactate from your bloodstream. Controlling for all of the variables is actually very difficult. Most people that eat a lot and don't seem to gain weight are simply not eating that much calorically or exercising a lot. When a more sedentary lifestyle kicks in they generally begin to align more with the general population.