Smaller people weigh less! I'm not really tall, 5'8", but I've got 10 stone of me to haul round races. I'm sure if I was 8 stone, and the same level of fitness, I'd be much quicker (and if I strapped a 2 stone backpack on, I'd be much slower)
I was on the treadmill tonight next to a short chick, I was running way faster than her (and sweating considerably more) - but that is because I'm fitter than her. Sorry if that sounds like I'm bragging, but that's the crack. My warm up speed was still 3k per hour faster than what she did her 10 minute run for, that speed to me was pretty effortless.
BTW, I don't normally look at how fast people are running on the tready next to me, unless I catch them looking at my speed in which case I look at theirs 
Edited: 29/04/2009 at 21:06