I think the point is that the horses couldn't do what they do without a skilled rider on their backs. Have you ever seen a horse jump a fence voluntarily, let alone several fences, taking twists and turns between each one, or start dancing as they do in dressage?
The horse is part of the rider's equipment as much as a bike or a javelin, it just happens to be a living, breathing thing with its own temperament. You have to know how to handle the horse in order to get the best out of it. You could, therefore, argue that the equestrian events actually require a greater degree of skill.
Incidentally, winning horses at the Olympics get given a rosette - you could hardly give them a medal after all