I just took up track this summer for a change. Season's nearly over but I managed a masters meet which was great fun. There are all levels of abilities as some guys just want to give everything a go. 1m high jump, not a problem - you still score points for your team even in last place.
The senior and junior meets seem to be highly competitive however. This doesn't stop you attending. I was told recently that I could compete with the seniors at 5000 and 10000 because the "youngsters" (who would thrash me over the distances) want to do the 800s and 1500s. What you should do is step along to your local athletics club and see what they send teams to, and which meets they hold themselves. Even the bigger, faster, higher clubs have lower league teams if they can get the numbers.
Anyway, the training is great fun. I hadn't done a 100m in nearly 25 years and the sprint group welcomed me with open arms!