*apologies in advance for the long post*
I've not long been running myself, but for what it's worth I wouldn't worry *too* much about clothing, it's really not going to make a huge amount of difference you're just starting out! Spend most of your time choosing the right shoes, and you can build up your kitbag from there!
Shorts:
I often run in normal footballers shorts which normal boxers underneath though I do have a pair of lined sports shorts (comfy and come halfway down my thighs) which i prefer to use. Personally not a fan of lycra, or incredibly 'short shorts'; I don't think i'll ever wear either!
Top:
I wear various T-shirts, most just normal sports branded T-shirts, nothing special! In the cold I put a long sleave top underneath, maybe not as comfortable as some of these hi-tech jobbies you can get, but does the job. I've got one T-shirt that's like a breathable mesh which I prefer to wear as it's most comfortable (but can be bloomin' cold though!)
Socks:
I usually use just normal white socks! recently bought some running specific training socks and they're VERY comfy, i don't think they help me run any faster though! lol. Normal white socks are fine really.
My current preferred running kit costs:
Shorts - £4 (on sale@JJB)
Top - £10 (at a bike show years ago)
Socks - £6 (local running shop)
Shoes - £50 (on sale@JJB)
Perfectly happy and comfortable running in it as well :)