A very complex subject, you're not alone, even the proffesionals struggle to get their racing weight right.
Racing Weight is the book of the moment on weight / sport. It's a bit heavy going but does give some useful insight to what is going on.
Don't write off protein shakes just yet, they can have a place, particularly either, straight after exercise or before bed, protein aids muscle repair. You will find a lot of "recovery drinks" have an element of protein in them.
Your apitite is a good indicator of calorie need, trust it, and do as you are doing, by eating a variety of High Quality foods.