I've been married for 26 years and we've never tried to restrict each others activities but maybe I'm lucky that we both know when to take a step back and put the other first be it training or whatever.
I cringe when reading comments that husband/wife doesn't allow time to train etc. I'd never be in a relationship where the other half decides when and what I can do.
Too dependent on terrain and so many other variables to give an answer. Pick a route, ride it once note your average speed, then ride it again in the near future with similar conditions after some structured training and compare results etc. Are you faster something's working ? If not ? Were you stopped ? You have to compare apples with apples sorry