I heard an interesting remark on a podcast the other day. The hosts were interviewing a triathlon coach about training camps, week-long retreats where you get together with other athletes and train under the guidance of professional coaches. They are deliberately held in remote locations, says this coach, because part of the benefit is getting away from the normal stresses of your life. He said that without the nine-to-five job to worry about, people are amazed at how much harder they can go.
The reason this happens is that stress takes it's toll on your body. It wears you out, fatigues you, in the same way that running or biking does. I'm in the middle of the semester, which is always that hardest time, and I'm definitely feeling those effects. I'll come home late after a day where my only exercise was my 35 minute volleyball class, but I will feel completely drained. Working out can be a great way to relieve stress, for sure, but sometimes you are already so physically worn out that a hard session of weightlifting seems counter-productive.
My point is to recognize that if you feel especially fatigued but your training volume hasn't gone up, the other factors in your life may be the cause. One of the biggest advantages that professional endurance athletes have is that they don't have another job; training is their sole responsibility. This opens them up to do so much more focused, productive exercise and make bigger sport-specific fitness gains. The rest of us have to do the best with the time that we have and recognize that we can't do everything all of the time.