I agree with Double and Andrew that speed loss is permanent over time. I believe I was reading the question a bit differently and, as a result, answering it a bit differently. When you lose some speed due to not focusing your training on it for a bit of time, you can recover your speed. Over the years, though, everyone will lose some speed.
So, there are two potential questions:
Will we lose some speed as we age or if our training focus changes? Yes.
If our training focus changes for a period of months or even a year or two, is the speed loss irrecoverable even if we change our training back to focus on that speed? Generally, I'd say no.
Also, as I clumsily alluded to and Andrew much more concisely brought up, running all-out is likely not even the best way to maintain or rebuild basic speed. More controlled strides and other drills to improve neuromuscular coordination will very possibly be more productive and less demanding.