Andrew, like you, I've noticed some dramatic changes in the past 5 years, give or take. I know more changes are coming down the pike but I now avoid buffets most of the time and my portions, especially in the evenings, have changed significantly. Even with these changes, I'm slightly heavier than I was 5-10 years ago. I'm sure some of this has to do with my training volume being reduced drastically but I'm also sure some of it has to do with natural changes due to aging.
As with you, I tend to notice weight changes by how my pants fit, though I usually know when the changes in fit are coming.