As any first year psychology student could tell me, I'm much more likely to value being a generalist, because of the fact that I am one myself. I've always been interested in a little bit of everything, and I love finding connections between things that don't seem related at all. I remember getting my first job when I was twelve, and since then I've always had at least one, plus a fairly wide array of hobbies in whatever field has intrigued me at that moment.
I didn't decide to become a game designer until I was 20 years old, but all the time I spent on other jobs and hobbies wasn't wasted. When solving a game design problem, I often find myself reflecting on lessons that I learned long before It had even occured to me that I might want to design games professionally. The best way to become better at designing games is to design games, but all other experience is as valuable as you allow it to be.
Looking for connections to game design is a little game I play when I find myself stuck with a task that I don't find interesting. Try to look deeper for lessons that are related to your passion. This tactic has made many college courses and dead end jobs much more interesting and applicable than they would ever have been otherwise.