It's nice to chip away at programming but if you don't have an academic basis to guide your studies it is going to keep you out of a lot of jobs when you get out. Take it from someone who knows - I've programmed useful things in just about every language, but because I didn't major in Computer Science (Physics/Philosophy instead) I'm unable to compete for the top tier of jobs. Hopefully this isn't permanent, as I'm teaching myself computer science now, but I could've saved myself a lot of work if I had just chosen a concentration more suitable for the jobs I was interested in.
You may be a confident auto-didact but even auto-didacts tend to have large blind spots. You don't know what you don't know, and school is there to tell you.
As a Computer ENGINEER, I can easily tell you that Computer SCIENCE is essentially a liberal arts degree. Seriously, they don't know st about technology, unless it comes safely wrapped in an API.
And by the way, studying Physics is orders of magnitude more difficult that learning CS. And for that alone, I would hire a Physicist over a CS guy any day of the week.
Ask any recruiter, and they'll tell you the same thing. Physics is a higher pecking order than CS.
I don't know where you are, but if you're in the U.S. (or even if you're not), feel free to drop me a line. Some of the best developers I know come from non-CS backgrounds and I know folks who are always looking. Email's in my profile.
You may be a confident auto-didact but even auto-didacts tend to have large blind spots. You don't know what you don't know, and school is there to tell you.