Start with these topics. Work your way through them, and make sure you understand everything you have read so far before progressing. Do the exercises, or it will be a waste of time.
0) Algebra, Trigonometry, Calculus
Make sure you have a decent grasp over high school level math topics. You might not need to use these topics frequently (though trig comes up a surprising amount), but they are necessary to establish a base level of mathematical maturity.
1) Linear Algebra
Obviously very important if you want to do any 3d work, but it comes up in later topics like graph theory.
Suggested book:
Linear Algebra and Its Application - Strang
2) Discrete Math
You need an understanding of proofs and logic before you can get to the real algorithms material. Counting and probability is also very important.
Suggested book:
Discrete Mathematics and Its Applications - Kenneth Rosen
This should take you some time as each of these topics usually corresponds to a college class. When you finish with those, start reading about CS Theory, Combinatorics, or pick up a graduate Algorithms text.
You say 'trig comes up a surprising amount'. It would be great if you could give a few examples of these instances. I have always thought of trigonometry as one of the areas I really enjoyed in school but something I have never used ever since.
Two recent projects come to mind, one where I was working on a program to draw and manipulate points in 2d space where I used rotation matrices, and another where I was testing the graphing routines in a monitoring service by sending load in sinusoidal spikes.
Trig? Fourier series and its introduction to functional analysis and Hilbert space and the ideas of inner products and orthogonality there. JPG image compression. Shannon's information theory. Characterization of time invariant linear systems, e.g., acoustics. Antenna theory. Phased arrays and beam forming. Imaging in optics. Holography. The fast Fourier transform. Seismic data analysis via the fast Fourier transform. Characteristic functions and Bochner's theorem. Power spectra. X-ray crystallography. The fundamental theorem of interpolation and the Nyquist sampling theorem (e.g., how music CDs work).
There's a cute idea to put 100 wireless customers all on the same wavelength. Basically have a tricky antenna pattern with a lobe for each user and so that each user gets only their own signal. It's all trig.
Indeed, I agree. Those all fall under the umbrella I gave. But those are all particular subject areas that many people can go through their career without touching.
Yes, all or nearly all! That the trig functions are the source of the most important orthogonal basis in Hilbert space theory need not be good to know just for physics or signal processing!
Let's see: Also I mentioned characteristic functions and Bochner's theorem which are core probability and not just physics or signal processing. Characterization of time invariant linear systems might be in mechanical engineering and might be tried even in economics! Seismic data analysis via the fast Fourier transform is, yes, in signal processing but is also mostly regarded as geology or just looking for oil or anything 'down there'.
I tried, as you can see, I really tried, to show how trig was for more than just your father's topics in physics and signal processing. I tried!
0) Algebra, Trigonometry, Calculus
Make sure you have a decent grasp over high school level math topics. You might not need to use these topics frequently (though trig comes up a surprising amount), but they are necessary to establish a base level of mathematical maturity.
1) Linear Algebra
Obviously very important if you want to do any 3d work, but it comes up in later topics like graph theory.
Suggested book: Linear Algebra and Its Application - Strang
2) Discrete Math
You need an understanding of proofs and logic before you can get to the real algorithms material. Counting and probability is also very important.
Suggested book: Discrete Mathematics and Its Applications - Kenneth Rosen
This looks like a good free option, but there are no exercises: http://courses.csail.mit.edu/6.042/fall10/mcs-ftl.pdf
3) Algorithms
Enough said.
Suggested book: CLRS, but here's a free option: http://www.cs.berkeley.edu/~vazirani/algorithms.html
This should take you some time as each of these topics usually corresponds to a college class. When you finish with those, start reading about CS Theory, Combinatorics, or pick up a graduate Algorithms text.