If the math is giving him the wileys, a book like 3D Math Primer for Graphics and Game Development is helpful. It assumes basic alegra, teaches vectors, matrices, and then gets into things like collisions, frustrum culling, quaternions, lighting calculations, etc.
If he is serious then he will need some reference book for the 3D graphics engine he will be using. I do all my 3D work in OpenGL, so I have
* OpenGL Reference Manual (API calls, basically)
* OpenGL Programming Reference (what you really need)
* OpenGL SuperBible (not too bad)
Someone else will have to recommend the appropriate books for DirectX.
Honestly, as a sometimes hirer of programmers, I have little interest in somebody who is skilled in a particular technology such as DirectX. That can be learned, easily enough. Technology changes fast enough that a programmer who is going to be with your company for more than 6 months, or a single production cycle, is just going to have to learn new APIs, etc. It's a given. I'm far more interested in somebody who understands solid programming practices. For example:
* Code Complete - for low level programming skills
* Design Patterns - often overemphasized, but chapter 2 in that book gives a great architecture for a complex piece of software. Really learning and understanding the techniques used will allow a programmer to write robust, extensible code.
* Effective C++ - or whatever the equivelant is for his language of choice. C++ is a professional language, and you have to know it's in's and out's.
I'm not in games, I produce other 3D software, so what I write may not apply, but in general I only hire people who know how to program, and program well. In the last 10 years I've gotten exactly 1 phone call saying my code crashed, and I expect the same from anyone who works for me. If you can do that, and learn a new API quickly, you have a job. So to me books like the above are more important than, say, 3D Engine Design, though I certainly understand the appeal of the latter.
With that said, if he is interested in 3D development, it can take a while to really understand what a graphic engine is doing, and if you don't understand it you end up "programming by magic". Which means you keep trying things until they work. Problem is, what you do may not work in all cases, or it may only work on certain cards, etc. if he really wants to get it, it might not be bad to write his own graphics engine. Yes, it will be slow and limited, but a basic engine that allows you to draw triangles and view it from any camera position will pretty much guarantee that he really understands the math, as long as he didn't cut and paste a bunch of code and then monkey with it until it "works". Then throw the thing away, as trying to add lighting, etc., will just be a waste of time, and go back to using DirectX or OpenGL. It's just an idea, and certainly not required, but I found my understanding reached the "Aha!" stage when I did it.
