Sorry I wasn't clear - my question was why do you recommend C and C++ over Python? is it specifically because of the distributed/GPU programming aspect?

I think his meaning was if you're looking to create new ML algorithms, rather than just apply the algorithms of others, you'll need C and C++. If you're just looking to apply the algorithms of others, or if you're 100% sure you'll only be (for example) using TensorFlow to develop new algorithms, Python should work great.

That, and also sometimes writing something in C is the optimal solution (faster than Python, and simpler than trying to modify TensorFlow or Theano backend).