Wish i had had the presence of mind to pause and do the same in a recent interview (where despite 'pushing back' against a counter comment, the interviewer insisted that I didnt 'push back' enough). I retrospect, i had asked a couple of probing questions in an earlier interview about his funding for the initiatives he had proposed, and he might not have taken kindly to those questions (unjustifiably, i feel).
Another git on the panel yawned in my face halfway thru a 45 minute interview. I've been in more than one interview situation where the interviewer seems tired out, hasnt had a good nights sleep etc. Comes across incredibly rude.