It’s not so much the exertion (although you shouldn’t discount the horrors of Sid Fernandez running the bases), as it is the lack of rest. You hear the same thing when the pitcher’s team all swing at the first pitch and he’s back on the mound after sitting down for all of three pitches, or when a defensive miscue causes a pitcher who has been cruising along to have to stay on the mound longer and “get four outs instead of three”.
I always cringe when the sacrifice “fails” and the pitcher ends up on first base. Now he’s taking leads and diving back to first instead of sitting in the dugout with his arm wrapped up drinking gatorade. The pitcher who throws a complete game may be on the mound for 90 minutes, but his routine is “work ten minutes/ rest ten minutes for three hours”, not work 90 minutes straight, and—anecdotally, of course—he gets screwed up when one of several things interfere with that cycle.