You could always split the difference. Lift your cards half an inch off table and wait. Don't make the "I'm about to toss these in the muck" wrist action, like you're faking a fold, just hold them slightly off the table. Are you lifting them to flip them over? Or lifting them to muck when he folds? Or lifting them to muck when he calls/raises?
Now you've just reflected the ambiguity back onto him. He doesn't know what it means, and you aren't really being a jerk. You're just prepared to respond when he completes his action. Why, you're just preparing to speed your action for the benefit of the table when he FINALLY completes his! Holding your cards like you're ready to make either motion, if anything, only puts more pressure on him to finish his action.
I'm not sure I even really believe my own advice here. I actually think the previous posters are probably right - don't react to his angle, just wait it out.