I've noticed this recently in three different applications:
1. Team Fortress 2
2. Endless Space
3. Windows Media Player
If I'm not moving the mouse or using the keyboard (using some form of input device), my video gets really choppy (the system seems to be running ok, but it resolves itself fast enough that I don't know if the system is bogging down during this time). Moving the mouse or hitting a key fixes it after a few seconds. Best estimate is that it takes about 65 seconds between stopping input and the video going choppy.
I haven't gone through the usual steps like reinstalling drivers and such yet. I will when I have time.
Edit: A few more specs.
Intel Core2 Quad 2.67GHz
HIS Radeon HD 6670 1GB DDR5 (Driver ver. 8.980.0.0)
I have a failing SATA HDD, but it's one of my backup drives (three 250GB drives). My main drive checks out fine.