Windows 7 Ultimate PC slowing down

I've noticed this recently in three different applications:
1. Team Fortress 2
2. Endless Space
3. Windows Media Player

If I'm not moving the mouse or using the keyboard (using some form of input device), my video gets really choppy (the system seems to be running ok, but it resolves itself fast enough that I don't know if the system is bogging down during this time). Moving the mouse or hitting a key fixes it after a few seconds. Best estimate is that it takes about 65 seconds between stopping input and the video going choppy.

I haven't gone through the usual steps like reinstalling drivers and such yet. I will when I have time.

Edit: A few more specs.
Intel Core2 Quad 2.67GHz
HIS Radeon HD 6670 1GB DDR5 (Driver ver. 8.980.0.0)
I have a failing SATA HDD, but it's one of my backup drives (three 250GB drives). My main drive checks out fine.

Load up resource monitor and see what's going on. It sounds like some background task deciding that the computer is idle and doing some CPU hogging process.

Well, that gives me a clue at least. After that time, I see "fmapp32.exe *32" take over the CPU. I googled it and one site said some malware can camo as that file to evade detection. I guess next step is to see if Malwarebytes can pick it up.

Updated MB and it found it. It was a trojan that got into my system. Hit remove, rebooted, and it's not popping up anymore.

Kinda seems a bit... clumsy in it's coding though. If the trojan responsible was written in a way that it didn't pull 90%+ of my processing when it ran, I probably wouldn't have even known I had been infected.

Yes, seems a bit odd. Unless the malware is intending to install SETI@home I would have thought most malware would keep a low profile.