The more stats you have the more work PT3 has to do, but I'm not sure how much effect the number of Groups will have.
Have you tried the different 'HUD Performance Level' settings on the HUD Options tab? Different settings will work better on different computers, so try them out.
You may also find these posts help:
Performance Tuning: Hard Drives
Performance Tuning PostgreSQL