Exact same issue here with an Intel 82579LM card. Kudos Brad! I'm glad I found your post. I have been fighting with this for a while now and am seeing the same behavior since upgrading to Windows 8.1 and/or updating my Intel driver. Seeing TCPIP spiking as identified by LatencyMon. Come on Intel, this is something that you have obviously caused.....fix it! This affects my audio playback, conference calls at work, and mouse movements. This is a major issue.
My configuration:
HP Z420 workstation with latest BIOS and driver updates as of today
Samsung SSD drive (problem existed before migrating from SAS drive to SSD drive)
NVidia NVS 510 with 3 monitors
Windows 8.1
Intel 82579LM network card
NO wireless
Logitech wireless keyboard and mouse
Logitech USB webcam
Logitech Bluetooth TouchPad (problem existed before adding this device)
Vishnu