Going from your cable/DSL ISP to your house and then back to the ISP adds quite a bit of lag as well, compared to a server sitting in a DC somewhere.
I've made a beefy VM or even used a spare server at the office on a 100/10mb pipe and a $5 VPS is much more responsive. Especially noticeable with things like an IRC shell.
I'm VPNed into a Raspberry Pi back at my house right now. Works fine. I just followed a tutorial or two but if I remember correctly, it wasn't much more than "apt-get install openvpn" and a few configuration steps. It's actually running on my XBMC Pi that's already always on and connected to a TV.
I'm guessing that it would cost less than $5/month on energy and I have one sitting on a drawer.
Also, I don't live in the US and proxying all my data through the US and back would introduce unwanted lag.