I run a Wireless Internet Server Provider in rural Texas. Deal with wireless issues on a daily basis. Looks like interference or poor signal strength to me. Do you have neighbors close by?, running on the same or overlapping your channel?. I'd try switching channels and see if your packet loss improves.
What your signal level and your noise level when you are experiencing packet loss? Are you far from your router? Try moving closer. Still have high latency/packet loss?
When consumer grade wireless routers begin to die, they exhibit strange, inconsistent behavior. You could try a different router at least. Stay away from Netgear and Belkin. They are the shittiest routers on the market. If you want a good wireless router that runs linux, try a Mikrotik. www.routerboard.com and www.mikrotik.com
A wired connection is ALWAYS better than wireless. Use wired when you can. In fact, try a cabled connection to your router. Still have high latency and packet loss? I highly doubt it has anything to do with software unless you are doing something unusual in make.conf. Are you an extreme ricer? ;)
Forgot to mention, if your wireless connection is being maxed out by another device on your network, that can also cause high latency and packet loss. Are you running a torrent on another wireless device?