Hi guys, I added this response to another post but its just as relevant here.
We use a VPS (and dedicated) and sometimes our sites/applications slow down. Remember if your host has one big 16GB dual CPU Server with say 4 disks in a RAID conf with a 10MB pipe BUT has 16 VPS environments all possibly running multiple sites in those environments then there is a lot of disk activity. Those disks can only read and write certain amounts of data no matter what. So if your host oversells and all the sites on all the nodes go wild then you will see a performance hit.
We are suffering slow speed on our sites from heavy bandwidth and disk activity from other VPS users. One of our VPS is idle much of the time and runs pretty fast, but sometimes we suffer slow speeds in those idles times because of the hardware activity. VPS environments are great, but they cannot allocate you dedicated resources on the NIC and disks (OpenVZ for certain) so you will suffer performance issues if others on the node are hogging!
In my opinion and testing, dedicated servers are much better and perform better consequently in line with our apps. VPS are cheap, quick and easy to setup.
With a dedicated server it means just that its all yours. Although VPS you get all your own software and environment and you share hardware. Anything that you share means slower.
As for security, there is an added layer of access for the host to get into any part of the VPS file system. It adds a security risk in my eyes, because you really do not know who has access to the nodes. That is my only concern with VPS along with the disk issues.