Wikipedia, as always, has a great introduction to VPS technology. If you are interested in information about specific virtualization software (such as OpenVZ, XEn etc.) see the Comparison of virtual machines article.
When you have selected a VPS plan and configured a server it is time to set up your web application framework. Here is more information on how to proceed.
Securing your server
For many Linux distributions you can use iptables for firewall configuration. See the Sitepoint article Secure your Server with iptables.
If you are running OpenSuse a simple firewall setup can be done in the Yast control panel. For more details see SuSEfirewall2.
The official Django website has a section on deploying Django and there is more information in chapter 20 in the online Django Book: Deploying Django.
For more tweaking and performance tips see Django Performance Tips by Jacob Kaplan Moss.
Ruby on Rails
The Ruby on Rails wiki has a number of pages on Rails deployment.
Digital Media Minute has a thorough guide on how to setup Rails on a Fedora Core 4 server. See Installing Ruby on Rails with Lighttpd and MySQL on Fedora Core 4. It covers configuration with Yum as well as database setup. Find all Fedora VPS plans here.
The official Ubuntu docs has a section on getting Ruby on Rails up and running on an Ubuntu server. Find Ubuntu VPS plans with 256 Mb RAM or more here.
If you want to use Debian see The Perfect Rails/Debian/Lighttpd Stack by Ezra Zygmuntowicz. You can filter out Debian VPS plans with 256 Mb RAM or more here.
Drupal can be deployed on many platforms, both Linux and Windows. See the official Drupal.org list of requirements.
- A small Drupal site may not need the full resources of a VPS, but may work a lot better under load. This discussion mentions that a 64 Mb VPS won't cut it so you should start with a minimum of 128 Mb RAM. See all VPS plans with 128 Mb RAM here.