For my own record and maybe for your convenience I wrote down the steps to install a Debian/Ubuntu VPS as web server. This setup is optimized for low memory usage (256-512 MB). This means nginx instead of Apache, no DNS and e-mail server (only outgoing mail) and some MySQL, PHP and nginx tuning.
This setup guide has been tested for a VPS on OpenVZ, Xen and KVM and for Debian 6.0 (Squeeze), Ubuntu 10.04 (Lucid Lynx) and 11.10 (Oneiric Ocelot). My favorite combination so far is KVM and Debian 6.0 with the Dotdeb repository (fast virtualization, stable Linux and the latest server software).
See here for the guide: http://blog.bokhorst.biz/6507/computers-and-internet/how-to-setup-a-vps-as-web-server/
