Tails (The Amnesic Incognito Live System) is a live GNU/Linux distribution geared towards anonymity. It includes Tor and I2P as well as protecting from scripting attacks, and tracking of your browsing habits through cookies etc.
To install Tails to a USB drive follow the instructions here (you will need two >= 4Gb USB flash drives).