A memory stick is a reliable way of holding files. Floppy Disks are one of the worst ways to save files in my opinion. CDs and DVDs are another good way to save files.
Here are some examples of prices you might pay from Best Buy:
Using one is rather simple. Here are some directions on how to use it.
First a description of how to back up files:
Plug your memory stick into your computer. Look for a
USB port to plug into. Here is an example of what
you are looking for. 
When you plug it in the computer will work for 30 seconds or so in recognizing it. While it does this it will show you a couple of messages.
After the messages are done it is ready to use.
Locate the files you wish to back up.
Right click on the file or folder you wish to back up.
It should turn blue. You should also see a list of words you can
click. 
Place your mouse over the word "Send To."
When you do this another list will appear. It will
show you all the places you can send the folder to.
Click Removable Disk. It may also say USB memory. Look for either.
Make sure you give yourself enough time to back up your files. Enough time is something like a minute or two.
Here is information on finding your files and checking how much space you have left on the disk.
Start up your computer like normal. It is important that your computer is running for this to work.
Place your memory stick in the USB port.
A window should pop up with options. Click the one that says
"Open folder." Click OK 
Checking how much space is left.
Place your memory stick in the USB port.
Give your computer about 30 seconds to recognize the memory stick.
Click My Computer. This should be on your desktop. If it isn't on your desk top click Start and look for My Computer.
Be aware that you may have several Removable Disks. You may have to right click a couple to find the right one.
On this computer it is Removable Disk F. Right click
on Removable Disk F. 
A list of options will come up. Click Properties.
A pie chart will appear and show you how much memory you
have used and how much is left.
