To get rid of files you no longer need, you drag them to the Trash. Files stay in the Trash until you Empty Trash either from Finder / Empty Trash on the menu bar or buy clicking and holding on the Trash Can.
Once the Trash is emptied, it appears that your files are gone for good. However, special file recovery software can access those files and "un-delete" them.
To make sure that your files are gone for good, chose Finder / Secure Empty Trash from the menu bar. Secure Empty Trash writes meaningless data over top of your file to make it impossible to recover.
UPDATE: I've gotten several inquiries about the possibility of recovering items deleted using Secure Empty Trash. I've done some research on this and can't find anyone who claims that this can be done. At least not with any commercially available product. (In other words, I'm sure the NSA can do it, but you can't) The moral of the story is MAKE SURE YOU REALLY NEVER WANT THAT FILE AGAIN before you use Secure Empty Trash.
UPDATE 2: For Leopard users, read this tip about how Time Machine may be able to recover your files that have been Secure Deleted (for better or for worse.)
Shopping online? You can help support Basics4Mac by making your Amazon purchases through this website. Simply click the GO link below and you will be sent to Amazon.com. Any purchases you make at Amazon by going through this site will give me a small commission. It does not affect the price of the items you buy.
Purchase any products from Amazon.com and support Basics4Mac. GO!