Traditionally, it's polite to return tokens of affection, letters, etc. The "My dear so-and-so, I regret to inform you that my heart is no longer engaged in this alliance, and so it is with only mild regret that I return to you your letters and the handkerchief you gifted to me ..." is sort of a Victorian ideal of breaking up with someone.
It's something I did in my dramatic teenaged years, and I found the closure so satisfying that I have done it since. I return jewelry, I delete emails, I make sure that the massive sweatshirt I stole gets returned so I'm not even temped to roll around in his or her scent... First of all is the aforementioned closure. Second of all, it clears out a lot of stuff that might remind you of your relationship - this goes for good reminders as well as bad ones, by the way - and it tells the other person that you have no intention of taking your revenge by spreading their love notes, personal effects, or ... pictures... all over the local area or the internet. They may not show you the same courtesy, but there you have it.
That said, I have had it taken the wrong way before, but all it required was a simple, "Yeah, I thought you'd like your stuff back."