Is Harry's Horcrux benign?

How come the Horcrux in attached to Harry's soul (for years) does not affect him like the Horcrux in the locket.

The Horcrux in the locket affected Ron's, Hermione's, and Harry's moods (when worn for any length of time) and even hindered Harry from making a Patronus (while he was wearing it).

The Horcrux attached to Harry's soul seems almost benign by comparison (unless of course Voldemort is near or trying to get into his mind or killing someone etc.)