Why do I have to hear everything instead of only the person I'm interested in listening to?

And that is a very good question.  With today's modern hearing aids we can limit background noise a lot but we never take it all away and we never should take it all  away for a number of reasons.   I will attempt to answer this with the following points in order of importance.  Please remember that this order of importance is mine and while the reasons are valid and can be backed up with research the real reason they are in this order is more practical everyday use in our soundscape of hearing.

1. We hear on three different levels and all of these levels are important to our overall well being.
1. Some of what we hear simply connects us to our environment around us, we call this the Primitive Level of Hearing.
2. Other sounds we hear tell us of events or things going on in the acoustical environment that that let us know if we are either safe or unsafe, this is the Warning Level of hearing. 
3. Some of the sounds we hear are informational in nature, birds chirping, musical notes and speech, this is referred too as the Symbolic Level of hearing

The symbolic level of hearing is what people are most concerned about.  The  reason for this concern is that the loss of hearing acuity on this level separates us from our friends, family and social network.  In reality the symbolic level of hearing is the least of the three levels as far as our body and mind are concerned. 

The warning level of hearing is tied directly into  our limbic system in the brain.  This is our emotional, primitive brain and the action/reaction center of the brain dealing with our safety and well being.   This is the home of our flight or fight responses and is solely responsible for  our survival.  When we are disconnected from this level of hearing we have a tendency to  become paranoid as we don't know what is safe and what is not safe and every noise in the environment becomes a threat.

The primitive level of hearing lets us know we are still alive and connected to  the world around us.  When this level of hearing in interrupted we feel like we have been disconnected from the world around us and this has the greatest impact on us psychologically. 

We need to hear on all of these levels, especially with hearing aids.  To limit any one or more of these levels will cause additional stress and anxiety which the hard of hearing person does not need in their lives.  A good hearing aid fitting should cover all of these levels of hearing and not just the  symbolic level.  Hope this answer some of our original question.