Your Setter’s problem sounds frustrating for both of you!
In your poor dog's mind, there is something very scary about the noisy dogs behind those fences and he just doesn’t want anything to do with it. He has most likely been very startled in the past by one of these dogs (I can certainly relate to that!) and whilst you know that the fence keeps them safely separated, he doesn’t!
I would start off by approaching the offending houses, remain calm and making sure you aren’t altering your body language in any way that may signal to your Setter that there’s something to be worried about. As you approach the house make sure you reward him (only for good behaviour, because he hasn’t started to show signs of fear yet) with his favourite treat. If he knows how to sit and drop, get him to do these things as well. Basically the aim is to get him to start focusing on something else before the trigger hits. Once he’s successfully responded to a few commands turn around and go back home – don’t push him to pass the ‘scary’ houses, this will only make things worse! Each time you take him out, repeat the commands and verbal praise but try to inch closer to the houses with the dogs. If he starts to show fear, back off and repeat the exercise further away, guaranteeing a positive outcome. This process is called desensitisation and whilst it can’t be achieved without a good dose of patience on the owner’s part, it does achieve results!
I would also recommend training him to understand the word ‘leave it’ meaning leave it and lets keep walking and the ‘lets go’ meaning okay that’s enough of what you’re doing we’re going now! This way in the future, when you have successfully desensitised him, you can easily say these commands to him if you pass a house with a dog ensuring that he listens to you and just keeps walking, ignoring the growls coming from behind fence!
Best of luck