I feel the same way as you do and I have a large dog. Having focused on reading dog poop posts for a few months, I'll give you my version of the diversity of dog walkers
- Sneaks, knowing their dog on a long leash up in your yard is not right.
- Clueless that you mind their dogs going in your yard and would not do it if they knew.
- Brazen, know you might mind, don't care, think they are right that you should not mind.
- Responsible, non-collaborative, walk the dog in the street to appropriate public area, scoop, dispose at home.
- Responsible, collaborative, agree with neighbors as to which of them don't mind, and use each other's yards.
Suggestions have been posted to ask the neighbor not to and in your case with a pack of dogs that are repeaters that is a good idea.
I wish one could post a symbol, maybe just a red or green dot, on the lamppost to signify if it's ok or not to let your dogs go here. Many people don't mind at all if dogs go in their yard and that makes it impossible to have consensus on what is reasonable behavior.