It's interesting (and humorous) to see some of the same people who complain about speeding carts...missing the obvious here.
What I personally do in this situation where there is a lot of traffic, is to slow down as I'm approaching the walkers coming at me and then stop just ahead of them...making them either step into traffic or the grass.
If there's a sidewalk, I will slowly shake my head...and tell them that they're breaking the law.
If there is no sidewalk, but they could easily step into the grass...I just slowly shake my head and smile.
So what if I'm inconvenienced for a few seconds?
Which has the same answer...if I am passed by other carts.
Big friggin deal. 
And let's not forget that this situation occurs much more frequently on neighborhood roads where there are no diamond lanes...albeit there's usually room in the lane to safely pass them