There is something tasty in there that he is after- could be snails, grubs, or worms, they love 'em.
You might try pulling the mulch away and tilling the soil a little around the plants to see if you can discover anything that a granular treatment can help with (like grub treatment)
Spices (like cayenne pepper) won't deter him- birds do not have salivary glands, so the pepper does not faze them.
A rubber snake or owl (as suggested) may help, but must be moved around or the birds catch on- and you will not be there to move it daily.
My best suggestion would be removing the mulch in the targeted spots, then laying down some flagstone pieces where the bird digs. If he can't get to what he wants, he'll move on- and you can remove the flagstone later if you prefer.
Most any landscaping center will sell small pieces that can be easily handled.
Good Luck!
