It is a small rubber gasket in the top of the refill assembly. You can get them at Lowes or Home Depot. You do NOT need to replace the valve or the assembly or the "guts". It is relatively easy to replace, the hard part is getting the top of the refill assembly off to access the gasket (be sure to shut off the water supply valve first). This video should help:
How to fix a toilet: replacing the Fluidmaster 400 fill valve seal (HD) - YouTube