Okay, the PLV has nothing to do or any effect on your 12v pump operation.
The water coming out of the mains inlet whilst using the pump is caused by the NRV (non return valve) which is usually incorporated in the fitting you connect your hose to, this valve the O ring has failed or slipped off, it is a simple valve and is common failure. Best fix is to buy a john Gust inline NRV and fit it in the line near the fill point, the fill fitting can be removed and it is a simple job to just snip the internal valve with side cutters, throw in bin and refit the connector and all will be good. .............. The NRV may have been in the old PRV, either way fit the John Guest one way more reliable. $20
The hand pump may need to be dismantled, cleaned and lubed with olive oil, it sounds like the little flapper valve is stuck allowing water past when pump is working.... May even have a bit of drill swirl caught.