The PSP will not stay charged.My son used to always play it while it was charging. I was wondering if this would bring about an issue with the battery.

I would answer yes on this question. Because the battery never get to its full charge and it's really important that it does for the battery durability.
It's preferable to play until battery is dead then playing with it while it is charging.
Even if new generation of batteries is better we still have to be carefull with the usage we make with it.
Have you tried to play until it shuts down itself and then fully recharge it?
It might help a little but to expect too much.

Hope that help.

During charging the battery gaining some power to recharge itself.When you use it while charging the battery forced to give you enough power to operate the PSP.It seems your are using power directly from charger instead from battery, and It could damaged the cell. All you can do is to replace it with same model battery and dont play it while charging.If possible turn off the unit while charging.God bless