1995 Chevy cavalier starts but won't keep running

Dont bang on the starter. Just replace it.

I would need to know the actual code numbers to comment, but in general those components would not keep the engine from starting.
Have you checked compression, spark and fuel pressure ?

use test light, connect clip on grnd. , touch tip on the starter wire while turning key , if it lights starter is faulty , no light, solenoid is faulty, if it lights and indicates starter is cause you can try to gently tap starter with plastic handle of screwdriver if tapping gets it moving then brushes in starter are dirty or near worn out hope this helps

http://www.autozone.com/autozone/repairinfo/repairguide/repairGuideContent.jsp?pageId=0900c1528003d007Your fusible link could be burnt and not getting a good connection,follow all wiring tied into positive battery cable for burnt or swollen wires.Click on link.

try locking your car with the key and unlocking it with the key, leave it for 30 second or so between the locking and unlocking , the car thinks some one has tampered with it

this is a common problem with the passlock system im sure like everyone else you will have to get a new ignition cylinder/switch

try by changing the coolant temperature sensor, this may be your problem. Coolant sensor problems will manifest themselves during cold starts and during the warm up period, such as hard starting, erratic idle, smoke, stalling, etc, good-day !

Have your starter wiring checked for a short. This was a common problem on these and Cavaliers.

obviously the altenator is running to earth inside ,could be a rectifier diode .change altenator and recharge battery

Check your egr vavle. i had this problem with my old cavalier and it was as easy as removing it and cleaning it. It's probably plugged up with carbon