So I think you can conclude the that the first motherboard was actually ok, so don't trash it :)
Just to more certain about your PSU you might test it with a PSU tester, from $20 upwards from Amazon or ebay. This won't actually tell if everything's ok with your PSU, but if there's something wrong, it may show it (fail=something wrong, pass=there might be something wrong).
OK Brian, I have two ideas, which you may have already eliminated.
The first idea is that the problem is thermal; that maybe your processor was overheating. This idea is made less likely by the fact that the second m/b also shows a problem, but does not eliminate it entirely. The TIM on your CPU may have become old and developed voids in the first case, in the second, is it possible that you didn't do the best job in mounting the CPU heatsink ? You are obviously competent to remove to remove a motherboard, and so I expect you thoroughly cleaned your PC of dust, including the CPU heatsink and PSU where possible in static-sensible fashion. You should install some temperature monitoring software to check this hypothesis. Quasi-real time monitors include Speedfan (which also has a graphical display showing temps for the past 15 minutes, but it can be difficult to work out what the readings mean), and coretemp, but I'm sure there are others. Sandra will give you temperature but takes a couple of minutes to update, Speccy might give your CPU temp but it doesn't give mine (HP specs for this model do not give a unique CPU) . Look up your CPU's max. allowed temperature on the AMD website. You might also try CPUworld.
The second idea is that your PSU is overheating (some PSUs have thermal sensors which shutdown the PSU if it gets too hot), or that your PSU simply not upto the job. It depends what you are doing and what your system configuration is (including peripherals). According to http://www.anandtech.com/show/2630/6
the system under test with HD4350 graphics card, which is one of the options for your model, draws 165W at idle and 195W under load; this should be compared with your PSU (which if it's the original) is rated at 220W. Put a couple of extras in that, and take into account that the PSU isn't able to give as much power as when it was new, this scenario is looking like a possible candidate. You can eliminate this by trying a different, working higher power PSU if you can find one that fits - about 500W (true, not claimed or peak) capacity (to estimate
the wattage the PSU you need for your system, see http://extreme.outervision.com/psucalculatorlite.jsp
(newegg and some PSU manufacturers also have them).
Of course it might be one of hundreds of other causes,,,
You might try posting your problem in the HP support forum, and hope that some HP guru picks it up.