The recoil rope works when it off the engine, however, when it is attached to the engine, it does not recoil like it should.

Cool!
A Project.
Remove the spark plug lead just so you can't have it accidently fire.
Remove engine cowlings as necessary to get at recoil starter.
Remove recoil starter (that's the bit that the pull rope is attached to.
Dis-mantle it until you can find what is wrong. Fix or replace.
Put it all back together.
or
Take it to a small engine repair shop and pay a bunch.

How To: Rewind a Recoil Rope (Fix a Broken ... - YouTube

? 2:52
https://www.youtube.com/watch?v=J0KJuqsJJ_k
Feb 16, 2012 - Uploaded by Champion Power Equipment
How To: Rewind a Recoil Rope (Fix a Broken Recoil Rope) .... Back off until rope hole in pulley lines up ...

How to Repair a Small Engine Recoil Starter - YouTube

? 4:49
https://www.youtube.com/watch?v=tmo6oYOJJ4U
Jul 15, 2011 - Uploaded by eReplacementParts.com
How to Repair a Small Engine Recoil Starter ... Would like to now use the pull cord on the mower but it won ...

Hi Donna:
Most of the pull start assemblies are attached so that their centre is directly over the centre of the driveshaft. There will be several bolts that connect the assemby to the engine, and when they are removed, the pull start assembly can be lifted off.
If you turn it over and look at the underside, the cover for the spring and rope will be attached, possibly with pop rivets, or it may be held in place by a centre bolt.
Patience and caution will be your friends as you determine how to dis-mantle it.
If the spring is broken, see if you can get some tips from the service people where you buy the replacement.

Although it looks like you simply un-bolt the recoil starter you need remove the whole blower (flywheel) housing as there are nuts on the back side.

• Once the housing is off, remove the broken rope and prepare a length of rope to replace it base upon the old ropes length. Probably close to 6 feet.Don't forget to put the pull handle on the new rope.
• Tighten the recoil pulley in the direction it normally turns as you pull the rope out until it stops.
• Slowly back of up to one turn to align the hole in the pulley with the housing hole and hold it in that position while you insert the rope and try a sturdy knot.
• Slowly let recoil assembly pull the rope back in and test that it is working properly before re-installing the housing.
• Once you are satisfied that the recoil is working properly re-install the remaining covers.

Remove the 3 nuts holding the recoil assembly to the flywheel. Observe how the old rope is attached and wound. Rope is sold by the foot at mower repair shops. 3-4 ft should do. The rope is held in the recoil assembly by a knot and held in the handle by a knot.

Most recoil assemblies are easy to fix...but not in an obvious way. Remove the recoil asembly from the small engine housing/flywheel cover. Usually three bolts hold it on. 10mm on Honda's. Pull the remaining rope thru the exit hole if possible. Otherwise, fish remaining rope thru the hole by turning rope winding pulley as if pulling handle was still attached, lining up any remaining rope end and pushing thru the exit hole in outer recoil housing or backing it out the inside pulley. Continue to wind rope winding pulley in a counterclockwise ( facing pulley...not outer metal) untill it will not tighten any further. Remove any more old rope. Line up pulley hole and exit hole. Melt end of new rope into a tapered shape..if not supplied that way. the pulley can be held in place by inserting a screwdriver thru the both the pulley and the metal housing spokes. Thread the new rope thru the exit and pulley hole. Tie the rope end sticking thru and draw tight. Cut 6 feet of rope...if not using a presized pull. Release screwdriver from spokes and rope should draw up onto the pulley. Tie off opposite end of the rope to keep from losing that end into the exit hole. put the handle on the rope and tie a knot in the end to retain the handle. Untie the tie-off and allow the handle to fully retract. If handle does not fully retract you may shorten the rope so that it does...as long as you have plenty of pull lenght

Remove nuts mounting recoil assembly. Observe how rope is wound around pulley. Find end knot in pulley wheel. Cutoff end knot and remove old rope. Find end knot in handle, use needle nose pliers pull thru old rope. Rope is sold by the foot at mower repair shops. Replace rope as observed before removal. Cut to size after wrapping recoil assembly and attaching thru handle at proper length if at recoil assembly or on mower handle. Ensure good knot in hole of recoil assembly and handle.

Remove recoil assembly from top of engine. Observe recoil wheel and find knotted end of rope in wheel, Observe direction of rope wrapped around wheel, cut knot off wheel end of rope remove rope. Rope purchased by the foot at mower repair shop. Feed new rope end into hole in recoil wheel wrap around in correct direction, feed rope thru recoil cover, remove old rope from handle, feed new rope thru handle to proper length, Knot rope to stay in handle.

The recoil spring has to be pre-loaded about 5 turns, even when the rope is fully returned. When you pull out the rope, you increase this tension, but if you are starting from zero, there won't be enough spring tension to fully recoil the rope. Wind the rope on the spool, then rotate the spool about 5 turns in the direction that would "push" the rope out through the hole. Now put the rope through the hole and put the handle on it. If you got the tension right, it should be sufficient to easily hold the handle up against the hole.

sounds like the dogs and or springs are worn.When you had the cover off the machine did you check that the dogs would freely move out when the rope was pulled? They some times stick and will not engage.