There are a few issues that cause a 5100 error code. 1) The encoding strip may need cleaning. 2) There is a paper jam. 3) The printer may need to be reset by turning it on and off. 4) The ink tanks may need to be properly snapped into their carrier. 5) The printer may need service.
The solutions to try
1) Cleaning the encoder strip. Use a cotton swab and isopropanol (rubbing alcohol) or window cleaner. A flashlight will help you see any grease on the encoding strip. With the printer on, open the printer by lifting the scanner unit until it locks. Unplug the power cord from the printer and wait until the LEDs are no longer lit. At this point you will be able to slide the cartridge carrier back and forth. To start, slide the carrier to the right. Look for the clear plastic strip (the encoding strip) in front of the carriage drive belt and look for any grease or other dirt on the strip. Remove the grease with a cotton wab moistened with the isopropanol. Look at the strip from several angles to check for any grease along the entire strip. Make sure you do not remove the grease from the chassis (metal rail). Move the cartridge carrier to the left and check the right side of the strip. When all of the grease is removed, lift the scanner unit support and close the unit. Plug in the power cord and turn the unit back on. Often grease gets on the encoder strip when you remove a paper jam. If you proceed to the next step, you may have to repeat this cleaning.
2) A paper jam
Carefully pull any sheet from the rear tray or paper output slot. Detach the rear cover and fully extend the rear tray support and try to pull the paper out slowly. Otherwise try turning the printer off and on - the sheet may be ejected. Remove the paper cassette and pull any paper out slowly that is in the front area. If needed, try removing the paper cassette and any paper in the rear tray. Then turn off the printer and unplug it. Set the unit upright on the left side and try to pull the paper out. Return the printer to the usual position when done. Replace the cassette, plug in the power cord and then turn on the printer. This will delete any documents in the printer queue. Check for foreign objects around the paper output slot. Remove any foreign objects with the unit off. Close the rear cover completely. Make sure that the paper isn't curled up.
3) Turn the unit off, unplug the power cord and wait 10 seconds or longer. Plug in the power cord and press the On button. Wait until the printer completes the cycling process before trying to print again.
4) Verify that the ink tanks are properly seated. With the printer on, lift the scanning unit up until it locks. Press on the black circle on each ink tank until it snaps firmly into place. Check if the ink lamps light up red. Make sure the protective shipping covers was removed from the ink tanks. If the unit has the remaining ink level detection feature disabled, replace the ink tanks. Close the scanning unit.
5) If none of the above solutions work, contact Canon if the unit is under warranty. Otherwise, decide if the service is worthwhile for an out-of-warranty printer.
I hope this helps.
If the paper tears when you try to remove it, open the scanning unit with the printer on and unplug the printer. Then look for and remove the paper. Use a flashlight to check for paper and other foreign objects in the corners. Slide the cartridge carrier back and forth to check the corners completely.
For pictures check the support site: http://www.usa.canon.com/cusa/consumer/products/printers_multifunction/office_all_in_one_inkjet_printers/pixma_mx860#ServiceAndSupport .
Search for the error code 5100 and then follow the links.