I have a 2001 jeep wrangler sport The fan speed only works on fan speed 1 through 3, fan speed four does not work at all

SOURCE: 2001 Jeep Wrangler heater blower stopped working

Yes I would go with the switch, BUT, does it have a resistor bank? That is, how does it reduce the speed? Many systems use resistors to lower the speed. Burnt or bad connections can cause this as well.
You can test the blower motor by connecting direct power from a source like the cigarette lighter if you are handy.
Most of the time the lower speeds konk out but the high speed continues to work this is the resistor bank that goes out.

SOURCE: 04 Wrangler AC fan only works on high speed (position4).

Its the switch behind the fan nob thats not making contact, you have to pop the cover off the dash where the radio fan and other gizmos are. Then remove the fan nobs by pulling them straight off after that unscrew the housing unit inside is the switch. You must carefully pry this open so as not to break the plastic clips holding it together the switch thats fryed should just pop out easy replacement fan switch! If my mind serves me right it costs 40$ to replace if you order from chrysler.

SOURCE: what is the fuse box diagram for a 1997 jeep

does this help
I took a stab at which box you needed...