You didn't mention what year model your bike is so I'll just generalize my response. Since you said that you replaced the solenoid, I'll assume that your bike is a 1988 model or earlier.
Your bike has a starter relay if it still has the stock wiring on it. Harley has used a starter relay since 1965 on the first model of ElectraGlide. Usually, it's underneath the battery tray or the seat or around that area.
On the back of your solenoid, you have three wire connections. Two very large connectors and one small connector. Make sure your bike is out of gear (in neutral) and use an old screwdriver to short between the large connector that comes from the battery and the small wire connection. The starter should engage and try to start the engine. If the ignition switch is on, it will start the engine. The starter will turn using this method with or without the switch being in the "on" position. If the starter works using this method, the problem is in either the relay or the neutral switch. If the starter does not turn the engine over, the problem is in the solenoid.
Now, let's check a few things. The small connector on the back of the solenoid should have a green or pink (depending on year) wire on it. Using a voltmeter or a test light, make sure you have voltage at the connector when you press the starter button with the switch in the "ON' position. If not, follow the wire to it's source, the relay.
The starter relay can be one of several different designs used throughout the years. It could be a small plastic cube, a small metal can, or a round phonelic relay. The relay should have four connections on it. A "hot" wire, a wire from the handlebar switch, the wire going to the starter, and a ground. The ground may be through the case itself. On the older Shovelhead bikes (1984 and earlier) there was a small short black wire that ran from the starter relay to the transmission for the ground. This wire must be intact or the relay would not work due to lack of a ground.
When you turn the switch on, one of the wires to the starter relay should become "hot". When you press the start button, you should hear a slight click and another of the smaller wires should now be "hot" as well, the one going to the starter.
On some year (1972 and later) models, the neutral switch was wired in with the starter relay. This was to prevent the bike from starting while "in gear" by disabling the relay. You'll have to figure this one out for yourself since I don't know what year model your bike is.
Now, you said you had power to the solenoid when you pushed the starter switch. So, let's assume that the starter failed the first test to told you aboue. If so, the problem is still most likely in the solenoid. Inside the solenoid, there is a large plunger with a copper disc on it. When you depress the starter switch, the coil in the solenoid becomes magnetized and pulls the plunger towards the back of the solenoid. This does two things, it engages the starter drive with the ring gear on the outer clutch drum and makes a high current electrical connection. The copper disc makes contact between the two large connections on the back of the solenoid from the inside. This connects the battery to the starter motor through the solenoid. If the black phonelic plate on the back of the solenoid is cracked or the contacts inside of it are badly burned, it will not work.
Now, if the solenoid is working correctly and you are getting voltage to your starter, it could be the brushes or something inside the starter. This is indicated if the starter trys to turn the engine over but just can't. It won't have enough power if the starter field windings are bad.
I hope I've given you something here that will help you solve your problem. This is basically the electrical part of the starter system. There are mechanical parts as well. If you hear the starter turning but the engine doesn't turn over, you have a mechanical problem. You can either repost or you can contact me directly at email@example.com
I'll help if I can. Good Luck!