RCA Whine, Newly Installed Amp

I had infinity speakers in my car when I bought it, but the amp was taken out. I put an amp in yesterday and cannot get rid of a high pitched whine through the speakers only when the car is on. I also had typical alternator whine but found that be from a bad ground (it's fixed now). Here's the situation, if the cars on and running I have the whine. When I unplug the two rca's for the front speakers from the amp the whining goes away, same for when I unplug just the rear rca's. If three or more are plugged in the whining returns. Any suggestions on how to get rid of it because I haven't found any.