Grainy, Spot Filled Images when using Manual Modes Only

I have a Canon 20D and am using a Canon 28-135 IS lens with it. The camera was bought refurbished about a year and a half ago. The lens is only a few months old. I notice that when I take pictures in the manual modes like Aperture Priority etc, my pictures turn out grainy and the dust spots are a lot more prominent. When I shoot in the automatic modes there is no graininess and the dust spots are very few. The dust spots I know can be taken care of with a cleaning, but the graininess really concerns me. I had a similar issue about a year ago and took it into a repair shop because the camera wouldn't even take pictures in the manual modes at all. They fixed the problem, but couldn't really tell me what they did. And now it appears that the problem has come back. Any idea what this could be?