2004 chevrolet venture battery light on

Chances are the alternator is failing and needs to be replaced soon. If there is an Auto Zone around your area or you have a volt meter, with the engine running. Place the volt meter to the battery and you should get a reading of 13.5 to 14.5 volts. With the engine not running, the battery fully charged should read 12.5 volts. If you have less then 13.5 volts on the battery with the engine running, chances are the alternator has gone bad. The other thing to look for is a blown fuse or broken wire at the alternator, such as thew ignition wire to the alternator.
The alternator does come out from the top. You need to loosen the motor mount and rock the engine forward and secure it. Then removed the windshield wiper module cover, and you should have access to get it out.
I recommend for safety reasons, if you are not a experienced mechanic that you take the van somewhere for that service. When it comes to rocking engines and securing them, you are better off having a professional doing it for you.
Good luck