To remove setup.exe file on

It takes up very little space (about 3 meg), I would recommend keeping the file on the device so you will have it when you need it.

You cannot play the .VOC files unless you use the proprietary software to either play the .VOC files or convert them to .WAV or .EXE.

But if you insist, you may use windows explorer and browse to the filesystem on the Digital Voice Recorder and delete the file named "Setup.exe" you will find in the directory called "SETUP". To do this, you may have to rightclick on Setup.exe, then click on properties and uncheck the "read-only" attribute. Now you should be able to delete the file. Repeat the process for the directory called "SETUP". If you try to do the directory first, you will be given the option to change the attributes of the directory and all files contained in it. That should work too.

WARNING:
At the very least, copy the contents (a single file called "setup.exe") to a safe place.