I had similar problem with my Sony Digital8 (also called HI8 ) handycam. My vista system was not able to detect my Handycam with a very very cheap FIREWIRE PCI Card I bought in local market for only $4, yes four dollars. It came with 6pin to 4pin Firewire cable(Sony calls it i.Link). I was trying to connect with same cable 4pin at camera end and 6pin on PCI Card, with no luck. It was even showing DV-IN on camera LCD with this 6pin to 4pin cable. I tried everything an expert would do, but got only frustration. Even tried a lot on Internet for the solution but it seems no one knew.
Then I started from scratch, TO DECIDED TO READ MANUAL. While reading the PDF manual found on Sony Support site, trust me, I found those golden words "USE ONLY 4PIN TO 4PIN i.Link Cable for Firewire connection."
I ran back to market and bought 4PIN TO 4PIN Firewire Cable for $1.5
At first trial, trust me, it detected "SONY DV Camcorder" and installed drivers automatically. I confirmed by going into Device Manager and found same entry in Imaging Device section. Then what in every possible software existed on my system, DV Capture was available, which was found always disabled.
Now the funny part it, technically there is no difference in 6pin to 4pin Firewire cable as two pins are used for Voltage which is discarded at 4 pin end. But it seems SONY Handycam does not wants itself to connect it to 6pin socket at PCI card end.
So these are the final things to remember.
1) Use only 4pin to 4pin Firewire cable (any cheaper make. $1.5 will do)
2) Use any cheaper FIREWIRE PCI card with at least one 4pin firewire socket. When installed XP or Vista will install driver automatically just confirm that "IEEE 1394 bus host controller" entry is shown in Device manager or not.
3) No matter you use cheaper or costlier PCI card or cable the quality will always be same coz D8 Handycams directly transfer encoded Video Data in Digital format thru Firewire cable. Firewire cable does not transfer RF video input. I get default video with 720x576 with 25 Frames/s in PAL mode.
4) No additional codecs are required to install in pc like others suggested. Required codecs are always there in XP and VISTA with windows movie maker.
5) No additional software is required, as you can directly capture with Windows Movie Maker in high quality. My choice is Adobe Premier CS3. If you want to make directly MPG file you may use ULEAD VIDEOSTUDIO 8 or later.
6) Do not blame
SONY as they already told you which exact cable to use.
7) Do not blame
Vista. Its nothing to do with it. Sony's PicturePackage for XP does not work on vista. For vista they offered PICTURE MOTION BROWSER which also have easy VIDEO Capture utlity with same high quality capture. Some people falsely claim that "VISTA DOES NOT WORK WELL WITH FIREWIRE" That is completely wrong. See, one or few people may have problem and they always keep shouting for thier fate. But it actually it could be thier local hardware or software problem or anything. There could be many reasons at the same time too. Many people even do not care to change to new cable and keep trying on same thing. Actually cables are very delicate rubber material. Many a times you get faulty cable in new petty pack box of devices. Or it could be some firewire chipset problem too. Last is driver problem. Many firewire card manufacturers have not yet made stable drivers for vista. Most problems are solved if newest drivers are downloaded and installed.
7) In handycam mode, you can control PLAY/STOP/PAUSE of Handycam from PC itself from all the video software you use, which is high convinience for capturing.
8) My sureshot capture sequence is as follows.....
a) Remove BATTERY from camera and switch on with AC Adaptor
b) Put it on PLAY/EDIT mode (also called VCR mode).
c) Connect 4pin to 4pin firewire cable on camera side and in firewire PCI Card. Now you see the words DV-IN on LCD of camera, if LCD is open.
d) Now start any Video Capture Software of your choice. You will find CAPTURE command in FILE menu in almost all softwares.