If you have determined the wiring to be correct, then yes, the remote is probably defective..
According to Lightolier literature, your Lightolier OS600 is called a "strap dimmer". Having 540W on a 600w rated switch is pretty close to the limit (90%) of the switch, and will cause the switch to fail sooner than say a 300w load. The 600w rating of the switch applies only if all the heating sink fins are in place. Often times, when installing in a box with another switch or dimmer, fins must be removed to allow the switch to physically fit. If any fins are removed, the rating of the switch is reduced. Page 30 of the manual
shows that a 600w strap dimmer that has a single fine removed is reduced to a 500w capacity dimmer, and if both fins are removed, it is reduced to a 400w dimmer. You may find that you will need a 1000w dimmer to reliably gain control of your 540w load, as it will control 600w even with both fins removed. This derating is required due to the heat created by the dimmer. Heat is a killer for electronics, and solid state dimmers are no exception.
Check out page 32 of that manual. On it, are wiring diagrams for dimmers and switches. I believe your is figure "G" that shows the 120v "hot" (black) connected to the black wire of ALL switches, the yellow wires of ALL the switches are connected together, but are not connected to anything else, the 120v neutral (white) connects to the MASTER white wire AND light fixture white wire and finally, the MASTER red wire connects to the light fixture black wire.
It is also important to note that these switches are for standard tungsten incandescent and quartz halogen lamps ONLY. They are NOT suitable for CFL, Fluorescent or other types frequently found in commercial / industrial buildings such as neon or other gas lamps such as Metal Halide, Mercury Vapor, High Pressure Sodium, etc.
I hope this helps & good luck!