First off one of the "vacation addresses" has to appear in the 'To' or 'Cc' lines in the message, to trigger a reply.

Next the message has to come from an address which has not sent to you in 'n' days of the vacation script running --- every time a response is sent, the address is saved, and not replied to again for the 'n' days you set, so only new correspondants get sent the message

Finally you have to turn on scripts under the advanced menu before it will run