Hi EAyres
Welcome 
You have a family that sounds very like mine ... altho being older my parents are deceased it's my ex-in-laws that only either appear fleetingly at Christmas or stick a card in the post.
For what it's worth I say do what you want, be firm with them and tell them that you are having a quiet Christmas on your own with your immediate family, this is how you want end of story!!
You are absolutely right to do what is best for you and your family especially given that they dont bother any other time, your son doesnt know them, etc.
Perhaps if they are prepared to meet up for an outdoor picnic in a park in the summer to begin the process of getting to know each other (especially the children) then that makes for a better environment for your son where he is not in an enclosed space with lots of people/noise etc and you can just pack up and go when things are too much for him. If they dont like it ... that's their problem.
Oh wait a minute, I think I am getting confused with your post. When you speak about "dad" you dont mean your dad, you mean your son's dad ... is that right?
Another alternative is to turn it round and say that you will visit them individually and that way you can go to each house so there are less bodies around and you can leave if it's too much for your son.
Bottom line is you do what's best for you and your son and they will just have to accept it. You could perhaps pick up son leaflets on autism or a book out of the library for his dad to read and get a better understanding of it so that he can educate his family. You could even get leaflets for each member of his family and pass the to them. If they are on the internet send them a link to the http://www.autism.org.uk/ website so they can find out more themselves. That leaves the ball in their court to get an understanding of it and how it impacts on your son.
Josie - Community Champion