They were all negatively affected by Hill House, but I think Luke and Nell were affected most, possibly because they're the youngest. It seems there's a psychic connection between all the siblings since they all reacted to what we presume was Nell's death at 3:03 ET. It may be that they've all inherited different psychic powers/abilities from their mom. Minimally, they all see ghosts. All the children, except Nell, seem to resent their father. One reason for this may be because he hasn't been forthcoming about what happened to their mother that fatal night at Hill House. Also, this reluctance of the father to say anything seems to have contributed somehow to him losing custody of the children to "Aunt Janet," the mother's sister. I think the kids resent this as well. Luke is cold and stiff because Nell is cold and stiff because she's dead. It's a twin thing.
A detail I love about this episode is that in one of Luke's speeches, he talks about cars paaing by watching there headlights hoping one day his mom would show up and at the end when we see his mom her eyes are headlight symbolizing her return.