In the Henry James source material, there's the possibility that he was smothered by the governess, and it also sounds like "Peter Quint, you devil!" is addressed to her rather than the ghost.
Peter Quint and Miss Jessel represent the awakening homosexuality of the children. This is what the governess identifies as evil and wants to exorcise from the children. But in doing so, she destroys the children's lives at the same time.
