So, if you notice he's going back in the house to see the memories, then either he burns the house or it burns on it's own. He goes in with a flashlight and you see the flashlight light where they were dancing explaining that it was a memory he was rewatching.