The room was called the Withdrawing Room because the men had a Smoking Room and when the ladies had finished their dinner they would go to the Withdrawing Room to talk or knit. They also had tea, sewed, and played music.
