That assumes that the children stay single as they grow up. If they all get a partner it is no longer one house - one person. You literally only need half as many houses.