Other than being used as the residence for the governor, it was later used by "other governors, government agents, and other officials, including superintendents of pearl fishery. For more details https://victorysgardens.com/