Governors Island architecture dates back to 1806; its two forts were so formidable, the British avoided New York City in the War of 1812.