As the island's institutions became abandoned toward the end of the 19th century, the population declined. Goldwater and Coler Hospitals were established and shortly thereafter became the only entities remaining on the island. It was desolate to the point that the New York City Fire Department was using the empty structures as training sites.
In 1968, Mayor John Lindsay appointed a committee for redevelopment of Welfare Island. The committee published a plan which was later incorporated into the General Development Plan produced by Governor Nelson Rockefeller's New York State Urban Development Corporation (UDC), which he formed in 1969.
The UDC then commissioned a development design for the island which was completed by Philip Johnson and John Burgee and exhibited at the Metropolitan Museum of Art. In a ceremony in 1973, Governor Rockefeller and Mayor Lindsay were present for the renaming of Welfare Island in honor of President Franklin D. Roosevelt. At the ceremony, architect Louis Kahn revealed his design for the memorial park to be constructed on the island's Southpoint.