The pergola, designed in 1908 by Frank Estabrook, was a focal point of the park for many generations. Elaborate formal gardens surrounded the structure until the1960s. This area was fondly remembered for the masses of tulip gardens that bloomed each spring. As city budgets were cut and park funding suspended, the pergola was neglected and left in a deteriorated state. The Downing Park Planning Committee is currently raising money to revitalize this area of the park.

In 1934 local architect Gordon Marvel designed the stone Shelter House on the eastern shore of Polly Pond as a place to change into skates and enjoy cups of hot chocolate during the cold winter months. As was true with the pergola, over time this structure was neglected and abandoned. The rehabilitation of this structure, combined with improvements to the pond and surrounding pathways, was the Downing Park Planning Committee's first revitalization project. The Shelter House is now the City of Newburgh's first Visitors Center and hosts a variety of community meetings and local art displays each month.

Two wood framed greenhouses and a row of garages were built in 1937. Once the source for all the plant material in the park, this area is abandoned and is in the Committee's master plan as a future renovation project.

The outdoor amphitheater was constructed in 1946 and was a site for weekly band concerts for many years. Originally designed with a goldfish filled moat at the front of the stage, this area is the center of our community day events and concert series.