As an architect, Gene Kaufman doesn’t typically save buildings; he designs them.

But when he heard of plans to change Paul Rudolph’s celebrated but shuttered government building in Goshen, N.Y., as part of a renovation plan, he decided to step in.

“To lose a building like this would be a tragedy,” said Mr. Kaufman, a partner at Gwathmey Siegel Kaufman Architects in New York City.

He has offered to buy and restore the 1967 building, which architecture experts hail as a prime example of raw Brutalist style and others consider an eyesore in a town known for its historic harness-racing track and Greek Revival, Federal and Victorian houses.