Thomas N. Armstrong III, who greatly expanded the Whitney Museum of American Art’s holdings when he was its director in the 1970s and ’80s but whose ambitious plans for a museum addition aroused a firestorm of opposition that led to his dismissal, died on Monday in Manhattan. He was 78.

The cause was cardiac arrest, his daughter Amory Armstrong Spizzirri said.

Mr. Armstrong was the director of the Pennsylvania Academy of the Fine Arts when he succeeded John I. H. Baur as the director of the Whitney in 1974. A patrician figure with a fondness for bow ties and colorful stunts, Mr. Armstrong set about strengthening the museum’s permanent collection, buying Frank Stella’s 1959 black painting “Die Fahne Hoch!” for $75,000 in 1977 and Jasper Johns’s “Three Flags” for $1 million, a price that seemed extravagant in 1980 and a steal today.

In a whirlwind fund-raising drive in 1982, he raised more than $1.25 million to buy Alexander Calder’s “Circus” (1926-31), an assemblage of more than 50 miniature performers and animals. It had been on loan to the museum but looked as though it might be sold in Europe to help settle the Calder estate’s tax debt.

“These works are pillars of the Whitney’s collection and of American art,” said Adam D. Weinberg, the current director of the Whitney, who once worked under Mr. Armstrong. “He was brilliant at bringing together coalitions of people to acquire artworks, for which we had a minimal acquisition budget. We still have works coming in that he negotiated as gifts years ago.”

“Art in Place,” a 1989 show highlighting the museum’s acquisitions of the previous 15 years, underlined the growth of the permanent collection to 8,500 works from 2,000.

Under Mr. Armstrong’s directorship, the museum had a number of important shows, including a Jasper Johns retrospective in 1978 and large exhibitions of Mark di Suvero, Cy Twombly, Marsden Hartley and Calder.

Desperate to secure additional space for the museum’s collections, he developed plans for a 10-story, $37.5-million addition to the Whitney’s main building.

The proposed addition, designed by Michael Graves and announced in 1985, drew immediate opposition. Neighborhood residents feared a behemoth, and many architects believed it would destroy the integrity of the existing Marcel Breuer building. After Mr. Armstrong gradually lost the support of many of the museum’s trustees, the plans were dropped in 1989, and the next year he was dismissed.

A champion of Andy Warhol’s work — he had organized an exhibition of Warhol portraits at the Whitney in 1979 — Mr. Armstrong became the first director of the Andy Warhol Museum, which opened in Pittsburgh in May 1994.

Nine months later he resigned. It was reported that he was unhappy at how difficult it was to raise money for the museum and that he and Ellsworth M. Brown, the president of the Carnegie Institute, which managed the museum and provided it with financing, could not agree about the museum’s direction.

Thomas Newton Armstrong III was born on July 30, 1932, in Portsmouth, Va., and grew up in Summit, N.J. He painted in high school and earned a bachelor’s degree in art history from Cornell in 1954.

After serving in the Army, he worked for Stone & Webster, an engineering and securities firm, in Manhattan. But determined to make a career in the arts, he began studying museum administration at the New York University Institute of Fine Arts in 1967. A study project at the Abby Aldrich Rockefeller Folk Art Collection in Williamsburg, Va., led to his appointment as a curator at the collection.

In 1971 he was named director of the Pennsylvania Academy of the Fine Arts, where he put into motion the renovation of the main building, a project completed in 1976.

Mr. Armstrong seemed a conservative choice for the Whitney, and on his appointment he expressed a certain diffidence about his credentials as a promoter of recent American art. “I’m not exactly the kind of person who can now be considered as an active participant on the contemporary scene,” he told The New York Times. “But I go to galleries all the time and I used to be a painter myself.”