James McGarrell was born in Indianapolis, Indiana. As a teenager he had a strong interest in jazz, particularly the music of early New Orleans and Kansas City. After exhausting the books on the subject in the public library, he began to look at art books on the neighboring shelves. At twenty, he dropped out of a pre-law curriculum at Indiana University to take a job at a steel castings plant, and began to draw and paint in the basement of his family home. He returned to the University in Bloomington as an art student, and continued his studies for a summer at the Skowhegan School in Maine, followed by two years of graduate school at UCLA, and a Fulbright grant to Stuttgart, Germany.

The work of James McGarrell has been included in five Whitney Museum Annuals and Biennials, two Carnegie International Exhibitions, Documenta in Kassel Germany and the American Pavilion of the 1968 Venice Biennale. His work has also been exhibited since 1955 in museums and galleries in America and abroad.

In 1995 McGarrell received the Jimmy Ernst Award for lifetime achievement from the American Academy of Arts and Letters. He was awarded The Oscar Williams and Gene Derwood Prize in visual arts from the New York Community Trust in 2008.