Chaim Gross attended the city's art academy and studied with painter Béla Uitz, though within a year a new regime under Miklos Horthy took over and attempted to expel all Jews and foreigners from the country. After being deported from Hungary, Gross began art studies at the Kunstgewerbeschule in Vienna shortly before emigrating to New York City in 1921.

In the U.S., Gross's studies continued at the Beaux-Arts Institute of Design, where he studied sculpture with Elie Nadelman and others, and at the Art Students League, with sculptor Robert Laurent. He also attended the Educational Alliance Art School, studying under Abbo Ostrowsky. In the late 1920s and early 1930s Gross exhibited at the Salons of America exhibitions at the Anderson Galleries and, beginning in 1928, at the Whitney Studio Club (the precursor to the Whitney Museum of American Art). In March 1932 Gross had his first solo exhibition at Gallery 144 in New York City.

In 1933, Gross joined the government's PWAP (Public Works of Art Project), which transitioned into the WPA (Works Progress Administration). Under these programs Gross taught and demonstrated art, made sculptures for schools and public colleges, and created works for Federal buildings including the Federal Trade Commission Building, and for the France Overseas and Finnish Buildings at the 1939 New York World's Fair. Gross was also recognized during these years with a silver medal at the 1937 Exposition universelle in Paris. Chaim Gross, Sculptor by Josef Vincent Lombardo, the first major book on Gross, came out in 1949 and included a catalogue raisonne of his sculpture.

In the 1950s Gross began to make more bronze sculptures alongside his wood and stone pieces, and in 1957 and 1959 he traveled to Rome to work with famed bronze foundries including the Nicci foundry. In 1959, a survey of Gross's sculpture in wood, stone, and bronze was featured in the exhibit Four American Expressionists curated by Lloyd Goodrich at the Whitney Museum of American Art, with work by Abraham Rattner, Doris Caesar, and Karl Knaths. In 1963,

In 1974, the Smithsonian American Art Museum held the exhibition, Chaim Gross: Sculpture and Drawings, and in 1976, a selection from Gross's important collection of historic African sculpture, formed since the late 1930s, was exhibited at the Worcester Art Museum in The Sculptor's Eye: The African Art Collection of Mr. and Mrs. Chaim Gross. In 1977, Gross had three retrospective exhibitions: at the Lowe Art Museum at the University of Miami, followed by the Montclair Art Museum; and at the Jewish Musuem in New York City.  In 1984, Gross was inducted into the American Academy of Arts and Letters, with Jacob Lawrence and Lukas Foss.