In the 1960s, former staunch segregationist Ivan Allen, Jr. became a moderate on racial issues and began to work quietly to help end segregation. Soon after becoming head of the Atlanta Chamber of Commerce in 1960, Allen led negotiations between white merchants and black civil rights activists protesting segregated businesses in Atlanta. On his first day in office as mayor of Atlanta (1962-1970), Allen ordered the removal of all 'colored' and 'white' signs in City Hall. He personally ended Jim Crow practices in the City Hall cafeteria when he sat down at a table with a black attorney. In December 1962, Mayor Allen shocked his African-American supporters when he yielded to pressure from white constituents and ordered the erection of a barricade to block integration of a white, middle class neighborhood. In the wake of national criticism, he ordered the wall removed. Then, in July 1963, Mayor Allen made amends for "the Atlanta Wall" fiasco when he risked his political career to support President John Kennedy's efforts to end segregation throughout the nation. Allen testified in favor of the Civil Rights bill before a Senate Committee in Washington, D.C. He was the only southern elected official to endorse the bill, which became the 1964 Civil Rights Act. During Allen's administration, Atlanta was dubbed "the City Too Busy To Hate." In the summer of 1966, Allen tried to live up to that image by going into the black inner city community of Summerhill to try to calm racial tension. In 1981, Coretta Scott awarded Ivan Allen, Jr. the Martin Luther King, Jr. Nonviolent Peace Prize.

Ivan Allen, Jr., grew up as the only son of businessman Ivan Allen, Sr. and Irene Beaumont Allen. He graduated from all-white Boys High School in Atlanta, majored in business administration at the Georgia Institute of Technology (1929-1933), and married Louise Richardson in 1936. The couple had three sons.

After serving in the Army during World War II, Allen briefly served as executive secretary to Georgia Governor Ellis. In 1946, Allen took over the family's business, the Ivan Allen Company, a successful office supply firm.