Arnsparger was born in Paris, Kentucky in 1926. He attended Paris High School, and became connected with the school's longtime football and basketball coach, Blanton Collier. The relationship would have a major impact on his future career.
After serving in the United States Marines during World War II, Arnsparger attended Miami University in Oxford, Ohio, where he was a member of Sigma Chi Fraternity (Alpha Chapter). After graduating from Miami in January 1950, Arnsparger remained in Oxford to work as an assistant for the Miami football team during the 1950 season.
On February 21, 1951, Arnsparger was hired by new head coach Woody Hayes of Ohio State University in Columbus, Ohio. He served as the Buckeyes' line coach until 1954.
In 1954, Arnsparger re-connected with Collier, who had been hired as head football coach at the University of Kentucky in Lexington, Kentucky. Arnsparger remained at Kentucky for the next eight years until Collier was fired on January 2, 1962. During the 1959 season, he was joined on the coaching staff by a young coach who had served at the University of Virginia the previous year. That coach was Don Shula, with the two coaches forging a strong bond that would tie them for much of the next quarter century.
Arnsparger moved on to an assistant position with Tulane University in New Orleans, Louisiana in 1962. After two years, he resigned the post on March 6, 1964 to become the defensive line coach for the Baltimore Colts under Shula.
In 1964, Arnsparger became the defensive line coach for the Baltimore Colts. That season, the Colts reached the National Football League (NFL) Championship game and remained one of the strongest teams of the 1960s, competing in Super Bowl III on January 12, 1969.
When Shula left to become head coach with the Miami Dolphins after the end of the 1969 NFL season, he brought along Arnsparger, who was promoted to defensive coordinator. In just two seasons, the formerly moribund team had reached the Super Bowl, with Arnsparger fashioning what became known as the "No-Name Defense." World championships in each of the next two seasons, including an undefeated season during 1972, made Arnsparger a prime candidate for a head coaching position.
Following the Dolphins' 24–7 victory over the Minnesota Vikings in Super Bowl VIII, Arnsparger was named head coach of the New York Giants. With the Giants he managed just seven wins in his thirty-five games. Arnsparger coached the Giants in three different home stadiums during his tenure: the Yale Bowl in New Haven, Connecticut in 1974; Shea Stadium, home of the cross-town rival New York Jets in 1975; and finally, Giants Stadium in 1976. Arnsparger was fired in the middle of the season on October 25, 1976, with the team having lost all seven of its games on the year.
Just two days after his dismissal from the Giants, Arnsparger was rehired by Shula and was restored to his previous position as Miami's defensive coordinator while adding the title of assistant head coach. In the team's first game under his leadership, the Dolphins won a 10–3 defensive battle with the New England Patriots, who had averaged thirty points per game entering the contest.
Miami finished the 1976 NFL season with a 6–8 mark, then narrowly missed a playoff berth the following season. During the next two seasons, the Dolphins reached the postseason, but dropped their first playoff game. During the strike-shortened 1982 NFL season, Miami reached Super Bowl XVII, but dropped a 27–17 decision to the Washington Redskins. Bill Arnsparger again had created an elite defensive unit, known as the Killer B's (so named because of the number of surnames beginning with "B" on the Dolphins defense).
On December 2, 1983, Arnsparger was hired as head coach at Louisiana State University, but finished his season with the Dolphins. As the Tigers' head coach, Arnsparger led LSU to two Sugar Bowl berths in three seasons, in 1984 and 1986. Both times, their opponent was Nebraska. In 1984, LSU finished in a tie for second behind Florida, but the SEC presidents barred Florida from going to the game due to massive rules violations and gave the berth to LSU instead. The 1986 squad was the school's first Southeastern Conference champion since 1970, and the Tigers' last in the pre-championship game era.
Shortly after the final regular season game in 1986, Arnsparger announced he was resigning to become the athletic director at Florida.
At Florida, Arnsparger was tasked with cleaning up an athletic department which had been roiled by NCAA violations and subsequent punishments. The Florida Gators football team had been found in violation of many NCAA rules in the early 1980s and was still suffering under significant sanctions and probation when Arnsparger arrived, a situation which weakened the financial strength of the entire athletic department.
More problems surfaced in 1989. Head football coach Galen Hall was accused of committing minor rules violations, which became a major issue with the NCAA because of the just-completed probation. Hall was forced to resign during the season in October 1989, and after an NCAA investigation, the football program was placed back on probation the following year. At about the same time, the NCAA was investigating allegations that the Florida Gators men's basketball program had paid star players and allowed sports agents to do the same. Less than a month after the head football coach has been forced to resign and less than a month before the start of the 1989-90 basketball season, long-time head basketball coach Norm Sloan was forced to take an early retirement and his entire coaching staff was dismissed. Sloan subsequently claimed that the allegations were false and that Arnsparger's zealous attempts to clean up Florida's athletic department turned into a "witch hunt" that unfairly punished him and his staff. Former Tennessee head coach Don DeVoe was quickly hired as interim coach.
Despite having both of the university's marquee sports placed on NCAA probation in 1990, Arnsparger was able to quickly rebuild Florida's athletic program by establishing better compliance procedures and by hiring new coaches who would bring success while following NCAA guidelines. In December 1989, Arnsparger hired Steve Spurrier, then the Duke University head coach and the Gators' Heisman Trophy-winning quarterback in 1966. Over his twelve years at Florida, Spurrier became the school's all-time wins leader and brought the school its first six conference titles and the 1996 national championship. After a disastrous 1989–90 season under DeVoe, Arnsparger hired Lon Kruger as Florida's basketball coach. Kruger led the Gators to their first Final Four appearance in 1994 and set the stage for later and greater success under Billy Donovan. Florida's other athletic teams also began to improve during Arnsparger's tenure; the Gator baseball team reached the College World Series in 1988 and 1991.
During his time at Florida, Arnsparger was thought by some athletic department staff and boosters to be "domineering" and that, keeping with his background in coaching, he was "inflexible... sticking by his game plan at all costs.". In hindsight, observers gave him credit for setting up the university's athletic department for unprecedented success after his tenure. Jeremy Foley, Arnsparger's successor as athletic director, credited him with "helping to right the ship during a very difficult time at the University of Florida."
On January 13, 1992, Arnsparger resigned to become the defensive coordinator of the San Diego Chargers. During his three seasons with the Chargers, the team's defense showed marked improvement, culminating with a berth in Super Bowl XXIX. Just days after the team's Super Bowl appearance, Arnsparger announced his retirement, citing the prostate cancer surgery he had undergone the year before.
Arnsparger died on July 17, 2015 at his home in Athens, Alabama at the age of 88. He was survived by his wife, his son David, his daughter Mary Susan, and his grandson Christian. He was buried in the Paris Cemetery in Paris, Bourbon County, Kentucky.