Brazley was born in Louisville, Kentucky. He attended Louisville (KY) Sececa High School and played college football at Western Kentucky University.
He went undrafted in 1980 but played defensive back for the Montreal Alouettes, Ottawa Rough Riders and Toronto Argonauts from 1980 to 1992. He was a CFL All-Star in 1983, the same season he won the Grey Cup with the Argonauts. Brazley also played for the San Diego Chargers as a replacement player in 1987. He ended his 13-year career in 1993, playing with the Toronto Argonauts.
He retired as a five-time CFL All-Star and won two-time Grey Cup Championships, both with the Toronto Argonauts.
He has a wife Jan, a son Nikolas, and a daughter Sunni who all live in Louisville, Kentucky. Nikolas played for the Kentucky as a wide receiver from 2006 to 2011.