Bullitt was born to William Christian Bullitt and Mildred Ann Fry. He was named after Joshua Fry, a notable figure in American colonial history and one of Mildred's ancestors. He was the oldest of ten children, though only six lived past age 25. Among his siblings were John Christian Bullitt, Susan Peachy Bullitt (who would later marry Senator Archibald Dixon) and Thomas Walker Bullitt, father of William Marshall Bullitt.
Bullitt attended Centre College, then studied law at the University of Virginia.
Bullitt was admitted to the bar in 1844.
He served as a state representative from Louisville beginning in 1851. In 1861 he was elected to the Court of Appeals (Kentucky's highest court at the time). He served as Chief Justice in 1864 and 1865.
Bullitt was arrested in July 1864 at the direction of General William Tecumseh Sherman for sympathizing with the Confederacy. He was sent to Tennessee with others suspected of similar crimes, possibly to be sent to the lines. He was returned to Louisville and the Court of Appeals on December 6, 1864.
General Stephen G. Burbridge stated that Bullitt's release was part of an exchange of prisoners between Confederate General Nathan Bedford Forrest and Federal General Washburne, and that Bullitt "was liable to re-arrest, ought to have been re-arrested and hung, and would have been arrested had he not escaped."
Bullitt fled to Canada and was removed from office in June 1865.
Bullitt was married to Elizabeth B. Smith in 1846. Among their children was Joshua Fry Bullitt, Jr. (born 1856), who would also become a notable lawyer and political figure.