In May, 1870, Judge Fisher resigned his place upon the bench, and was appointed by President Grant United States Attorney for the District of Columbia. At the end of five years he resigned this office and returned to his home in Delaware, with no intention of again entering public life. In June, 1889, however, the position of First Auditor of the Treasury was tendered him by President Benjamin Harrison. This position he accepted and held until the change of administration in 1893. He then returned to the home of his childhood, lived quietly in his extensive library, and devoted the last years of his life to reading and literary pursuits. He was one of the most agreeable of men. His mind was so well stored with reminiscence and general information that it was a treat to both old and young to be in his company and listen to his entertaining and instructive conversation. His generous Christian spirit and honesty of purpose endeared him to all who came within the range of his friendship. After a short illness, he died in the City of Washington, February 10, 1899, aged eighty-one years.
In 1840 Mr. Fisher married Eliza A. McColley, who survives him with four children — George P. Fisher, Jr., a lawyer in Chicago, Charles Fisher, Miss Virginia Fisher, and Mrs. Annie Fisher Cahoon. He was a devoted and exemplary husband and father, and has left in each stricken heart the impress of his own pure and useful life.