James J. Patterson was born in Philadelphia on June 22, 1838, the son of John and Ellen Van Dyke Patterson. He grew up in rural Juniata County near Academia where his family had taken up farming and local business. He attended local schools and the Tuscarora Academy, the first secondary school in the county, a Presbyterian institution in Academia for which his father had donated land and money. He entered Dickinson College in Carlisle, Pennsylvania in September, 1856 with the class of 1859, enrolling in the classical course. While at the College, he was an early member of Phi Kappa Sigma and active in the Belles Lettres Society. Following graduation with his class, he took up the post of principal of Boalsburg Academy in Centre County, Pennsylvania.
After three years as an academy principal, in August, 1862, Patterson answered Lincoln's call for volunteers and joined the company Captain Robert McFarlane was recruiting in the county. He was commissioned and was second in command in what became Company G of the 148th Pennsylvania Volunteers. McFarlane soon went on to higher command and Patterson took over as company commander. After training at Camp Curtin, the 148th became part of Hancock's First Division. The unit experienced its first engagement at Chancellorsville in the spring of 1863 and then at Gettysburg, where the company fought in the Wheat Field and on Cemetery Ridge. Action at Spotsylvania, Cold Harbor, and at Petersburg, where Patterson suffered a leg wound, followed. His injury ultimately resulted in his medical discharge in December, 1864, after a period at the mobilization center in Carlisle. He was a merchant for a time and then returned to academy teaching. Patterson served twice as principal of his old school in Juniata County, between 1869 and 1873 and again between 1879 and 1880. He was a principal in Mifflintown and then, in 1887 relocated to Milford, Nebraska, where he ran schools until 1894. In his later years he was a businessman in Arkansas, and lived in retirement in Alpena Pass, Arkansas.
Patterson married Elizabeth Jack of Boalsburg on December 18, 1863 while on leave from the army. The couple had nine children. He gained the distinction of being the oldest living graduate of the college and traveled to visit Carlisle as often as he could. He was awarded an honorary degree in 1932. His stories of Civil War era Dickinson were much in demand. Patterson noted, for example, that Samuel Beck, class of 1859 from Maryland, was unrestrained in his support of the South and that only one Southerner in his class, Duke Slavens, could discuss the matters of the day dispassionately. He also observed from his later time stationed in Carlisle that the townspeople were far happier with Herman Merrills Johnson as president than they had been with Charles Collins, who they considered a southern sympathizer. James John Patterson died at the Fitzsimons Hospital in Denver, Colorado on January 3, 1934 while visiting one of his daughters. He was ninety-five years old.