Jim was born in Leeds Alabama to Mamie Earnest and Clarence Grady Lee. He attended Leeds public schools until high school when he attended boarding school at St. Andrew's School in Sewanee Tennessee, graduating in 1948.

Jim enlisted in the Navy in 1950 upon learning that the G.I. bill would pay for his future education. He served on the destroyers USS Renshaw and USS Radford during the Korean Conflict. After the end of hostilities, aboard the USS Radford, Jim participated in Operation Castle and Operation Ivy, escorting and observing above ground hydrogen bomb tests in the South Pacific.

Following the end of his enlistment in 1954, Jim obtained a BA in English from Middle Tennessee State University, an MA from University of Arkansas and a PhD from Auburn University. Jim began his employment at North Texas State University (now UNT) in 1958, rising through the ranks from Instructor to Assistant Professor, Full Professor and eventually as Chairman of the English Department. Jim retired in 1999 after 41 years at UNT.

Jim was a founder of the Center for Texas Studies at UNT as well an author and editor with the UNT Press, a press specializing in publishing works about the South or by Southern authors. Jim was the editor of "Texas Goes to War", and author of "Texas my Texas", "The Girls of the Golden West" and "A Texas Jubilee" among others. Jim drew upon the colorful characters from his boyhood in rural Leeds, Alabama, fictionalizing and resetting them in his adopted home state of Texas.

Jim was Past President of the Texas Folklore Society, a member of the Texas Institute of Letters and the Texas Literary Hall of Fame and recipient of the A.C. Greene Award, given annually to a distinguished Texas author for lifetime achievement during the West Texas Book Festival.

Upon leaving UNT Jim moved to Ft. Worth, swearing he would never live in Denton again and worked as an Editor for the TCU Press. After the release of his last book, "The Girls of the Golden West" in 2017, Jim led a quiet life in Ft. Worth until 2024 when he abruptly recanted and moved into a retirement home back in Denton.

Jim was just settling into his new home when a bout of pneumonia landed him in the hospital. Despite the best efforts of the staff at Texas Health Presbyterian Denton's ICU, Jim passed away in the early morning hours of July 8th from heart failure.

Jim is survived by his sons Steve and John, his granddaughter Alyssa, his great granddaughters Adeline and Violet and Steve, John and Alyssa's partners Susan, Anne Marie and Zac.

Jim requested there by no service or memorial. He will be interred in the family plot at the Bold Springs Presbyterian Cemetery in Leeds Alabama, alongside his parents.