Farmer's first race came at Opa-locka Speedway near Miami, Florida in 1948, when he competed in a 1934 Ford. He eventually settled in Hueytown, Alabama, which he considered his hometown, and became one of the founding figures of the Alabama Gang — the loose fraternity of Alabama-based racers, most famously associated with the Allison family, that became one of the most celebrated groups in American stock car racing history.
Before entering NASCAR's top division, Farmer built his reputation racing modified stock cars in the northeast and was among the earliest drivers to make the transition from the modified series into the early Grand National Series. His career victories, estimated between 700 and 900 in total, were largely accumulated during the late 1950s and early 1960s in local and regional competition, where he dominated short tracks throughout the southeast.
In the NASCAR National Late Model Sportsman Series — the series that later became the Xfinity Series — Farmer won three straight championships from 1969 to 1971, confirming his mastery of the short-track racing that defined his career. He was named NASCAR's most popular driver four times, recognition of the affection fans held for him across decades of competition.
Farmer competed in 36 NASCAR Cup Series races between 1953 and 1975, but deliberately limited his Cup starts, preferring the Late Model Sportsman circuit where he excelled. His best Cup finishes were a pair of fourth-place results: the 1972 Talladega 500 and the 1968 Middle Georgia 500 near Macon, Georgia. He raced the white and gold number 97 car, later also running the number F-97 on a white, gold, and red Ford Fairlane in the mid-1960s.
Beyond his driving, Farmer served as Davey Allison's crew chief in the Busch Series, linking his long career to the next generation of the Alabama Gang.
Farmer's career longevity became a story in itself. He ran two Busch Grand National races in 1992 and competed in the season-opening ARCA race at Daytona in 1993. In June 2005, at age 72, he won a heat race over current NASCAR Sprint Cup drivers at the Sprint Prelude to the Dream at Eldora Speedway and finished eighth in the feature event. As of 2022, Farmer was still regularly competing in a late model at Talladega Short Track, the one-third mile oval dirt track in Eastaboga, Alabama, racing alongside his grandson Lee Burdett.
In March 2021, a tornado damaged his house, race shop, race hauler, and dirt late model, but did not end his involvement in the sport.
On July 12, 1993, Farmer was a passenger in a helicopter being piloted by NASCAR Winston Cup driver Davey Allison at Talladega Superspeedway. Allison was attempting to land the aircraft in a nearby parking lot so the pair could watch David Bonnett — son of Alabama Gang driver Neil Bonnett — practice. The helicopter crashed, killing Allison and leaving Farmer with a broken collarbone and fractured ribs.
Farmer's honors are extensive. He was named one of the 50 Greatest Drivers in NASCAR history in 1998 and inducted into the International Motorsports Hall of Fame, which waived its customary five-year retirement rule because Farmer showed no sign of ever retiring. He was a member of the inaugural class inducted into the Talladega-Texaco Walk of Fame. He is also a member of five halls of fame in total. His induction into the NASCAR Hall of Fame in 2022 brought his legacy formal recognition at the sport's highest institutional level.