Ludwig began his career in touring cars with Ford during the Deutsche Rennsport Meisterschaft (DRM) in the 1970s. He won the DRM championship in 1979 and 1981. His 1979 season was highlighted by an outright victory at the 24 Hours of Le Mans, where he drove a Kremer Racing Porsche 935 K3 alongside American brothers Don and Bill Whittington. The win was remarkable: the 935 was a turbocharged car derived from the Porsche 911 road car, and it defeated a field of purpose-built racing prototypes in wet conditions. Ludwig was named as a concurrent DRM champion in 1979.
Ludwig's connection to Le Mans proved exceptional in its depth. After his 1979 win with a production-derived Porsche, he returned to the race with Joest Racing in 1984 and 1985, winning both years in their Porsche 956 No. 7. Three Le Mans victories across three different eras of regulation marked him as one of the rare drivers for whom the Circuit de la Sarthe was consistently productive rather than occasionally fortunate.
Following the deaths of Manfred Winkelhock and Stefan Bellof in 1985, Ludwig turned away from sports car racing and accepted a drive in the 1987 World Touring Car Championship with Ford. He narrowly missed the title, finishing runner-up to BMW's Roberto Ravaglia by a single point after a post-season disqualification. He and Klaus Niedzwiedz had finished second at the Bathurst 1000 in Australia behind teammates Steve Soper and Pierre Dieudonné, but both Ford Sierra RS500s were excluded for non-compliant wheel arch dimensions.
Ludwig moved to the Deutsche Tourenwagen Meisterschaft (DTM) and won the championship in 1988 driving a Ford Sierra RS500. He then repeated the achievement at Mercedes-Benz in 1992 and 1994. His DTM record of three titles placed him among the most decorated competitors in the series' history.
In 1997, Ludwig returned to sports car racing with Mercedes-Benz, and in 1998 he became the FIA GT World Champion. He retired when the series was discontinued after the 1998 season, only to return briefly in June 1999 to win the 24 Hours Nürburgring on the Nordschleife for the third time, driving a Zakspeed Viper.
When the DTM relaunched as the Deutsche Tourenwagen Masters in 2000, Ludwig made another comeback and won a race at the Sachsenring circuit at the age of 50 before retiring again. He subsequently competed as a privateer at the 24 Hours Nürburgring in 2004 and 2005 with Uwe Alzen, and in 2006 helped set a new distance record at the race in a Porsche 997 GT3, finishing second overall.
Ludwig transitioned into broadcasting after retiring from competitive driving, working as a television commentator on DTM races. His record of three Le Mans victories, three DTM/DRM titles, and an FIA GT World Championship establishes him as one of the most complete German drivers of the post-1970s era.