Anthony Mann refused to direct the film, saying nobody would understand it. He also said he believed the script was bad, and Audie Murphy and James Stewart would not be believable as brothers. After the film opened to poor reviews and business, Stewart never spoke to Mann again.
This was originally intended to be the sixth Western combining the talents of James Stewart and Director Anthony Mann, but Mann pulled out of the project because he wasn't impressed with war hero-turned-actor Audie Murphy. Stewart and Mann never made another movie together.
James Stewart wasn't impressed with the script, but accepted the role of Grant McLaine because he believed the part would allow him to show off his skills as an accordion player. However, all of his accordion playing was re-recorded by a professional prior to the movie's release.
Filmed in Silverton, Colorado (called Junction City in the movie) using the Denver and Rio Grande Western Railroad, not the Durango and Silverton Narrow Gauge Railroad as usually stated. The DRGW still ran the Narrow gauge operations at the time this movie was filmed. The Durango and Silerton is a Heritage Railroad that runs on the old rail grade using original equipment.