The movie was based upon L.P. Hartley's novel of the same name. The opening line of the novel has become somewhat well-known: "The past is a foreign country: they do things differently there." That same line--spoken by the voice-over narrator--opens this movie.
Regarding filming in Norfolk, director Joseph Losey said in an interview, "Norfolk helped me a lot because Norfolk hasn't changed. Most of the costumes were genuine. We made very few others, and we all lived in the house. They wore the costumes all the time, and ate, as well as acted in their costumes. Once you've got the exact house, accessories, costumes, something then springs to life."
After the novel's original publication in 1953, author L.P. Hartley sold the movie rights to Alexander Korda, who claimed that he wanted to make a movie with Deborah Kerr in the role of Marian. However, some years went by with no movie emerging, making Hartley convinced that Korda had only bought the movie rights in order to sell them to someone else (at a considerable profit). He told an interviewer years later that "I was so annoyed with Korda that I put a curse on him", rather in the manner of his novel's thirteen-year-old hero, and noted, alarmingly, that Korda had died suddenly the following day.
Deborah Kerr originally was cast as Mrs. Maudsley but withdrew after director Joseph Losey left her waiting several hours in a hotel for his return call. Margaret Leighton eventually got the part.