Every Little Step in the Past Counts