Literature on maternal exposures and the risk of epigenetic changes or diseases in the offspring is growing. Paternal contributions are often not considered. However, some animal and epidemiologic studies on various contaminants, nutrition, and lifestyle-related conditions suggest a paternal influence on the offspring’s future health. The phenotypic outcomes may have been attributed to DNA damage or mutations, but increasing evidence shows that the inheritance of environmentally induced functional changes of the genome, and related disorders, are (also) driven by epigenetic components. In this essay we suggest the existence of epigenetic windows of susceptibility to environmental insults during sperm development. Changes in DNA methylation, histone modification, and non-coding RNAs are viable mechanistic candidates for a non-genetic transfer of paternal environmental information, from maturing germ cell to zygote. Inclusion of paternal factors in future research will ultimately improve the understanding of transgenerational epigenetic plasticity and health-related effects in future generations.