Testosterone exposure during critical periods of development exerts major organizing effects on the hypothalamic-pituitary adrenal (HPA) axis. Here we examined how neonatal gonadectomy (GDX) with or without testosterone treatment during the first week of life alters the HPA response to adult testosterone replacement in 65 day-old male rats. As adults, neonatal GDX rats showed higher levels of plasma corticosterone and Fos activation in medial parvocellular part of the paraventricular (PVN) of the hypothalamus under basal conditions and during 30 min of restraint exposure. These responses were normalized with testosterone treatment on postnatal days 1–5, but were not restored with adult testosterone replacement. As adults, neonatal GDX rats also showed a decrease in the number of androgen receptor and arginine vasopressin (AVP) positive cells in the bed nucleus of the stria terminalis and in the medial nucleus of the amygdala, and both of these responses were reversed with postnatal testosterone treatment. In stressed and unstressed animals, the number of androgen receptors and AVP expressing neurons in both of these nuclei correlated negatively with corticosterone concentrations in plasma and Fos levels in the PVN. Taken together, our findings suggest that testosterone exposure during the neonatal period primes the adult HPA response to testosterone by altering androgen receptor levels and function within afferent mediators of basal and stress-related input to the HPA axis.