Inbreeding threatens the survival of small populations by producing inbreeding depression, but also exposes recessive deleterious effects in homozygosis allowing for genetic purging. Using inbreeding-purging theory, we analyze early survival in four pedigreed captive breeding programs of endangered ungulates where population growth was prioritized so that most adult females were allowed to contribute offspring according to their fitness. We find evidence that purging can substantially reduce inbreeding depression in Gazella cuvieri (with effective population size $N_e$ = 14) and Nanger dama ($N_e$ = 11). No purging is detected in Ammotragus lervia ($N_e$ = 4), in agreement with the notion that drift overcomes purging under fast inbreeding, nor in G. dorcas ($N_e$ = 39) where, due to the larger population size, purging is slower and detection is expected to require more generations. Thus, although smaller populations are always expected to show smaller fitness (as well as less adaptive potential) than larger ones due to higher homozygosis and deleterious fixation, our results show that a substantial fraction of their inbreeding load and inbreeding depression can be purged when breeding contributions are governed by natural selection. Since management strategies intended to maximize the ratio from the effective to the actual population size tend to reduce purging, the search for a compromise between these strategies and purging could be beneficial in the long term. This could be achieved either by allowing some level of random mating and some role of natural selection in determining breeding contributions, or by undertaking reintroductions into the wild at the earliest opportunity.