Small populations are often exposed to high inbreeding and mutational load that can increase the risk of extinction. The Sumatran rhinoceros was widespread in Southeast Asia, but is now restricted to small and isolated populations on Sumatra and Borneo, and most likely extinct on the Malay Peninsula. Here, we analyse 5 historical and 16 modern genomes from these populations to investigate the genomic consequences of the recent decline, such as increased inbreeding and mutational load. We find that the Malay Peninsula population experienced increased inbreeding shortly before extirpation, which possibly was accompanied by purging. The populations on Sumatra and Borneo instead show low inbreeding, but high mutational load. The currently small population sizes may thus in the near future lead to inbreeding depression. Moreover, we find little evidence for differences in local adaptation among populations, suggesting that future inbreeding depression could potentially be mitigated by assisted gene flow among populations.