Intensive land expansion of commercial oil palm agricultural lands results in reducing the size of peat swamp forests, particularly in Southeast Asia. The effect of this land conversion on macrofungal biodiversity is, however, understudied. We quantified macrofungal biodiversity by identifying mushroom sporocarps throughout four different habitats; logged peat swamp forest, large-scale oil palm plantation, monoculture, and polyculture smallholdings. We recorded a total of 757 clusters of macrofungi belonging to 127 morphospecies and found that substrates for growing macrofungi were abundant in peat swamp forest; hence, morphospecies richness and macrofungal clusters were significantly greater in logged peat swamp forest than converted oil palm agriculture lands. Environmental factors that influence macrofungi in logged peat swamp forests such as air temperature, humidity, wind speed, soil pH, and soil moisture were different from those in oil palm plantations and smallholdings. We conclude that peat swamp forests are irreplaceable with respect to macrofungal biodiversity. They host much greater macrofungal biodiversity than any of the oil palm agricultural lands. It is imperative that further expansion of oil palm plantation into remaining peat swamp forests should be prohibited in palm oil producing countries. These results imply that macrofungal distribution reflects changes in microclimate between habitats and reduced macrofungal biodiversity may adversely affect decomposition in human-modified landscapes.