This paper empirically tested eight key features of WHO guidelines to age-friendly community by surveying 211 informal caregivers and 402 self-care adults (aged 45 to 85 and above) in Malaysia. We examined the associations of these eight features with active aging and social connectedness through exploratory and confirmatory factor analyses. A structural model with satisfactory goodness-of-fit indices (CMIN/df = 1.11, RMSEA = 0.02, NFI = 0.97, TLI = 1.00, CFI = 1.00, and GFI = 0.96) indicates that transportation and housing, community support and health services, and outdoor spaces and buildings are statistically significant in creating an age-friendly environment. We found a statistically significant positive relationship between an age-friendly environment and active aging. This relationship is mediated by social connectedness. The results indicate that built environments such as accessible public transportations and housing, affordable and accessible healthcare services, and elderly friendly outdoor spaces and buildings have to be put into place before social environment in building an age-friendly environment. Otherwise, the structural barriers would hinder social interactions for the aged. The removal of the environmental barriers and improved public transportation services provide short-term solutions to meet the varied and growing needs of the older population.