Sleep is a crucial factor in healthy aging. However, most middle-aged adults experience high levels of sleep disorders. While previous findings have suggested exercise training could benefit the quality of sleep, the effects of multi-component exercise on sleep quality are less examined. Accordingly, the current study aimed to assess the effectiveness of a multi-component exercise program on the quality of sleep among middle-aged adults. Twenty-four middle-aged adults were randomly assigned either to a multi-component exercise (MCE) group or a control group. The participants in the MCE group attended a 90-min session per week for 12 weeks. The control group was instructed to maintain their daily routine for 12 weeks. The primary outcome was the sleep quality evaluated by the Pittsburgh Sleep Quality Index (PSQI). The secondary outcome was physical fitness, including muscular strength and endurance, balance, and flexibility. Regarding sleep quality, the global mean score (p = 028), sleep disturbances (p = 011), and sleep efficiency (p = 035) of the PSQI scores were significantly reduced in the MCE group after the 12-week intervention. Regarding physical fitness, the flexibility of the MCE group improved significantly after the intervention (p = 028), yet, no significant change was observed in the control group. Additionally, the muscular strength of the control group declined significantly after the 12-week period (p = 034). Our results revealed the effectiveness of the MCE intervention in improving sleep quality and physical fitness in middle-aged adults. Further studies using larger sample sizes, objective measures of sleep quality, different types of exercise training, as well as different populations, are warranted to extend our current findings.