Students' evaluations of teaching staff can be considered high-stakes, as they are often used to determine promotion, reappointment, and merit pay to academics. Using Facets, the reliability and validity of one student rating questionnaire is analyzed. A total of 13,940 respondents of the Human Science Division of International Islamic University Malaysia were involved in the study. The analysis shows that the student rating questionnaire used was valid and reliable, and it allows identification of staff for the institution's prestigious teaching excellence awards, and those needing in-service training. It was found that there was no significant difference in terms of rank, age and gender of the staff. The study also shows that the majority of staff have problems keeping the class interested and getting students to participate in class activities. Faculty also hardly discussed common errors in assignments and tests.
The importance of instilling leadership skills in students has always been a main subject of discussion in Malaysia. Malaysian Secondary School Students Leadership Inventory (M3SLI) is an instrument which has been piloted tested in year 2013. The main purpose of this study is to examine and optimize the functioning of the rating scale categories in M3SLI by investigating the rating scale category counts, average and expected rating scale category measures, and steps calibrations. In detail, the study was aimed to (1) identify whether the five-point rating scale was functioning as intended and (2) review the effect of a rating scale category revision on the psychometric characteristics of M3SLI. The study was carried out on students aged between 13 to 18 years (2183 students) by stratified random sampling in 26 public schools in Sabah, Malaysia, with the results analysed using Winsteps. This study found that the rating scale of Personality and Values constructs needed to be modified while the scale for Leadership Skills was maintained. For future studies, other aspects of psychometric properties like differential item functioning (DIF) based on demographic variables such as gender, school locations and forms should be researched on prior to the use of the instrument.
This paper describes the process of assessing the unidimensionality and validity of egalitarian education (EE) items based on the Rasch measurement model. Egalitarian education was measured by a self-developed 5 EE items of Likert-scale format. The process of assessing the validity of EE items involved a collection of data from 400 Malay teachers, who are teaching in government school around peninsular of Malaysia where the measurement of construct validity for the overall EE items were established using Winsteps. Various Rasch measurement tools were utilized to demonstrate the true unidimensionality and validity measure of the EE items and in meeting the needs of the Rasch measurement model. The findings show that the validity and unidimensionality of EE items can be truly established and can satisfy the characteristics of the Rasch measurement model.
The fairness of raters in music performance assessment has become an important concern in the field of music. The assessment of students' music performance depends in a fundamental way on rater judgements. The quality of rater judgements is crucial to provide fair, meaningful and informative assessments of music performance. There are many external factors that can influence the quality of rater judgements. Previous research has used different measurement models to examine the quality of rater judgements (e.g., generalizability theory). There are limitations with the previous analysis methods that are based on classical test theory and its extensions. In this study, we use modern measurement theory (Rasch measurement theory) to examine the quality of rater judgements. The many-facets Rasch rating scale model is employed to investigate the extent of rater-invariant measurement in the context of music performance assessments related to university degrees in Malaysia (159 students rated by 24 raters). We examine the rating scale structure, the severity levels of the raters, and the judged difficulty of the items. We also examine the interaction effects across musical instrument subgroups (keyboard, strings, woodwinds, brass, percussions and vocal). The results suggest that there were differences in severity levels among the raters. The results of this study also suggest that raters had different severity levels when rating different musical instrument subgroups. The implications for research, theory and practice in the assessment of music performance are included in this paper.
Adolescences is an important transitional phase in human development where they experience physiological as well as psychological changes. Nevertheless, these changes are often understood by teachers, parents, and even the adolescents themselves. Thus, conflicts exist and adolescents are affected from the conflict physically and emotionally. An important state of emotions that result from this conflict is anger. This article describes the development and validation of the 34-item Adolescent Anger Inventory (AAI) to measure types of anger among Malaysian adolescents. A sample of 2,834 adolescents in secondary school who provide responses that were analyzed using Rasch model measurement framework. The 4 response category worked satisfactorily for the scale developed. A total of 11 items did not fit to the model's expectations, and thus dropped from the final scale. The scale also demonstrated satisfactory reliability and separation evidence. Also, items in the AAI depicted no evidence of DIF between 14- and 16-year-old adolescents. Nevertheless, the AAI did not have sufficient items to target adolescents with a high level of physical aggressive anger.