Pre-Service Teachers': Analysis and Constraints in Making HOTS-Based Multiple Choice Questions

Berti Yolida; Rini Rita T Marpaung; Dewi Lengkana

doi:10.57142/jesd.v2i1.59

Authors

Berti Yolida Doctor of Education Study Program, Faculty of Teacher Training and Education, Lampung University, Bandar Lampung, Indonesia
Rini Rita T Marpaung Biology Education Study Program, Faculty of Teacher Training and Education, Lampung University
Dewi Lengkana Biology Education Study Program, Faculty of Teacher Training and Education, Lampung University

DOI:

https://doi.org/10.57142/jesd.v2i1.59

Keywords:

analysis questions, constraints, HOTS-based multiple-choice, Pre-service teachers

Abstract

Students in pre-service teacher education have difficulty formulating questions, particularly HOTS questions. The purpose of this study was to describe the distribution of questions, the percentage of HOTS (Higher Order Thinking Skills) questions, and the obstacles faced by pre-service teachers in creating HOTS-based questions. The research method uses qualitative methods with descriptive data presentation. The research was conducted at the Biology Education Study Program with a sample of 2020 pre-service teachers, with 57 pre-service teachers selected by purposive sampling. The data source is multiple-choice questions made by pre-service teachers and tested on students in pilot schools in Lampung Province, Indonesia. The results of the study showed that 72.46% of the questions were in the valid category and 18.84% of the HOTS questions were in the low category. The obstacles found were difficulties in compiling questions, especially HOTS questions, difficulties in making homogeneous alternative answers, and distractors.

References

Adom, D., Adu-Mensah, J., & Dake, D. A. (2020). Test, measurement, and evaluation: Understanding and use of the concepts in education. International Journal of Evaluation and Research in Education (IJERE), 9(1), 109. https://doi.org/10.11591/ijere.v9i1.20457

Aiken, L. R. (1985). Three Coefficients for Analyzing the Reliability and Validity of Ratings. Educational and Psychological Measurement, 45(1), 131–142. https://doi.org/10.1177/0013164485451012

Akbar, A. (2020). Student Ability in Preparation of Multiple Choice Problems (in Indonesian). Attadib: Journal of Elementary Education, 4(1), Article 1. https://doi.org/10.32507/attadib.v4i1.629

Alrawili, K. S., Osman, K., & Almuntasheri, S. (2020). Effect of scaffolding strategies on higher-order thinking skills in science classroom. Journal of Baltic Science Education, 19(5), 718–729. Scopus. https://doi.org/10.33225/jbse/20.19.718

Appelhaus, S., Werner, S., Grosse, P., & Kämmer, J. E. (2023). Feedback, fairness, and validity: Effects of disclosing and reusing multiple-choice questions in medical schools. Medical Education Online, 28(1), 2143298. https://doi.org/10.1080/10872981.2022.2143298

Barnett, J. E., & Francis, A. L. (2012). Using higher order thinking questions to foster critical thinking: A classroom study. Educational Psychology, 32(2), 201–211. https://doi.org/10.1080/01443410.2011.638619

Bassachs, M., Cañabate, D., Nogué, L., Serra, T., Bubnys, R., & Colomer, J. (2020). Fostering Critical Reflection in Primary Education through STEAM Approaches. Education Sciences, 10(12), 384. https://doi.org/10.3390/educsci10120384

Binti Misrom, N. S., Muhammad, A. S., Abdullah, A. H., Osman, S., Hamzah, M. H., & Fauzan, A. (2020). Enhancing Students’ Higher-Order Thinking Skills (HOTS) Through an Inductive Reasoning Strategy Using Geogebra. International Journal of Emerging Technologies in Learning (IJET), 15(03), 156. https://doi.org/10.3991/ijet.v15i03.9839

Brookhart, S. M. (2010). How to Assess Higher-Order Thinking Skills in Your Classroom. ASCD publication Alexandria, Virginia USA.

Capan Melser, M., Steiner-Hofbauer, V., Lilaj, B., Agis, H., Knaus, A., & Holzinger, A. (2020). Knowledge, application and how about competence? Qualitative assessment of multiple-choice questions for dental students. Medical Education Online, 25(1), 1714199. https://doi.org/10.1080/10872981.2020.1714199

Carter, J. (1994). Book Reviews: Evaluation Thesaurus (4th ed.), by Michael Scriven. Newbury Park: Sage Publications, 1991, 391 pp. Evaluation Practice, 15(1), 109–110. https://doi.org/10.1177/109821409401500117

Considine, J., Botti, M., & Thomas, S. (2005). Design, format, validity and reliability of multiple choice questions for use in nursing research and education. Collegian, 12(1), 19–24. https://doi.org/10.1016/S1322-7696(08)60478-3

Erdoğan, E., & Çargıt, B. (2018). Comparison of Pre-service Teachers’ Social Skill Levels. Universal Journal of Educational Research, 6(12), 2907–2912. https://doi.org/10.13189/ujer.2018.061225

Fatimah, L. U., & Alfath, K. (2019). Analysis of Difficulty Problems, Differential Powers and Functions of Distractors (in Indonesian). Al-Manar, 8(2), 37–64. https://doi.org/10.36668/jal.v8i2.115

Gettings, M. (2016). Putting It All Together: STEAM, PBL, Scientific Method, and the Studio Habits of Mind. Art Education, 69(4), 10–11. https://doi.org/10.1080/00043125.2016.1176472

Haladyna, T. M. (2004). Developing and validating multiple-choice test items (3rd ed). Lawrence Erlbaum Associates.

Iñarrairaegui, M., Fernández-Ros, N., Lucena, F., Landecho, M. F., García, N., Quiroga, J., & Herrero, J. I. (2022). Evaluation of the quality of multiple-choice questions according to the students’ academic level. BMC Medical Education, 22(1), 779. https://doi.org/10.1186/s12909-022-03844-3

Kulgemeyer, C., Borowski, A., Buschhüter, D., Enkrott, P., Kempin, M., Reinhold, P., Riese, J., Schecker, H., Schröder, J., & Vogelsang, C. (2020). Professional knowledge affects ACTION‐RELATED skills: The development of PRESERVICE physics teachers’ explaining skills during a field experience. Journal of Research in Science Teaching, 57(10), 1554–1582. https://doi.org/10.1002/tea.21632

Meryansumayeka, Putri, R. I. I., Zulkardi, & Hiltrimartin, C. (2020). Secondary students’ higher-order thinking skills in solving PISA- like mathematical tasks. Journal of Physics: Conference Series, 1480(1), 012034. https://doi.org/10.1088/1742-6596/1480/1/012034

Opstad, L. (2021). Can Multiple-Choice Questions Replace Constructed Response Test as an Exam Form in Business Courses? Evidence from a Business School. ATHENS JOURNAL OF EDUCATION, 8(4), 349–360. https://doi.org/10.30958/aje.8-4-1

Osterlind, S. J. (1998). Constructing test items: Multiple-choice, constructed-response, performance, and other formats (2nd ed). Kluwer Academic Publishers.

Purwasih, J. H. G. (2020). Obstacles of Prospective Educators in Making Multiple Choice Questions Higher Order Thinking (HOT) (In Indonesian). Jurnal Sosial Humaniora (JSH), 13.

Puthiaparampil, T., & Rahman, M. M. (2020). Very short answer questions: A viable alternative to multiple choice questions. BMC Medical Education, 20(1), 141. https://doi.org/10.1186/s12909-020-02057-w

Ramadhan, S., Mardapi, D., Kun, Z., & Budi, H. (2019). The Development of an Instrument to Measure the Higher Order Thinking Skill in Physics. European Journal of Educational Research, 8(3), 743–751. https://doi.org/10.12973/eu-jer.8.3.743

Rosidin, U. (2017). Learning Evaluation and Assessment (In Indonesian). Media Akademi.

Tican, C., & Deniz, S. (2019). Pre-service Teachers’ Opinions about the Use of 21st Century Learner and 21st Century Teacher Skills. European Journal of Educational Research, 8(1), 181–197. https://doi.org/10.12973/eu-jer.8.1.181

Winarti, H. (2021). Description of the Teacher’s Ability to Make Problems Based on the 2013 Curriculum at High School in Landak District (In Indonesian). https://doi.org/10.5281/ZENODO.4659018