Close ended-tests in Colombia

This entry is related to the Geneese and Upshur reading on Choosing and devising tasks (1996, ps.176-196)

Regarding close-ended tests, we can refer a plethora of examples in our Colombian context: all of the ICFES exams, English official tests, psychological tests, questionnaires and surveys. They are supposed to quantitatively and partially measure or to have a sample of our knowledge  or opinion in different areas or our psyche state, respectively,  and they are useful to achieve some goals: the first exams are appropriate to do surveys  for national and international statistical ranking, to make decisions on educational planning, procedure and performance to help students improve in their learning process, to permit students ascend in the academic realm: elementary, secondary, undergraduate, graduate education and career’s performance. The second one is intended to generally measure your English skills and, thus, your success on these exams will contribute to make your dreams come true:  get a well-paid job, get a promotion in your job or field, get meaningful revenues, show-off with your English certificate, etc.  The purpose of psychological tests is not to find out what level of schizophrenia you have reached, but to see how good enough you are at handling situations, controlling your emotions and acting wisely and it works to get into jobs, master degrees and other fields. Questionnaires and surveys are used to collect data for any quantitative or qualitative research, investigation, and inquiry for any field from the most complex sciences to the lousiest demographic opinion surveys for settling down businesses nearby areas with lots of audience.

All these examinational purposes in theory are of an entire magnificence, but in practice they may end up biased, distorted or in the oblivion because our government, policy makers (most of them ignoring pedagogical issues) and their educational policies have two aims: to be gatekeepers or to establish a gate so only a certain elite can access to a higher and good education, making people with low incomes  far away from educational system or have more difficulties to access (not only due to this policy but economic, political and cultural problems) or setting apart  most vulnerable teachers from the governmental teaching staff, by administering an extremely difficult,  generic and out-of-place exams and, this way, the most flatterer influencing teachers can get a room in that staff. In sum, the aim is to use these tests as means of social and political control (Lopez & Bernal).  The second aim is to propose those nonsensical exams as obligatory requirement to get graduated or  to get a job or a promotion.  This is the case of Pruebas Saber 11°, Pruebas Saber Pro and Pruebas del Concurso Docente, all of them invented by ICFES that I have to take to access undergraduate education, to get graduated and to compete for a job in the governmental teaching staff.  

Let’s start talking about pruebas saber 11°. All of us know that this standardized exam is to partially measure the knowledge we acquired during High School, to make surveys about secondary education and to access the university life. We know that close-ended test are time-consuming regarding  purposeful and meaningful readings for eliciting   stems or questions and  effective alternative responses or distractors, but when scoring and giving results, this ends up more practical.  The main competence evaluated is reading comprehension, and this type of test is suitable for the purpose. This test control the precise way of the particular response wanted by test makers and assess a particular aspect of each subject. In theory, this exam should be appropriate, understandable and feasible for a current 11^th grader related to instructional objectives and instructional activities. Moreover, this test may fulfill several points counting on some traits of assessment principle as practicality, validity, authenticity. Also, it might fulfill some of the guidelines for close-ended tasks in both stems and distractors, since stems were related to the content, with no double negatives, inadvertent cues or verbatim repetition, and assessed what was supposed.  Regarding distractors in general, they were equally hard, attractive and plausible, grammatically and semantically alike and grammatical compatible with stems and related to the readings. And there were some multiple choice questions with several correct answers, some answers were derived from other questioning items, which made test somewhat difficult but still manageable.

However some problems arise when regarding reliability and washback effects on students. The test is unreliable due to several factors.  Test makers take for granted that all students know to handle a hundred- questioned folding exam and answer it on a response sheet, by just spotting a circle with a specific kind of pencil. In my experience, it was not that easy, people just had troubles unfolding the exam (a real mayhem) and some others got confused with the answer sheet because this was unfamiliar at school.  As far as we have discussed, test takers emotional states really counts because test takers can do bad at this exam if they are sick, have family problems, did not have breakfast, get nervous and get blocked,  and so on. The test administration was not so good, because indications and time constraints were so stressing: hundreds of questions in 2 four-hour sessions only a day.  Now, the issue get more complicated when referring results, success to access undergraduate programs and schools rankings. This exam caused a very negative washback effect for both students and schools. My very former classmates got frustrated and mad when getting the results; most of them got discouraged or feel bad or illiterate, did not even want to think about university studies but finish their High School and start working because neither they could  reach the score to access state universities nor their parents would  be able to afford their undergraduates programs in private universities. As for ranking schools based on results, it has always ended up unfair because the most well-known and wealthy schools prepare their students since Kinder Garden with multiple choice test  for every single subject, even for the simplest decision making of what place to address at school at certain moments. In eleventh grade, directors and teachers choose the most accurate students to take the exam under the name of the school and make the others take it particularly. In exchange, most public schools haven’t settled these politics into their curriculum, since its aim is not to prepare students for exams but learn for life (in classes, just some go for memorization and banking education though), and all students must take it compulsorily.  It is evident these public schools don’t get a well-ranked and its students don’t get good results; just few exceptional cases get good scores and can access a higher education. 

Regarding ECAES exam or pruebas saber pro, it is just a requirement to get the undergraduate diploma whether you success or fail and an excuse to make money. I don’t see any other purpose of this exam, since it affects all the principles of assessment on test takers’ performance, except practicality. This exam is still practical because it is a close-ended task with multiple choice of all the competence and has an only open-ended question with prompt sentences to make an essay on a topic related to our field, in my case, Spanish Teaching or just pedagogy. It may also fulfill the guidelines for close-ended tasks on stems and distractors. But it was so practical as generic and nonsensical for many people from different fields of study. The close-ended task consisted of three domains: statistical, psychological and reading comprehending.  The second and the third parts were nicely presented but it was an insult for a Spanish Teacher that has read lots of literature and pedagogical books because the readings, the stems and the distractors were somehow intended for scholars. On the contrary, the statistical part was so hard and out of context because most of the taste takers studies belong to human sciences and we got nothing to do with numbers for decades. Moreover, the results caused a very negative washback on my partners, owing to their ignorance on maths and English; there was a friend of mine that got a superficial level on reading comprehension, which caused him a severe emotional trauma, he could not conceive, jumping into conclusions, that he was not good enough at reading if he was actually an acute literature reader.

Finally,  concurso docente’s exam just sucked because it was impractical, unreliable, non-valid, unauthentic and have terrible washback effects and sociopolitical consequences. Regarding practicality, although the test cost little money and was well-structured as a good close-ended test, it was impractical because there were a multitude taking this test due to the two-year test postponement by the government.  Time constraint was a paramount factor to leave questions unanswered or answered by chance, due to the difficulty of test.  This fact also affected validity, in its content and in general, because there was a huge incongruence between the test questions and human sciences teachers’ knowledge; I talked to many colleagues and they didn’t know what to answer because most of them stopped studying mathematics years ago. Going on reliability was not present in the test, since people felt so shocked to the extent to puke in the restrooms, some of them had to come from other cities (because the exam was not administered in every single city, obviously)  without having slept well or eaten well,  and so on. The administrators of tests were not prepared on giving instructions of the sections and time allotted for each part; it was chaotic. In some questions, there were more correct responses, ambiguous and, in some cases, unauthentic, especially in the psychological test, where you had to answer current pedagogical questions with illogical answers. As for washback effects, all of my partners, teachers and I felt frustrated, deceived and disillusioned to work for the government teaching staff, because it was an exam that did not  measure our knowledge, even in the literature and linguistic sections, since it was more for semiotists and philologists rather than Spanish teachers.  

But there is a political paramount issue within this problematic, and it is the fraud that some test makers and test takers did. It turns out that one month before the exam administration, test makers or some ICFES workers answered the generic exam and distributed answers, by charging a considerable amount of money to well-off or politically influenced teachers, fact questionable and dishonest. It has been the most corrupted and deplorable idea from those people to compete with a fraud among thousands of needed honest teachers, and taking advantage of that situation: if they have no political power to get a job, they just have a carte blanche to ensure their first step to get a post in the government. It is absurd that we, teachers or lectures, give talks about democracy, ethics and diplomacy and we find an abysmal incongruence among thoughts, talks and actions. And a more miserable fact is that we are competing with the same compatriots, killing and stealing each other while the government is pleased to consider people as combatting animals, and making policies to receive exaggerated budgets for itself and its hermetic elite.

To wind up, these tests has a sociopolitical aim, which is to ensure a good education for the elite and obstacle knowledge and aspirations for the working class, misleading people to misery and making them succumb to the criminal power. That’s why we, teachers, have to begin helping students develop their critical thinking so they can act and vote wisely. Also teachers should struggle, by using our mental power to influence politicians and policy makers to improve this educational system , as Messic argues (1989).