Chulalongkorn University has sleepwalked its way into an unfortunate muddle. Five final-year students have just discovered that they failed a first-year course and should no longer be students. By the rules of the university they should have been thrown out 2 years ago.

For those not familiar with the details of the case, it all began when their teacher of a first-year Law and Society course was too ill to submit the grades. His students were allowed to continue their studies while their grades were pending.

It is not altogether clear why it has taken 2 years for the delayed grades to be issued. (If, heaven forefend, the instructor had died, would they still be waiting?). But when their papers were eventually marked, it turned out that they had failed. And this F pushed their Grade Point Average (GPA) for Year 1 below 2, which is the threshold that Chula demands for students to continue studying.

But they had continued, successfully completing Year 2 and Year 3. So now the Chula high hiedyins are casting about for some way out of the dilemma.

If their GPA, including the new grade from over 2 years ago, is now less than 2, they will be allowed to re-enrol as first-year students at the university’s expense and re-live the past 3 years. (Ah, if only life could be rewound that way, how many past wrong turns could be avoided! Why, I might even escape having to write these things every week.) Or they can transfer their current credits to any other university willing to accept a Chula reject.

Most solutions on offer follow the totally impractical lines of ‘What you should have done is …’. But let me focus on the issue of how Chula, like all other testing institutions, has been tripped up by faulty testing.

Tests can be roughly divided into three groups. One will test if you have learned what you were taught. These achievement tests, if the course of study is short and clearly defined, are not that difficult to write (though this ability is rarely found in the Thai education system).

Then there are the tests that try to measure what you can do in, for example, speaking English, or driving a car, or doing brain surgery. Such tests of competence don’t care what you have been taught or where. It’s a question of the level of your ability at this moment, however acquired.

Valid tests of this kind are much more difficult to construct, mostly because the competencies that you are normally interested in are horrendously difficult to define with any accuracy. Not that this stops people devising and using tests that claim to test competence but patently don’t. Lots of tests of English competence in the Thai education system would bamboozle a native speaker.

And then there is the holy grail of testing. A test that measures not how well you have learned in the past, nor how well you can do at the present, but how well you will do in the future.

There are, for example, tests that claim to predict who will be good at learning foreign languages before you have even learned the first word. They look for the attitudes and skills that successful language learners often possess. But these factors, apart from being inherently vague, are at best helpful conditions, rather than necessary or sufficient. The tests I have seen do not impress.

HR managers would give their right arms for a predictive test that was reliable. But alas, they are as rare as unicorns or cabinet ministers you can trust.

And Chula tests the ability to succeed in next year’s classes not by any single test, but by GPA. This is a mathematical average of (rather crude) evaluations of past performance, used as a measure of academic aptitude. As a measurement of future intelligence, this is as valid as giving a score out of 4 for temperature, pulse, blood pressure and cholesterol level, calculating the mean, and using that to predict if you’re going to be healthy next year.

The problem with predictive evaluations is that under normal circumstances, there is no way to validate them. With 10 candidates for 1 position, you administer a test, choose the one who comes top and reject the rest. You will find out how well this one fares, but no one can know if the rejects would have done better.

It is only in unusual situations that someone is selected when they have failed the test, and then goes on to perform adequately, that the selection method can be proved invalid.

Which is what just happened at Chula.