This was a well received opinion piece published in theantdaily.com written by one of us.
Posted on 09/11/2013 – 18:32
COMMENT: Recently, a local daily carried a story regarding a report by Education First which has a chain of English language schools internationally, showing that Malaysia ranked 11th among a group of 60 countries in English proficiency. Not only that, the report also put Malaysia as the top Asian country in English proficiency, one position above Singapore.
There seems to be cause for celebration. It implies that reports of employers and job placement agencies complaining of erosion of our command of English are not in sync with the results of this report. It also implies that the RM270 million spent over the last three years on mentoring 7,500 English teachers by the government was bearing fruit. But hold the champagne or the non-alcoholic grape juice version. Things are not as rosy as it is portrayed by Education First’s report.
If one downloads and reads Education First’s English Proficiency Index 2013 (Third Edition) (EPI), one would be able to learn a few more things about this report.
Firstly, the EPI was compiled using two online tests. The First Test comprised 30 questions which were available (and still are) free of charge to any visitors to Education First’s website (or that of its associates). The Second Test was a 70-question English Placement test conducted on enrolment on Education First’s students in those countries where it has operations. The analysis of the data as reported in the EPI did not distinguish between test takers who took the free online test (First Test) and those who were Education First’s students taking the Second Test. Education First also did not give any indication of the nationality of the students taking the Second Test. It could well be that the majority of students taking the Second Test in Singapore were foreign students as the national school system in Singapore has adequate teaching and assessment regime. And more importantly, all other surveys and reports have been indicating the higher English proficiency of Singaporeans among all the population of Asia. Thus the Singapore result could well have been a reflection of the English proficiency of the foreign students learning English there. I believe Singapore learners enrolled in Education First’s programmes who are not foreign students would have been those who have a lower English proficiency than the general population (which would have been the reason for them enrolling in English classes in the first place). Thus this would at best be a gauge of the level of English proficiency of those enrolled in Education First’s schools in Singapore which may not reflect that of the general population. Interestingly, Education First’s website does not show that it has any operation or associates in Malaysia. Thus one could only assume that the Second Test was not available in Malaysia and the result for Malaysia was based solely on the First Test.
It could well be that the free online First Test was taken by Malaysia-based takers who were a mix of Malaysians and foreign students. This may explain why test takers in Malaysia scored higher than those in Singapore (assuming that the majority of takers of the tests in Singapore were foreign students of Education First). As there was no data captured on the nationality of takers for the two tests, we will not be able to read too much into the results. The result could also reflect the possible better command of English of foreign students in Malaysia rather than a representation of the general population.
It is also not a good practice to compare the scores of takers in two different tests and combine the results where some but not all participants had taken both tests.
To be fair, EPI 2013 did explain its methodology reasonably clearly and also stated clearly the limitations of the analysis. With regard to the free online First Test, it says, “We recognize that the test-taking population represented in this index is self-selected and not guaranteed to be representative of the country as a whole. Only those people either wanting to learn English or curious about their English skills will participate in one of these tests. This could skew scores lower or higher than for the general population.” Thus one should not read too much into the findings of EPI 2013. However, the Second Test, which was administered to students of Education First, would measure more accurately the English Proficiency of the test takers and at the very least the aggregate profiles of these takers could be ascertained easily.
With the data from 750,000 test takers, the EPI is a reasonable measure of their English proficiency, but unless there is demographic data associated with the report, one will not be able to draw concrete conclusions from the report.
Thus we can only claim that test takers of the free online First Test based in Malaysia were collectively ranked 11th among 60 countries where the tests data were analysed. The result from the EPI does not represent any of the general population of those countries. In other words, Malaysia still needs to do a lot to arrest the decline of English proficiency among our people. We should not be celebrating.
Dr Chow Yong Neng has served the education and training industry for over 17 years in diverse capacities.