I am extremely disappointed in the uncaring decision and comments posted by the NABP Task Force committee. The document, which fails to address our concerns, is filled with glaring omissions and inaccuracies.
The following are my comments, point by point:
NABP 1: “Recommendation 1. Members agreed that TOEFL iBT should remain as the sole English proficiency test for FPGEC certification for the following reasons: Uniformity of the requirements for FPGEC certification to ensure that all applicants meet the same criteria.”
Comments: 1a. Uniformity and monopoly seems to go together. These two words are interchangeable in this situation. However, test takers are paying the price for uniformity/monopoly and are frustrated by the limited number of qualified TOEFL test facilities when ETS falsely advertises its high accessibility. We are not talking about the number of test centers that NABP mentioned in the task force report but the real number of secure testing facilities such as Internet Café.
1b. Limited number of secure testing facilities and inconvenience According to statistics, the tremendously high requirement of TOEFL imposed by NABP forced test takers to repeat the test several times. The passing rate of TOEFL - Speaking is 11 percent. Therefore, the average number of times test takers must take the test before passing is 9 times. If the number of secure testing facilities is sufficient or, in other words, test takers can register for the test any time they want in a period of 1-2 months, uniformity is possible but in TOEFL iBT, there is a limited number of secure testing facilities. I have not been able to find a spot to take the test in my area since January. They are booked up until May 30 this year. Wake up! Be realistic and practical. Visa Screen for nurses recognized other English tests because of problems in accessibility. This is also one of many reasons that more American universities recognized IELTS in the past years.
1c. It is too ideal to meet the same criteria in TOEFL when inconsistency of scoring system is common. In test takers’ point of view, the TOEFL has no standard. When test takers requested rescoring of the same test or requested an independent rating, the majority of them received changes in their scores of up to 6 points in the speaking section. If TOEFL uses the same criteria or same quality of raters, their scores should have not been much different.
1d. English is universal language. America is becoming more diverse. Any English tests recognized by thousands of institutions around the world should be interchangeable. There is no point in uniformity or encouraging the monopoly, especially when testing administration, testing accessibility, and scoring process are problematic. Please check report of ETS and the Better Business Bureau.
1e. Uniformity with other health care professionals The United Department of Health and Human Services has approved both TOEFL and IELTS for certificate for foreign health care workers. Only NABP insists on using one test, which encourages the monopoly. Visa screen for nurses allow alternative exams to increase accessibility; IELTS, and TOEIC. Why does NABP keep the same English requirement as that of HHS if uniformity is an issue?
NABP 2: “Accessibility of the proficiency examination to the greatest number of applicants; and Task force members considered the fact that the TOEFL iBT is offered 30 to 40 times a year and is available in 178 countries; whereas, the University of Michigan’s Michigan English Language Assessment Battery (MELAB) is only offered three times a year and is available in only two countries; and the International English Language Testing System (IELTS) is offered 30 to 40 times a year but is limited to 121 countries.”
Comments: 2a. It is not true. Even though TOEFL is offered in more countries than IELTS, it is not easy to find a seat to take the test. I cannot register in my area until May 30th of this year. This is because more students want to enter and study in the United States and they have to take TOEFL required by institutions. Pharmacists must compete with students to register to get a seat for TOEFL. If NABP recognized both tests, they will be available in a maximum of 178+121 countries. The more tests, the more accessibility.
2b.The quantity of test facilities cannot compensate for the quality of secure testing facilities. Only a handful of TOEFL testing centers are not too disturbing for test takers.
NABP 3: “Also discussed was the fact that though the IELTS is a face-to-face format, it utilizes only one interviewer for scoring as compared to TOEFL iBT, which utilizes four raters to score applicants. It was agreed that the use of only one interviewer increased the level of subjectivity and was determined to possibly affect examination results. In addition, IELTS raters are retested once every two years, whereas TOEFL iBT raters must pass a calibration test every time they begin a scoring section.”
Comments 3a. The number of raters should not make different if they can grade correctly. In real life, pharmacists deal with thousands of patients each year. Either one rater or 4 raters or 10 raters are not significantly different. One rater who has sufficient experience and is able to grade correctly is, of course, better than 10 raters who cannot grade the test correctly. This can be seen on inconsistencies of TOEFL scores on the same test of the same test takers when tests have been rescored. How can it be possible that the same test takers receive different ratings, decreased from fair to limited or even from good to limited on the same questions? This complaint has been submitted to NABP over and over but NABP failed to address it.
3b. Tricky advertising technique by TOEFL When talking about 4 raters in TOEFL, it does not mean that the same question is graded 4 times. It is just a technique that ETS uses to advertise. The test is graded only 1 time but different graders grade different questions. If ETS claims that TOEFL raters must pass a calibration test every time they begin a scoring section and assume that rater has the same standard, then using 4 raters or 1 rater should not make any differences since all raters should be able to grade at the same standard which creates no or negligible deviation. When a test taker requests for rescoring, only 1 rater grades his/her test. To my surprise, the grade is always different. NABP has all of the data but fails to address the problem about inconsistency of scores after rescoring.
3c. Quality of grading online is questionable. On top of that, IELTS grades the test while they are in the test room. TOEFL is graded online. It is quite possible that online grading may not be as reliable as grading at the test center since raters may be distracted or do not concentrate enough since no one can see them.
3d. Recognizing speaking skill is common sense. Patients do not need to pass a caliber test of ETS. In the real world, patients do not have to retest to understand pharmacists. Listening skills have been acquired since birth. It is common sense and easy to tell whether speakers are easy to understand or not. Retest or not of raters means nothing. The test should focus on how easy test takers are to understand or satisfactorily understand questions and answer what is being asked in short in answer in a reasonable time frame, not 40 seconds or 1 minute. ETS punishes test takers when test takers fail to completely answer question in one short time period. In real patient settings, patients can ask pharmacists to elaborate if they want more explanation.
NABP 4: “Members discussed how the TOEFL iBT also tests applicants’ English proficiency in an integrated manner whereby they must combine their communication skills. Integration involves either listening or listening and reading information and then formulating a spoken response to a question that has a correct answer. A non-integrated or independent manner of testing only requires a candidate to respond to a general question based on personal opinion or experience and has no right or wrong answer. Members agreed that an integrated test format provided the best method for determining proficiency.”
Comments 4a. Integrated tasks are not exactly simulating real pharmacy practice. In addition, reading and listening ability are tested in other sections. Indeed, IELTS also offers integrated tasks in the speaking section but it does not focus on academic settings as TOEFL does. In real pharmacy practice, pharmacists talk to patients face to face, not on a recorder. There is no time limit to think and answer questions. Both listeners and speakers can clarify questions. In TOEFL iBT, this aspect is missing. When test takers cannot get a high mark, most of the time, it is because time allotted to read the passage or think in order to organize is not enough and, sometimes, other test takers who are doing the test in the same room shout at the top of their lungs, distracting the test takers. Topics in TOEFL are completely unrelated to pharmacy professional. Test takers have no idea how to answer strange questions. A good speaking test should simulate a real working environment. In a pharmacy setting, pharmacists do not need to listen or read about how tigers are domesticated and answer patient’s questions within 1 minute. To be able to earn a high mark in TOEFL, test takers must answer completely when being asked. This complaint has been sent to NABP before but has not been addressed.
4b. In addition, TOEFL iBT lacks interactive response. NABP does not mention in the report the complaints about nonverbal communication which is missing. Some experts confirmed that in communication in real life, non verbal accounts for 90% of all communication. This topic is also missing in the task force report. It has not been addressed.
4c. The content and format of TOEFL is not similar to real pharmacy practice. TOEFL focuses on reducing costs so it does not offer real interviews. The Doctors' Board offers a communication skill test which simulates a patient care setting by having a face to face Interview. It is sad that NABP refuses to find a better test to grade speaking while patient safety is at stake. TOEFL iBT measures a different aspect of speaking ability which is unrelated to pharmacy practice since its content and format measures speaking ability in an academic environment or how well students can speak and understand in a lecture hall or classroom, not in patient consultation patient care. The NABP who are pharmacists refuse to listen to pharmacists on the other side. The criteria used in determining speaking performance were based on ETS’s advice rather than looking at a real patient setting or listening to pharmacists who have experience with TOEFL iBT and ETS.
To be continued....