To NABP: Disagreements to NABP's false statements - TOEFL iBT

[Antimonopoly]

I am extremely disappointed in the uncaring decision and comments posted by the NABP Task Force committee. The document, which fails to address our concerns, is filled with glaring omissions and inaccuracies.

The following are my comments, point by point:

NABP 1: “Recommendation 1. Members agreed that TOEFL iBT should remain as the sole English proficiency test for FPGEC certification for the following reasons: Uniformity of the requirements for FPGEC certification to ensure that all applicants meet the same criteria.”

Comments: 1a. Uniformity and monopoly seems to go together. These two words are interchangeable in this situation. However, test takers are paying the price for uniformity/monopoly and are frustrated by the limited number of qualified TOEFL test facilities when ETS falsely advertises its high accessibility. We are not talking about the number of test centers that NABP mentioned in the task force report but the real number of secure testing facilities such as Internet Café.

1b. Limited number of secure testing facilities and inconvenience According to statistics, the tremendously high requirement of TOEFL imposed by NABP forced test takers to repeat the test several times. The passing rate of TOEFL - Speaking is 11 percent. Therefore, the average number of times test takers must take the test before passing is 9 times. If the number of secure testing facilities is sufficient or, in other words, test takers can register for the test any time they want in a period of 1-2 months, uniformity is possible but in TOEFL iBT, there is a limited number of secure testing facilities. I have not been able to find a spot to take the test in my area since January. They are booked up until May 30 this year. Wake up! Be realistic and practical. Visa Screen for nurses recognized other English tests because of problems in accessibility. This is also one of many reasons that more American universities recognized IELTS in the past years.

1c. It is too ideal to meet the same criteria in TOEFL when inconsistency of scoring system is common. In test takers’ point of view, the TOEFL has no standard. When test takers requested rescoring of the same test or requested an independent rating, the majority of them received changes in their scores of up to 6 points in the speaking section. If TOEFL uses the same criteria or same quality of raters, their scores should have not been much different.

1d. English is universal language. America is becoming more diverse. Any English tests recognized by thousands of institutions around the world should be interchangeable. There is no point in uniformity or encouraging the monopoly, especially when testing administration, testing accessibility, and scoring process are problematic. Please check report of ETS and the Better Business Bureau.

1e. Uniformity with other health care professionals The United Department of Health and Human Services has approved both TOEFL and IELTS for certificate for foreign health care workers. Only NABP insists on using one test, which encourages the monopoly. Visa screen for nurses allow alternative exams to increase accessibility; IELTS, and TOEIC. Why does NABP keep the same English requirement as that of HHS if uniformity is an issue?

NABP 2: “Accessibility of the proficiency examination to the greatest number of applicants; and Task force members considered the fact that the TOEFL iBT is offered 30 to 40 times a year and is available in 178 countries; whereas, the University of Michigan’s Michigan English Language Assessment Battery (MELAB) is only offered three times a year and is available in only two countries; and the International English Language Testing System (IELTS) is offered 30 to 40 times a year but is limited to 121 countries.”

Comments: 2a. It is not true. Even though TOEFL is offered in more countries than IELTS, it is not easy to find a seat to take the test. I cannot register in my area until May 30th of this year. This is because more students want to enter and study in the United States and they have to take TOEFL required by institutions. Pharmacists must compete with students to register to get a seat for TOEFL. If NABP recognized both tests, they will be available in a maximum of 178+121 countries. The more tests, the more accessibility.

2b.The quantity of test facilities cannot compensate for the quality of secure testing facilities. Only a handful of TOEFL testing centers are not too disturbing for test takers.

NABP 3: “Also discussed was the fact that though the IELTS is a face-to-face format, it utilizes only one interviewer for scoring as compared to TOEFL iBT, which utilizes four raters to score applicants. It was agreed that the use of only one interviewer increased the level of subjectivity and was determined to possibly affect examination results. In addition, IELTS raters are retested once every two years, whereas TOEFL iBT raters must pass a calibration test every time they begin a scoring section.”

Comments 3a. The number of raters should not make different if they can grade correctly. In real life, pharmacists deal with thousands of patients each year. Either one rater or 4 raters or 10 raters are not significantly different. One rater who has sufficient experience and is able to grade correctly is, of course, better than 10 raters who cannot grade the test correctly. This can be seen on inconsistencies of TOEFL scores on the same test of the same test takers when tests have been rescored. How can it be possible that the same test takers receive different ratings, decreased from fair to limited or even from good to limited on the same questions? This complaint has been submitted to NABP over and over but NABP failed to address it.

3b. Tricky advertising technique by TOEFL When talking about 4 raters in TOEFL, it does not mean that the same question is graded 4 times. It is just a technique that ETS uses to advertise. The test is graded only 1 time but different graders grade different questions. If ETS claims that TOEFL raters must pass a calibration test every time they begin a scoring section and assume that rater has the same standard, then using 4 raters or 1 rater should not make any differences since all raters should be able to grade at the same standard which creates no or negligible deviation. When a test taker requests for rescoring, only 1 rater grades his/her test. To my surprise, the grade is always different. NABP has all of the data but fails to address the problem about inconsistency of scores after rescoring.

3c. Quality of grading online is questionable. On top of that, IELTS grades the test while they are in the test room. TOEFL is graded online. It is quite possible that online grading may not be as reliable as grading at the test center since raters may be distracted or do not concentrate enough since no one can see them.

3d. Recognizing speaking skill is common sense. Patients do not need to pass a caliber test of ETS. In the real world, patients do not have to retest to understand pharmacists. Listening skills have been acquired since birth. It is common sense and easy to tell whether speakers are easy to understand or not. Retest or not of raters means nothing. The test should focus on how easy test takers are to understand or satisfactorily understand questions and answer what is being asked in short in answer in a reasonable time frame, not 40 seconds or 1 minute. ETS punishes test takers when test takers fail to completely answer question in one short time period. In real patient settings, patients can ask pharmacists to elaborate if they want more explanation.

NABP 4: “Members discussed how the TOEFL iBT also tests applicants’ English proficiency in an integrated manner whereby they must combine their communication skills. Integration involves either listening or listening and reading information and then formulating a spoken response to a question that has a correct answer. A non-integrated or independent manner of testing only requires a candidate to respond to a general question based on personal opinion or experience and has no right or wrong answer. Members agreed that an integrated test format provided the best method for determining proficiency.”

Comments 4a. Integrated tasks are not exactly simulating real pharmacy practice. In addition, reading and listening ability are tested in other sections. Indeed, IELTS also offers integrated tasks in the speaking section but it does not focus on academic settings as TOEFL does. In real pharmacy practice, pharmacists talk to patients face to face, not on a recorder. There is no time limit to think and answer questions. Both listeners and speakers can clarify questions. In TOEFL iBT, this aspect is missing. When test takers cannot get a high mark, most of the time, it is because time allotted to read the passage or think in order to organize is not enough and, sometimes, other test takers who are doing the test in the same room shout at the top of their lungs, distracting the test takers. Topics in TOEFL are completely unrelated to pharmacy professional. Test takers have no idea how to answer strange questions. A good speaking test should simulate a real working environment. In a pharmacy setting, pharmacists do not need to listen or read about how tigers are domesticated and answer patient’s questions within 1 minute. To be able to earn a high mark in TOEFL, test takers must answer completely when being asked. This complaint has been sent to NABP before but has not been addressed.

4b. In addition, TOEFL iBT lacks interactive response. NABP does not mention in the report the complaints about nonverbal communication which is missing. Some experts confirmed that in communication in real life, non verbal accounts for 90% of all communication. This topic is also missing in the task force report. It has not been addressed.

4c. The content and format of TOEFL is not similar to real pharmacy practice. TOEFL focuses on reducing costs so it does not offer real interviews. The Doctors' Board offers a communication skill test which simulates a patient care setting by having a face to face Interview. It is sad that NABP refuses to find a better test to grade speaking while patient safety is at stake. TOEFL iBT measures a different aspect of speaking ability which is unrelated to pharmacy practice since its content and format measures speaking ability in an academic environment or how well students can speak and understand in a lecture hall or classroom, not in patient consultation patient care. The NABP who are pharmacists refuse to listen to pharmacists on the other side. The criteria used in determining speaking performance were based on ETS’s advice rather than looking at a real patient setting or listening to pharmacists who have experience with TOEFL iBT and ETS.

To be continued....

 

[Antimonopoly]

NABP 5: “Finally, members discussed that in March 2008, after careful consideration of the other English language proficiency examinations available, the NABP Advisory Committee on Examinations (ACE) recommended to the NABP Executive Committee that the FPGEC forego the adoption of a new examination and continue to require the TOEFL iBT since the examination reaches the greatest number of FPGEC applicants and provides continuity and uniformity in testing from location to location. Members agreed that ACE’s recommendation, being sound and objective, should be upheld and that the TOEFL iBT remain the sole FPGEC certification examination.”


Comments 5a. The concerns about quality of test facilities were not adequately addressed before the decision was made. The decision ignores the complaints of test takers. When 50 pages of complaints are made by hundreds of test takers, concerns should be investigated at the test facilities, not just consulting with ETS officers. When a robbery occurs, police should not ask the thief how the robbery happens and then, rush to say, “He did not steal. You dropped the money yourself. I asked him and he told me that he did not do it on purpose. That man saw your money and then he took it so he is not guilty.”

5b. NABP states in 2005 that IELTS standard is recognized as equal to TOEFL. IELTS is also recognized by many institutions in America. English is a universal language. No institutions encourage the uses of the monopoly. Democracy supports competition. Monopoly reduces productivity. One of many reasons foreign pharmacists migrate to the United States is that it is a democracy, of the people, by the people, for the people. In this decision, voices of test takers are not listened to.


5c. Using one test will increase subjectivity. Kenya is proud to say that it exported a good president to the United States. If the United States were a closed country, this could not happen. The United States is more diverse. NABP states that using one rater increases subjectivity. By the same token, I am afraid that using one test will increase subjectivity as well.

NABP 6: “Recommendation 2: The task force recommends that NABP retain 26 as the passing standard requirement for the TOEFL iBT as it is identical to the passing standard utilized by other health care professions and meets the expectations of accredited programs.”

“Background:

Task force members reviewed the passing standard for the following health care professional associations, all of which require a score of 26 for the speaking portion of the TOEFL iBT:

1. Commission on Graduates of Foreign Nursing Schools;

2. Educational Commission for Foreign Veterinary Graduates;

3. Federation of State Boards of Physical Therapy;

4. International Commission on Health Care Professionals; and

5. National Board for Certification in Occupational Therapy.

Also noted was that 39 out of 40 state nursing boards that utilized TOEFL iBT required a passing standard of 26.”



Comments 6a. This claim is not true. From where has this piece of information been retrieved? As far as I know, CGSNF requires only 83 of total score of TOEFL, nurses can practice and receive their certificate at that requirement without a score of 26 on the speaking section. A requirement of 26 on the speaking section is for visa screening or those who want green cards.

6b. As NABP states above, information is false. Correct information is below:

o Commission on Graduates of Foreign Nursing Schools;

§ Requires TOEFL passing score: 83, with no speaking requirement TOEIC: 725, IELTS: 6.5 (http://www.cgfns.org/files/pdf/req/cp-requirements.pdf)



o Educational Commission for Foreign Veterinary Graduates;


ECFVG - Educational Commission for Foreign Veterinary Graduates


§ The old minimum requirements for the TOEFL iBT: 80 overall, Listening: 26, Writing:17, and Speaking: 26.

§ The new minimum requirements for the TOEFL iBT: No overall, Listening:25, Writing:22, and Speaking:22, Reading:23 (On the basis of a standard setting exercise conducted in June 2008), with section scores accepted from separate internet-based TOEFL exam administrations conducted over a two-year period.

§ Or IELTS: 6.5 (overall band score), Listening: 6.5, Writing: 6, Speaking: 7

§ Or the Canadian Academic English Language (CAEL) Assessment is 60 overall, Listening band: 60, Writing band: 50, and Speaking band:60

o Federation of State Boards of Physical Therapy;

o International Commission on Health Care Professionals; and

o National Board for Certification in Occupational Therapy.


NABP 7: “Members voiced concern that there is no assurance that FPGEC candidates have ever communicated with patients on the same level as US students as provided by adherence to ACPE criteria.

Guideline 12.1 provides, in part, that pharmacy graduates must be able to:

1“Communicate and collaborate with patients, care givers, physicians, nurses, other health care providers, policy makers, members of the community, and administrative and support personnel to engender a team approach to patient care; and

2Retrieve, analyze, and interpret the professional, lay, and scientific literature to provide drug information and counseling to patients, their families or care givers, and other involved health care providers.


Members concluded that the TOEFL iBT passing standard of 26 provides a means of measuring foreign graduates’ ability to competently communicate.”

Comments 7a. It is not true that those who receive a score below 26 cannot competently communicate. If above information is correct, why do those who claim that their primary language is English have to take the test 5-6 times before receiving a score of 26. According to ETS data, the average score of American candidates is 22 and candidates from English speaking countries is also 22. To my surprise, Denmark is the only country that has average score of speaking section equal to 26. If candidates from English speaking country cannot earn a score of 26, does that mean that they cannot speak English correctly and communicate and collaborate with patients in the standard of ETS and NABP altogether. This high mark requirement creates a loophole and raises the question, “Who gets the money from unnecessary retaking of the test, which does not correctly measure language ability?”

7b. NABP claims that passing standard for the following health care professional associations, all of which require a score of 26 for the speaking portion of the TOEFL iBT. This statement is not true. The Veterinarian board has approved new requirements for TOEFL. The new speaking requirement is at 22. It lowered the speaking section from 26 to 22.

NABP 8: “Recommendation 4: NABP Should Close Investigation into Petitioners’ Complaints and Allegations Pertaining to the TOEFL iBT.The task force recommends that NABP close the investigation into the petitioners’ complaints alleging that the TOEFL iBT is unfairly administered. These allegations have been adequately answered and bear no merit for NABP either consider lowering the TOEFL iBT standard for the speaking portion or utilize other English language proficiency tests for FPGEC certification.”

Comments 8. ETS failed to provide secure testing facilities. Several complaints have been made to ETS and NABP and the complaints have been ignored and not appropriately addressed.

NABP 9: “NABP staff also provided a demonstration of the TOEFL iBT practice test containing examples of high, medium, and low responses to assist applicants in preparing for the examination.”

Comment 9. According to test takers’ experience, please let me know why on the same test of the same test taker, on the same questions, when test takers request rescoring, the grading can be changed from low response to high response by different raters. Please answer us. This complaint has been filed to NABP but was ignored in this report. This test is subjective and TOEFL raters themselves are confused, making it hard for them to differentiate which answers are low, medium, or high, especially from different raters. One candidate received 18; after her rescoring the score of 24. Can you please explain this?


NABP 10: “It was acknowledged that during 2000-2008, of the approximately 12,000 candidates who passed the FPGEE, an overwhelming majority became FPGEC certified after passing either the Test of Spoken English or the TOEFL.”


Comment 10. Please sincerely disclose the number referring to “overwhelming majority”. As far as I know several thousand FPGEE candidates cannot pass TOEFL iBT and receive the certificates. Why is this piece of information hidden in the NABP database? I am afraid that if NABP discloses correct data, the number of future candidates who decide to take the FPGEE will significantly drop. I know that the number is approximately 5,000 (for candidates who passed FPGEE but cannot receive their certificate.) If I had known this number before, I would not have started this FPGEE process. It causes embarrassment to me, emotional distress to my family, and every member of my family is paying the price for this process. It is unethical to take $700 from any of us and say “shut up and go back to your home country. We have enough pharmacists here. We don’t want you. We just want your money. ”



To be continued.....

 

[Antimonopoly]

NABP 11: “Members were informed that it was a limited number of individuals who comprise the subgroup of petitioners repeatedly requesting an alternative to TOEFL iBT or a lowering of the passing standard. After careful examination of the petitions, members concluded that:
“Of those requesting an alternative English proficiency test, none had provided proof of successful passage of either the MELAB or IELTS”

Comment 11. NABP failed to read the complaints and properly address problems in a sound and reasonable manner. Many test takers state that they receive 8 on speaking sections on IELTS while fail to achieve a score of 26 on TOEFL iBT. If we send proof, would NABP recognize IELTS?

NABP 12: “Of those alleging grading discrepancies, none had indicated that they requested an independent rating.”

Comments 12a. Again, NABP fail to read complaints thoroughly. Independent rating means “rescoring” since we receive a letter from ETS when we request it. It states that an independent rater has rated your test. Test takers state many times in their complaints that they requested rescoring and their scores have changed. NABP has their records but NABP neglected to pay attention to their concerns. Some of them received a score of 22 and later received 18 or 27 after rescoring. It is ridiculous that test takers pay $60 for rescoring and only one rater grades their test and that rater can change their destiny. Only one rater and one company, ETS, can change their destiny. Is it fair? This happens over and over as time goes by.

12b. Pharmacists are smart people and they deserve to be treated and listened to better. If the economy is an issue in not considering their complaints at this time and NABP is happy with the number of foreign pharmacists who can pass the English test at an 11 percent passing rate, please announce the number of those who took FPGEE in the past, and the number of those who received certificates each year. This will show NABP management’s sincerity regarding the FPGEE certification program. Candidates will have better understanding before they jump onto the bandwagon and keep paying for TOEFL iBT. Or probably, NABP takes $700 from test takers and does not care whether they can qualify to practice in a fair process. According to Freedom of Information Act (FOIA), test takers have the right to know the percentage of those who can achieve the FPGEE certification. Many pharmacists have good lives back home but have the wrong idea on how difficult it is to earn the US pharmacy license. They have families and live back home. It is not fair for them that NABP hides information and lets ETS manipulate their lives.

NABP 13: “Of those alleging unfair and illogical scoring process, none stated that they utilized the TOEFL iBT practice test to prepare.”

Comments 13. This is not true and test takers are not aware that they have to state that they utilized the TOEFL iBT practice test to prepare. Most of them bought all test preparation materials, used tutors, and went on with TOEFL preparation courses available in the market. As test takers, nobody wants to pay $150 without any preparation. I even paid $40 for a pretest from ETS and I did 6 questions but I received a half line evaluation on each question from ETS’s rater. It is a waste of money. Some test takers could not even access the test preparation that they bought. They complained to ETS but ETS just said, “Sorry we can do nothing.”

NABP 14: “Of those complaining about the allotted time, none had conceded that the time frame was consistent and realistic as compared to the North American Pharmacist Licensure Examination.”

Comment14a.TOEFL and NAPLEX are incomparable. NAPLEX is a multiple choice exam. Test takers are required to click on the answer, not to elaborate on the content that they have a chance to listen to only once. It takes a few seconds to click on an answer. The tasks are different. NAPLEX questions are related to pharmacy practice, in which pharmacists have a background while TOEFL questions are not what candidates are expose to in everyday life. Test takers have no clue as to what to say about some topics. Test takers complain about this topic over and over but NABP never listen nor take them seriously.

NABP 15: “Of those complaining about environmental issues, none provided compelling evidence that real-life work situations are less stressful, and without noise or disruptions of any kind.”

Comments15. It is a code of conduct at test sites anywhere in the world that test companies must provide secure testing facilities with a quiet environment. It does not matter if the work setting is noisy. In the workplace, patients can ask pharmacists to repeat or vice versa if there are too many interruptions. The interruptions from microphones distract test takers and prevent them from performing at their maximum potential. It is unethical that ETS fails to provide adequately secure testing facilities while NABP unthinkingly supports it.

NABP 16: “Overall, the members agreed that the TOEFL iBT is a test that is transparent and objective and, with proper preparation, candidates can obtain a passing score.”

Comments 16a. Again, if it is as NABP claims, would it please provide specific details of “proper preparation” in this context? Does it mean that candidates must practice speak to complete answering a question in 40 seconds or 1 minute in all 6 questions? Does it mean that candidates must be alert after finishing up 3 hours of the first 2 sections without a meal and do well on the speaking section after a 10 minute break? In a real patient setting, it takes about 2 minutes or more for a pharmacist to completely consult with a patient.

16b. Server problems and transmission errors are not addressed. This does not include the situation where tests have been delayed because of server problems. Some candidates did not have a chance to eat since they had to wait for the server to function. This topic about technology and computer problems was not addressed either in this report. A number of test cancellations or rescheduling without notification to test takers were also reported to NABP but not addressed. Some test takers traveled across the country, staying overnight in hotels only to have their tests canceled. It must be admitted that it is hard to rely on technology, especially in remote areas of the United States and the third world countries. Inconveniences occurring from disorganization of ETS are not addressed. Other topics related to the Internet are ignored completely. This topic is huge and wide spread. Please listen and take action.

16c. NABP defends ETS. The claim by NABP that the test is objective is false. It is surprising that when complaints from test takers have been made, they are not deeply investigate or appropriately addressed. Instead, NABP defends all problems for ETS. Does NABP think that we test takers are all liars? We have dignity. We can work anywhere in the world. We made complaints hoping that NABP was a fair and trustworthy organization that would sincerely investigate cases rather than defend ETS.

16d. The tests that are not multiple choices or true-false but require elaboration or offer open-ended questions will never be objective tests because the answers vary greatly. Factors related to that degree of objectivity or subjectivity vary among raters, test questions and answers. Different raters have different levels of experiences and knowledge to judge test takers’ performance. It is easy to see from discrepancies of scoring variation from the same test takers in the different tests and even in the same tests when they require rescoring.

16e. Response to a message from NABP: Refrain from contacting NABP committeeNABP in its email tells candidates to refrain from contacting NABP committee regarding TOEFL iBT complaints since NABP will not consider them. When problems occur in any steps of the licensure process, in this context, problems in TOEFL testing procedure, ETS continues to inadequately address candidates’ concerns. If candidates do not contact NABP which utilizes ETS as a subcontractor, which organization should they talk to? Would it be the attorney general, governor, senator, president, or no one? No, I won’t keep it to myself but will tell future test takers not to pursue the FPGEE certification program since nobody cares anymore.

Organizations such as the big 3- automakers: Ford, General Motor and Chrysler are in big trouble since they continue to believe in their old policies, with little attention to customer satisfaction worldwide. Japanese automakers entered the US market and are doing well since they care about their customers and are thankful for any comments their customers send to them. Some Japanese companies value their customer feedback and even pay for those comments or surveys and continue to improve their products to satisfy their customers. Word of mouth methods spread quickly among customers.

The global labor market has become more competitive than ever. It is impossible to stop people from migrating from one place to another. The world is becoming smaller. Economy, length of time and costs in relocation, convenience, period of time to acquire licensure, exchange rate and currency value, and working conditions are the main factors in deciding whether or not they would either uproot their families or leave their families. Some of them relocate because of socio-economic issues.

There is a worldwide shortage of pharmacists. The financial crisis in the United States and H-1 visa program create problems for those who would like to acquire a work visa and start their Internship. Some candidates feel that they work hard for FPGEE and TOEFL for nothing since they cannot work legally. Furthermore, one more important factor for those who migrate for financial gain is the possible future depreciation of the US dollar, which is still a controversial issue because of high debts and the trade deficit. The Euro-dollar has become stronger than the US dollar. The US Gold reserve ranks in the bottom 5 in the world. With all of these reasons, future candidates must think twice before they decide to start any licensure process not wanting to waste hard earned money or lose progress in their career waiting for the US licensure. Check the statistics from reliable sources before you make a decision. Otherwise, your journey may not be pleasant. Your goals of making money may turn to disaster (high debt). I am not kidding you.

I will end my comments for now, with little hope that the United States pharmacy licensure governing body will reconsider changing its current standpoint, defending itself by blaming the majority of candidates who ask for the right to have a fair English testing procedure and fair requirements. It is normal that people are afraid of change but if change improves the health care system, improves pharmacists’ quality of life, and improves patient care, nobody will be blamed for that. However, if the problems continue in any steps of the licensure process, sooner or later, changes must be made, one way or another. The sooner changes have been made; the less the problems will damage the system.


If nothing has been adequately done to solve the problems, finally, candidates will learn by themselves and find other options for their lives. The update about situations or experiences will, of course, not stop at those candidates. They will do everything to warn others, “This way is rough. Don’t walk! You will be injured.”