Educational Testing Service

Educational Testing Service (ETS), founded in 1947, is the world's largest private nonprofit educational testing and assessment organization.^[3] It is headquartered in Lawrence Township, New Jersey, but has a Princeton address.

Educational Testing Service

Type	501(c)(3)
Founded	1947
Headquarters	660 Rosedale Road, Princeton, New Jersey, U.S.
Key people	Robert S. Murley (Chairman)^[1] Amit Sevak (president and CEO)^[2]
Products	TOEFL and TOEIC tests, GRE General and Subject Tests, CET1 Exam by Certtia tests and Praxis Series assessments
Services	Testing, assessments and research for educational use
Subsidiaries	Edusoft Ltd. ETS Canada ETS Global B.V. Kira Talent Vericant
Website	www.ets.org

ETS' welcome sign, as seen from Rosedale Road in Lawrence Township

Messick Hall at ETS headquarters

Lord Hall at ETS headquarters

ETS develops various standardized tests primarily in the United States for K–12 and higher education, and it also administers international tests including the TOEFL (Test of English as a Foreign Language), TOEIC (Test of English for International Communication), Graduate Record Examination (GRE) General and Subject Tests, and The Praxis test Series—in more than 180 countries, and at over 9,000 locations worldwide. Many of the assessments it develops are associated with entry to US tertiary (undergraduate) and quaternary education (graduate) institutions, but it also develops K–12 statewide assessments used for accountability testing in many states, including California, Texas, Tennessee, and Virginia. In total, ETS annually administers 20 million exams in the U.S. and in 180 other countries.

History

ETS is a U.S.-registered 501(c)(3) non-profit organization created in 1947 by three other nonprofit educational institutions: the American Council on Education (ACE), The Carnegie Foundation for the Advancement of Teaching, and The College Entrance Examination Board.^[3] ETS was formed in 1947 to take over the testing activities of its founders (whose organizations were not well suited to running operational assessment programs), and to pursue research intended to advance educational measurement and education.^[4][5] Among other things, ACE gave to the new organization the Cooperative Test Service and the National Teachers Examination; Carnegie gave the GRE; and the College Board turned over to ETS the operation (but not ownership) of the SAT for graduating high school students.

Scientific contributions

In keeping with the purposes for which it was established, ETS developed a program of research that covered not only measurement and education but also such related areas as statistics, educational evaluation, and psychology, particularly cognitive, developmental, personality, and social psychology.^[6] This broad-based research program attracted many individuals who distinguished themselves in their fields, often while at ETS but also in subsequent professorial positions. Among the more influential scientists have been Harold Gulliksen (whose book, Theory of Mental Tests, helped codify classical test theory);^[7][8] Frederic Lord (item response theory); Samuel Messick^[9] (modern validity theory); Robert Linn (known for testing and educational policy); Norman Frederiksen (performance assessment); Ledyard Tucker (test analysis, including inventing the "Angoff Method" of standard setting); Donald Rubin (missing data and causal modeling from observational data); Karl Jöreskog (structural equation modeling and confirmatory factor analysis); Paul Holland (differential item functioning, test equating, causal modeling); Howard Wainer (differential item functioning, Testlet Response Theory, statistical graphics); John Carroll (language testing and cognitive psychology); Michael Lewis (infant cognitive, social, and emotional development); Irving Sigel (children's cognitive development);^[10] Herman Witkin (cognitive and learning styles); K. Patricia Cross (adult education); Samuel Ball (an evaluation researcher who documented the positive educational effects of Sesame Street); David Rosenhan (known for the Rosenhan experiment, which challenged the validity of psychiatric diagnosis); Jeanne Brooks-Gunn (the effects of poverty on infant, child, and adolescent development); Robert J. Mislevy (Evidence-Centered Design); and Anthony Carnevale (education and the workforce).

Members of the ETS staff have been among the presidents of the National Council on Measurement in Education (NCME); the Psychometric Society; the Measurement and Evaluation Division of the American Educational Research Association (AERA); the Evaluation, Measurement and Statistics Division of the American Psychological Association (APA); the APA Developmental Psychology Division; and the Jean Piaget Society. They have been among the executive editors of the Journal of Educational Measurement, Journal of Educational and Behavioral Statistics, Educational Evaluation and Policy Analysis, Journal of Educational Psychology, Journal of Applied Developmental Psychology, and Discourse Processes. Major citations received while on staff have included elected membership to the National Academy of Education (K. Patricia Cross, 1975; Gregory Anrig, 1981; Paul Holland, 2005; Randy E. Bennett, 2022; Irwin Kirsch, 2022); (the APA Distinguished Contributions to Knowledge Award (Norman Frederiksen, 1984), the APA Distinguished Scientific Contribution Award (Frederic Lord, 1988; Howard Wainer, 2009); the AERA E.F. Lindquist Award (William Turnbull, 1981; Frederic Lord, 1988; Samuel Messick, 1994; Paul Holland, 2000; Wendy Yen, 2008; Howard Wainer, 2015; Charles Lewis, 2018; Randy E. Bennett, 2020); the NCME Career Contributions to Educational Measurement Award (Frederic Lord, 1990; Paul Holland, 2004; Howard Wainer, 2007; Neil Dorans, 2010; Linda Cook, 2017; Shelby Haberman, 2019); The Psychometric Society's Lifetime Achievement Award (Howard Wainer, 2013), and the Jean Piaget Society's Lifetime Achievement Award (Irving Sigel, 2002); among many other awards.

The high caliber of scientific staff allowed ETS to produce both new knowledge and methodology, especially in measurement and statistics, much of which has been taken up by assessment organizations around the world. Among the key scientific contributions were:

• co-invention of item response theory, an integrated framework for asking and answering a variety of practical problems related to the design and analysis of tests;^[11][12][13]
• creation of an approach and software for structural equation modeling and confirmatory factor analysis (LISREL), used throughout the social sciences to test theoretical relationships among variables;^[14]
• seminal contributions to modern validity theory, including the idea that validity was a unitary concept and that the evaluation of score meaning requires consideration of the consequences of test use as those consequences may imply functional problems with the test;^[15]
• development of widely used approaches to data analysis when there are missing data;^[16]
• generation of approaches to causal modeling from observational data;^[17][18]
• invention of the In-Basket Test (used throughout the world to assess applicants for managerial jobs in a wide variety of industries);^[19]
• development of methods for detecting test unfairness, including invention of the Standardization approach to Differential Item Functioning (DIF) and application of the Mantel-Haenszel method;^[20]
• creation of the holistic-scoring approach to writing assessment, a means of rapidly and reliably judging the quality of essay text, which allowed direct writing assessment to become a more affordable alternative to multiple-choice questions for large-scale testing programs;^[21][22]
• development of research-based procedures and standards for occupational licensing and certification.^[23]

Current status

ETS' international headquarters is located on a 376-acre (1.52 km²) campus outside of Princeton, New Jersey in Lawrence Township, Mercer County;^[24][25][26] processing, shipping, customer service and test security is in nearby Ewing. ETS also has a major office in San Antonio, Texas, which houses its K–12 Assessment Programs division, and smaller offices in Philadelphia, Pennsylvania, Washington, DC, Hato Rey, Puerto Rico, Sacramento, and Monterey, California.^[27] Overseas office locations, all of which are associated with for-profit subsidiaries that are wholly owned by ETS, include Amsterdam (ETS Global BV headquarters), London (ETS Global BV), Seoul (ETS Global BV), Paris (ETS Global BV), Amman (ETS Global BV), Warsaw (ETS Global BV), Beijing (ETS China), Delhi (ETS India) and Kingston, Ontario (ETS Canada). Not including its for-profit subsidiaries, ETS employs about 2,700 individuals,^[28] including 240 with doctorates and an additional 350 others with "higher degrees."

To help support its nonprofit educational mission, ETS, like many other nonprofits, conducts business activities that are unrelated to that mission (e.g., employment testing). Under US tax law, these activities may be conducted (within limits) by the nonprofit itself, or by for-profit subsidiaries.^[26] Most of the "off-mission" work conducted by ETS is carried out by wholly owned, for-profit subsidiaries, including ETS Global BV, which contains much of the international operations of the company, ETS China, ETS India and ETS Canada.

About 25% of the work carried out by ETS is contracted by the College Board, a private, nonprofit membership association of universities, colleges, school districts, and secondary schools. The most popular and well-known of the College Board's tests is the SAT, taken by more than 3 million students annually. ETS also supports The College Board's Preliminary SAT/National Merit Scholarship Qualifying Test (PSAT/NMSQT) and administers the Advanced Placement program, which is widely used in US high schools for advanced course credit.

Since 1983, ETS has conducted the National Assessment of Educational Progress (NAEP), known as the "Nation's Report Card", under contract to the US National Center for Education Statistics. NAEP is the only nationally representative and continuing assessment of what US students know and can do. ETS is responsible for coordination among the nine NAEP Alliance contractors, for item development, and for design, data analysis, and reporting.^[29]

In addition to the contract work that ETS undertakes for nonprofit and government entities like the College Board, the National Center for Education Statistics, and state education departments, the organization offers its own tests. These tests include the Graduate Record Examinations (GRE) (for graduate and professional school admissions), the Test of English as a Foreign Language (TOEFL) (for post-secondary admissions), the Test of English for International Communication (TOEIC) (for use by business and industry), and the Praxis Series (for teacher licensure and certification).^[30]

In England and Wales, ETS Europe, a unit of the ETS Global for-profit subsidiary, was contracted to mark and process the National Curriculum assessments on behalf of the government. ETS Global took over this role in 2008 from Edexcel, a subsidiary of Pearson, which had encountered significant and repeated problems in carrying out the marking and processing contract.^[31][32][33] As was the case for Edexcel, The first year of ETS Global's operation was struck by a number of problems, including the late arrival of scripts to examiners, a database of student entries being unavailable,^[34] and countrywide reports of problems with the marking of the papers. The opposition Conservative Party (Tory) criticized the awarding of the contracts to ETS, and produced a dossier listing previous problems with ETS's service.^[35] The ETS contract with the QCA was terminated in August 2008, with an agreement to pay back £19.5m and cancel invoices worth £4.6m.^[36] Subsequently, the contract for National Curriculum assessment marking and processing was again awarded to Edexcel. Like the two prior contracts, the Edexcel contract has encountered significant quality problems^[37] and the tests themselves, the focus of longstanding controversy in the English education community and among the public, have been subjected to a massive boycott by schools.^[38]

In 2009, ETS released the My Credentials Vault Service with Interfolio, Inc to "simplify the entire letter of recommendation process".^[39]

Criticism

 

Pond with fountains behind Messick and Lord Halls. Steven Brill reported in 1974 that ETS is known "around Princeton ... for its extravagance."^[40]

ETS has been criticized for being a "highly competitive business operation that is as much multinational monopoly as nonprofit institution".^[41] Due to its legal status as a non-profit organization, ETS is exempt from paying federal corporate income tax on many, but not all, of its operations.^[40] Furthermore, it does not need to report financial information to the Securities and Exchange Commission, though it does annually report detailed financial information to the IRS on Form 990, which is publicly available.^[42]

In response to growing criticism of its monopolistic power, New York state passed the Educational Testing Act, a disclosure law which required ETS to make available certain test questions and graded answer sheets to students.^[43]

Problems administering England's national tests in 2008 by ETS Europe were the subject of thousands of complaints recorded by the Times Educational Supplement.^[44] Their operations were also described as a "shambles" in the UK Parliament, where a financial penalty was called for.^[45] Complaints included papers not being marked properly, or not being marked at all^[46] and papers being sent to the wrong schools or lost completely.^[47] It has even been suggested that the quality of service is so poor that the Department for Children, Schools and Families (formerly the Department for Education and Skills) might not be able to publish the 2008 league tables of school performance.^[48] However, the contract was ended by "mutual consent".^[49] The UK government asked Lord Sutherland to conduct an inquiry into the failure of the 2008 tests. The report included in its main findings:

• primary responsibility for this summer's delivery failure rests with ETS Global BV, which won the public contract to deliver the tests;
• ETS's capacity to deliver the contract proved to be insufficient. A lack of comprehensive planning and testing by ETS of its systems and processes was a key factor in the delivery failure;

In 1983, students of James A. Garfield High School in East Los Angeles, California, achieved unexpectedly high exam results on the ETS Advanced Placement Exam. ETS implied that the students may have cheated to obtain such results based on common mistakes across different exams. The students were required to prove their abilities and innocence by taking a second exam, which they did successfully.^[50]

Americans for Educational Testing Reform (AETR) claims that ETS is violating its non-profit status through excessive profits, executive compensation, and governing board member pay (which the IRS specifically advises against^[51]). AETR further claims that ETS is acting unethically by selling test preparation materials, directly lobbying legislators and government officials, and refusing to acknowledge test-taker rights. It also criticises ETS for forcing GRE test-takers to participate in research experiments during the actual exam.^[52]

In 2014 the BBC reported that the Home Office has suspended English language tests run by ETS after a BBC investigation uncovered systematic fraud in the student visa system. Secret filming of government-approved English exams needed for a visa showed entire rooms of candidates having the tests faked for them.^[53]

Tests administered

• Graduate Record Examinations (GRE)
• Preliminary SAT/National Merit Scholarship Qualifying Test (PSAT/NMSQT)
• College Level Examination Program (CLEP)
• Test of English as a Foreign Language (TOEFL)
• Test of English for International Communications (TOEIC)
• Certified English Test One (CET1)
• Test de français international (TFI)
• California High School Exit Exam (CAHSEE)
• California Standardized Testing and Reporting (STAR) Program – replaced by CAASPP (California Assessment of Student Performance and Progress) in 2015.
• the Praxis test (successor to the NTE)
• the National Assessment of Educational Progress (NAEP)
• the Examen de Admisión a Estudios de Posgrado (EXADEP)^[54]
• British Trade Test Institute (BTTI)
• Major Field Test for Master of Business Administration

See also

• SAT
• SPEAK (test)

References

1. "Company Overview of Apollo Education Group, Inc.: Robert S. Murley". Bloomberg Business. Retrieved November 12, 2015.
2. Jaschik, Scott (March 29, 2022). "A New Leader for ETS". Inside Higher Ed. Retrieved 10 May 2022.
3. History of the Educational Testing Service
4. Fuess, C.M. (1950). "The College Board: Its first fifty years". New York: The Columbia University Press.
5. Educational Testing Service (1992). "The Origins of Educational Testing Service". Princeton, NJ: ETS.
6. Bennett, R.E.; van Davier, M. (Eds.). (2017). "Advancing Human Assessment: The Methodological, Psychological, and Policy Contributions of ETS". Methodology of Educational Measurement and Assessment. Cham, Switzerland: Springer Open. doi:10.1007/978-3-319-58689-2. ISBN 978-3-319-58687-8.
7. Burkhart, F. (November 3, 1996). "Harold Gulliksen, 93, Pioneer in Testing, Dies". The New York Times. Retrieved April 17, 2010.
8. Gulliksen, H. (1950). "Theory of Mental Tests". New York: Wiley.
9. Landis, D.; Tzeng, O.C.S. (2002). "Samuel J. Messick (1931-1998)". American Psychologist, 57(2). pp. 132–133.
10. McGillicuddy-DeLisi, A.; Shafrir, U.; Johnson, J.; Renninger, K. (2008). "Remembering Irving Sigel". Journal of Applied Developmental Psychology, 29(4). p. 253.
11. Lord, F.M. (1980). "Applications of item response theory to practical testing problems" Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
12. Lord, F.M. (1952). "A Theory of Test Scores". Psychometric Monographs, 7.
13. Lord, F.M.; Novick, M.R. (1968). "Statistical Theories of Mental Test Scores". Reading, MA: Addison-Wesley.
14. Joreskog, K.G.; Van Thillo, M. (1972). "LISREL: A General Computer Program for Estimating a Linear Structural Equation System Involving Multiple Indicators of Unmeasured Variables (RB-72-56)" (PDF). Princeton, NJ: Educational Testing Service.
15. Messick, S. (1989). "Validity". In R.L. Linn (Ed.), Educational Measurement (3rd Ed.). New York: MacMillan. pp. 13–103.
16. Dempster, A.P.; Laird, N.M.; Rubin, D.B. (1977). "Maximum Likelihood from Incomplete Data via the EM Algorithm". Journal of the Royal Statistical Society, 39(1), Series B (Methodological). pp. 1–38.
17. Rubin, D. Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies, Journal of Educational Psychology, Vol. 66, No.5, (1974), pp. 689.
18. Holland, P. (1986). "Statistics and Causal Inference". Journal of the American Statistical Association, 81(396). pp. 945–960–103.
19. Frederiksen, N.; Saunders, D.R.; Wand, B. (1957). "The In-Basket Test". Psychological Monographs: General and Applied, 71(9). pp. 86–88.
20. Holland, P.W.; Thayer, D.T. (1988). "Differential item performance and the Mantel-Haenszel procedure". In H. Wainer & H.I. Braun (Eds.), Test Validity. Hillsdale, N.J.: Lawrence Erlbaum.
21. Coffman, W.E.. (1971). "Essay Examinations". In R.L. Thorndike (Ed.), Educational Measurement (2nd Ed.) Washington, D.C.: American Council on Education. pp. 271–302.
22. Elliot, N. (2005). "On a Scale: A Social History of Writing Assessment in America". New York: Peter Lang. pp. 160–165.
23. Esser, B.F.; Kruger, D.H. "Benjamin Shimberg, 85, Expert on Testing in the Professions". The New York Times.
24. Alan Stoskopf (Spring 2000). "Sat + Ets = $". Rethinking Schools. 14 (3). Retrieved 2007-07-04.
25. . CNN. 2002-06-28. Archived from the original on October 28, 2007. Retrieved 2007-07-04.
26. Randy Elliot Bennett (2005). "What Does It Mean to Be a Nonprofit Educational Measurement Organization in the 21st Century?" (PDF). ETS. Retrieved 2007-07-04.
27. ETS. "Contact Us". Retrieved 2010-05-12.
28. Jennifer Merritt (2004-04-26). . Business Week. Archived from the original on 2007-10-28. Retrieved 2007-07-04.
29. National Center for Education Statistics (2023). "Previous NAEP Contractors". Retrieved 2023-03-17.
30. ETS (2010). "ETS". Retrieved 2010-05-15.
31. Examiners knew about maths error
32. Call for a GCSE shake-up as pass mark sinks to 16%
33. Admin staff 'marking GCSE papers'
34. Lipsett, Anthea (May 15, 2008). "Headteachers angry at Sats 'nightmare'". The Guardian. London. Retrieved 2008-05-17.
35. Curtis, Polly (19 July 2008). "A history of exam failures". The Guardian. London. Retrieved May 1, 2010.
36. "Sats marking contract is scrapped". BBC News. August 15, 2008. Retrieved May 1, 2010.
37. Guardian (4 June 2009). "Hundreds of Sats examiners wrongly disqualified". The Guardian. London. Retrieved 2010-05-14.
38. Guardian (6 May 2010). "Hundreds of primaries to boycott Sats". The Guardian. London. Retrieved 2010-05-14.
39. . Archived from the original on 2009-12-19. Retrieved 2010-01-14.
40. Brill, Steven (October 7, 1974). "The Secrecy Behind the College Boards". New York. 7 (40): 67–83. Retrieved October 19, 2020.
41. Nordheimer, Jon; Frantz, Douglas (September 30, 1997). "Testing Giant Exceeds Roots, Drawing Business Rivals' Ire". The New York Times. Retrieved 2007-07-07.
42. "Teacher Watch: ETS Monopoly Continues". HorseSense and Nonsense. 11 November 2005. Retrieved 2007-07-07.
43. "Educational Testing Service - Hoover's profile". Answers.com. Retrieved 2007-07-07.
44. Warwick Mansell (4 July 2008). Chaos casts doubt over tests deadline. Times Educational Supplement.^{[permanent dead link]}
45. MPs criticise testing 'shambles'. BBC. 20 May 2008.
46. More questions about Sats results. BBC. 17 July 2008.
47. Schools hunting missing papers. BBC. 24 July 2008.
48. Mike Baker (18 July 2008). League tables 'might be scrapped'. BBC.
49. "Sats marking contract is scrapped". BBC News. August 15, 2008.
50. Stand and Deliver Revisited
51. United States Internal Revenue Service (February 7, 2007). "Good Governance Practices for 501(c)(3) Organizations" (PDF). Retrieved 2009-05-31.
52. Americans for Educational Testing Reform (10 May 2009). (PDF). Archived from the original (PDF) on 24 July 2011. Retrieved 2009-05-30.
53. Student visa system fraud exposed in BBC investigation
54. ETS. "EXADEP". Retrieved 2010-08-01.

Further reading

• Bickerstaffe, George, "Students Without IT Need Not Apply", Financial Times (London), October 26, 1998, p. 17.
• Brennan, Lisa, "ETS, Kaplan in Legal Skirmish over Test Security", New Jersey Law Journal, January 23, 1995, p. 3.
• Celis, William, III, "Computer Admissions Test Found to Be Ripe for Abuse" New York Times, December 16, 1994.
• Elson, John, "The Test That Everyone Fears", Time, November 12, 1990.
• Honan, William, "Computer Admissions Test to Be Given Less Often", The New York Times, January 4, 1995.
• Kladko, Brian, "Computer Technology Passes Judgment on Students' Essays", Record (Bergen County, N.J.), July 9, 2001.
• Merritt, Jennifer, "Why the Folks at ETS Flunked the Course", Business Week, December 29, 2003, p. 48.
• Nairn, Allan, The Reign of ETS: The Corporation That Makes Up Minds, New York: Ralph Nader, 1980.
• Nissimov, Ron, "SAT Officials to Stop Flagging Disabled Students' Tests", Houston Chronicle, July 22, 2002.
• Nowlin, Sanford, "Standardized Test Giants Lock Horns in Court over Allegedly-Stolen Secrets", San Antonio Express-News, April 8, 2001.
• Owen, David, None of the Above: Behind the Myth of Scholastic Aptitude, Boston: Houghton Mifflin, 1985.
• Sidener, Jonathan, "Educational Testing Service of Princeton, N.J., Develops New Grading System", Arizona Republic, February 1, 1999.
• Tabor, Mary B.W., "Disabled to Get an Extra Chance for S.A.T.s", The New York Times, April 1, 1994.
• "Testing Company Claims State's Bidding Process Is Unfair", Associated Press State & Local Wire, January 6, 2003.
• Vickers, Marcia, "Hate Exams? Here's a Chance to Profit from Them", The New York Times, Business Section, October 5, 1997, p. 4
• Weinstein, David, "ETS to Create Standardized English Test for Chinese Government", Associated Press State & Local Wire, July 9, 2002.
• Williams, Dennis A., "Testers V. Cram Courses", Newsweek, August 12, 1985.
• Winerip, Michael, "No. 2 Pencil Fades as Graduate Exam Moves to Computer", The New York Times, November 15, 1993.

External links

• Official website
• ETS Signs New College Board Contract^{[permanent dead link]}
• 2004 Form 990 from the IRS – lists ETS' executives' incomes
• Educational Testing Service in Europe, Middle-East and Africa
• Americans for Educational Testing Reform (AETR)