the Sam Jackson College Experience

Just a quick mention to get out this scary / funny word out: Our friends at ePrep wrote way back in February that SAT tests were being recycled, and no, I don’t mean the paper. FairTest Examiner wrote in April about the same issue:

The College Board cancelled the January 28, 2007 SAT scores of 900 Koreans because some students previously had access to the questions. The reason test items circulated in advance is that the exam was identical to the SAT administered in December 2005.

 

Though the December 2005 SAT was not made public under “Truth in Testing” provisions, which apply to questions and answers from only four of the seven SAT administrations each year, students post items from every exam on the Internet. In addition, some coaching schools have run sophisticated operations to collect entire exams, either by sending in teams of test-takers to memorize the exam or by obtaining entire forms.

 

The College Board offered no reason to believe that the prior-exposure problem was confined to one Asian nation.

Other thoughts: it must suck to be one of those test-question memorizers. I tend to remember the ones I think I missed, which haunt me until I get my scores… at which point they continue to haunt me.

Wait, it gets better. The College Board explanation for the repeated test?

The College Board, which owns the SAT, and its contractor, the Educational Testing Service, justified the test recycling practice by claiming that it costs “probably $350,000″ to create each new exam. But 326,000 students took the January SAT, paying a base registration fee of $41.50. That means test-makers took in more than $13 million at this administration. Given that huge revenue stream and the fat surpluses historically enjoyed by the College Board and ETS), the companies had no credible financial excuse for cutting corners.

The lesson from all this? Taking those practice tests could be even better than you first expected!

Oh, by the way, that official online College Board SAT prep course is still 100% free, check out my post on it from last year.

I was looking through my folder of old posts and I saw one from ePrep mentioning some Senate pressure last December for the ACT. I had a very interesting discussion about the SAT with some midwestern readers last year, and I thought I would just make a quick post for the record to explain why I never took the ACT.

There are two reasons I didn’t take the ACT, aside from its lack of trendiness here in the Northeast.

• Reason number one is that I had good SAT scores in hand by the time I had the opportunity to ever sit for the ACT.
• Reason number two is that although looking at practice books made the ACT look like a better and more fun test than the SAT, it was almost impossible to take from my then-HQ in Exeter, NH. There were one or two changes to take the ACT throughout the entire year–and both meant getting up around 4:30 and driving to Maine. I don’t know how bad the situation is back home around Boston, but the unavailability of testing centers in the Northeast scuttled my desire to take the ACT.

Now you know!

I got 51 problems but a #2 pencil ain’t one.

Booz Allen Hamilton has a funny name but they’re all about serious business. The “global strategy and technology consulting firm” was contracted by the College Board in the fallout of the SAT scoring mistakes from last October. They audited the testing process and made a report, which was hidden away. At least until Senator LaValle used subpoena powers and demanded the report be made public–they were required to do so by New York’s “truth in testing” state laws but the College Board stated that they were “exempt from state law.” I love this choice quote: “Once again The College Board’s approach to problems is to shroud them in secrecy,” Senator LaValle said in a July 13 news release. “Their attitude gives rise to greater questions as to their true purpose.” What do you mean, Senator LaValle? Is there still a problem?

“The board’s lack of accountability in this matter goes to the heart of the arrogance that exists when an organization basically holds a monopoly in a certain market,” LaValle said in a statement following the release of the report. “This report still does not answer why such a large number of testing errors occurred on the October 2005 SAT. In addition, the board has not provided an explanation of why it took several months to inform over 4,000 students of inaccurate test scores.”

Additionally, The National Center for Fair and Open Testing responded to the report by releasing a list of questions that the study did not answer. Among them: What was the root cause of the problem? When did The College Board receive the first request for hand scoring of the October SAT? How many weeks did it take for the board to respond to that request? When did The College Board first discover a broad problem? Why was there a delay between that time and when test takers and others were informed of the problem?

[thanks, Daily Stanford]

Now, I went into that report (it’s a powerpoint presentation) and picked out some of the most fun parts of it. Actually, it’s more a case of me fearmongering by promoting potentially frightening sections of a report which just goes to show how flimsy the SAT is as part of a situation which just demonstrates how much that not-for-profit status is bullshit.

Read the rest of this entry »

On The Use of Background and Ability Profiles to Predict College Student Outcomes and other papers…

I wrote yesterday about an upcoming paper from MSU researchers about prediction of future academic success from current measured background characteristics. Realizing that I could, I requested a prepublication copy of the paper. Fewer than five hours later, at 7:41 this morning, what arrives in my inbox but a response from head researcher and chairperson of the MSU psychology department Dr. Neal Schmitt? So I quickly leapt at the 14,000 word, 56 page paper and accompanying 120 pages of side material. I mostly just skimmed those supplements, but they were interesting too. I see why people want to take psych 101 so much.

Now, I’m not going to pretend that I caught all the references (there were something like 50 papers cited, and I stopped looking them all up after a few pages. Kristof-Brown et al. 2005 (CONSEQUENCES OF INDIVIDUALS’ FIT AT WORK: A META-ANALYSIS OF PERSON–JOB, PERSON–ORGANIZATION, PERSON–GROUP, AND PERSON–SUPERVISOR FIT) was very popular. And I’ll be honest again–even after reading the study, the result tables (e.g., Results of Multivariate Latent Growth Modeling of Fit, Satisfaction, and Student Outcomes, table 4 p.49) didn’t make 100% sense to me. Which is good, because that fact is essentially an argument for graduate study in psychology, and I don’t pretend to be a savant for psychology. I just try to sound out the words and derive meaning from context as best is possible.

It’s not exactly the most accessible of beach reading; the hypotheses are phrased like this: [H1] Perceptions of person-organization fit will lead to satisfaction, which in turn will lead to behavioral outcomes such as intent to turnover and performance. Totally straightforward for the next 48 pages, just like that. (Just kidding, there are actually 7 hypotheses with subsections for some of them.)
Sample was of 2,771 undergraduates, though there was not insignificant attitrition over the course of the study. Related to that, here’s a fun snippet:

As can be seen, the final sample has a smaller proportion of African American students and a correspondingly greater proportion of Caucasians and Asian Americans. The average SAT/ACT scores and high school GPAs of the final sample were higher than those of the initial respondent group.

Blacks were also intentionally over-represented. : \

Read the rest of this entry »

Through “a series of multiple-choice questions,” researchers plan to predict how students will succeed (or not succeed) in college. What test could I be talking about: the SAT, the ACT, the AP, IB, or maybe even the TOEFL? No, it’s some new test–so new it doesn’t have a dehumanizing acronym applied to it yet! Better yet, it doesn’t use language or math to predict aptitude–it uses background information! Awesome! Wait…

(Article truncated here)

update Aug 2: the Research being done at MSU by MSU researchers, MSU has belatedly put up a nice, extra-informative article on the subject HERE.

update Aug 3: I emailed Dr. Schmitt, chair of the MSU psych department, and got the prepublication papers, which are very interesting. I’m reading through them now (~175 pages, so sort of skimming) and will report on them soon. Yeah, I know, excellent use of my time.

Professors Find Ways to Predict Student Success

By KATHLEEN POLESNAK

http://www.statenews.com/article.phtml?pk=37042
The State News

Grade-point averages and SAT scores can only do so much to predict student success at college. That’s why two MSU professors researched other ways to anticipate how student life beyond the books plays into college performance.After looking at students’ interests, background experiences and motivational characteristics from 10 universities during a four-year period, psychology professors Neal Schmitt, Frederick Oswald and a team of undergraduate and graduate students found predictors for student potential, including their likeliness to cheat, drop out of school and attend classes regularly.

The College Board approached Schmitt and Oswald about conducting the study, which it plans to use to enhance college admissions.

“I think it’s a step forward in terms of learning about college performance more broadly and how to admit students that will create a more well-rounded student body,” Oswald said.

Oswald and Schmitt started by examining college mission statements to find which student traits universities desire. They narrowed them down to 12 categories, including artistic and cultural appreciation, social responsibility and citizenship, and career orientation. [...]

Read the rest of this entry »