the Sam Jackson College Experience

On The Use of Background and Ability Profiles to Predict College Student Outcomes and other papers...

I wrote yesterday about an upcoming paper from MSU researchers about prediction of future academic success from current measured background characteristics. Realizing that I could, I requested a prepublication copy of the paper. Fewer than five hours later, at 7:41 this morning, what arrives in my inbox but a response from head researcher and chairperson of the MSU psychology department Dr. Neal Schmitt? So I quickly leapt at the 14,000 word, 56 page paper and accompanying 120 pages of side material. I mostly just skimmed those supplements, but they were interesting too. I see why people want to take psych 101 so much.

Now, I'm not going to pretend that I caught all the references (there were something like 50 papers cited, and I stopped looking them all up after a few pages. Kristof-Brown et al. 2005 (CONSEQUENCES OF INDIVIDUALS' FIT AT WORK: A META-ANALYSIS OF PERSON–JOB, PERSON–ORGANIZATION, PERSON–GROUP, AND PERSON–SUPERVISOR FIT) was very popular. And I'll be honest again--even after reading the study, the result tables (e.g., Results of Multivariate Latent Growth Modeling of Fit, Satisfaction, and Student Outcomes, table 4 p.49) didn't make 100% sense to me. Which is good, because that fact is essentially an argument for graduate study in psychology, and I don't pretend to be a savant for psychology. I just try to sound out the words and derive meaning from context as best is possible.

It's not exactly the most accessible of beach reading; the hypotheses are phrased like this: [H1] Perceptions of person-organization fit will lead to satisfaction, which in turn will lead to behavioral outcomes such as intent to turnover and performance. Totally straightforward for the next 48 pages, just like that. (Just kidding, there are actually 7 hypotheses with subsections for some of them.)
Sample was of 2,771 undergraduates, though there was not insignificant attitrition over the course of the study. Related to that, here's a fun snippet:

As can be seen, the final sample has a smaller proportion of African American students and a correspondingly greater proportion of Caucasians and Asian Americans. The average SAT/ACT scores and high school GPAs of the final sample were higher than those of the initial respondent group.

Blacks were also intentionally over-represented. : \

In any case, here is what the study sounds like:

In similar fashion, we believe that those who are highly extraverted will perceive increased fit and satisfaction with college life over time. At most universities, including those attended by the students in our study, most college freshmen spend their first year living in close proximity in dormitories. In addition, in most dorms, students will have one or more roommates, will likely share bathrooms and other facilities, eat some or all of their meals in college cafeterias and take part in social activities organized by their resident assistant. Those students who are extraverted are more likely to make friends in these situations and perceive that they fit in this atmosphere, whereas less extraverted students will not enjoy the social closeness and lack of privacy they tend to encounter in dormitory life. This close living proximity inevitably requires tolerance of others’ behavior and idiosyncrasies as well as willingness to share and compromise, so we believe that Agreeableness will also be positively associated with perceptions of fit and satisfaction with students’ social situation. Hence it seems reasonable to pose the following hypothesis:
H7: Extraversion and Agreeableness should be related positively to changes in perceptions of social fit and social satisfaction.

Now, the bolded section (emphasis mine) does illustrate the fact that not all of the hypotheses here are exactly mind-blowing. Still, very interesting stuff, as everything has nice clean data behind it. Some of the hypotheses were more spectacular than the seventh one, above.

In any case, I do take issue with some of the logic and reasoning here. It doesn't seem airtight in many places. For instance:

Skipping classes and being late to classes may be proximal evidence of physical and psychological withdrawal from the university community, which in turn may be correlated with eventual transfer to another college or with dropping out of college entirely.

This despite recent studies which have shown a huge portion of college populations to skip class fairly regularly; it's just a fact of college life, when you can make your own schedule. Is it becuase they're listening to the lectures on their iPods? Maybe, but it might just be that they're extracting themselves from class to do other things around the college community. I'm not saying that many people cut class for community service, but it isn't always just for sleeping in.

Some of it was just a little bit... sketchy.

To measure CWB, we adapted items from Bennett and Robinson’s (2000) organizational deviance scale. Items reflect targets of deviance that are both interpersonal (e.g., “made fun of another student at your school”) and organizational (e.g., “stuck gum under your desk”) in nature.

My favorite part was the citing of one question on the "vocational interest" section of their survey which had "Refinish Furniture" as an option. It did not go unselected as a favorite pasttime. After the abstract and hypotheses and methodology there was another... 30 pages or so, of math and statistics, which wasn't very exciting. After that, some charts. Again, if you want to know more, I can definitely send along those papers or you can contact Dr. Schmitt et al. yourself. It's interesting, but not far enough along--and I hope it never is. The title to one of the accompanying papers alone tells us just how frightened we should be of this sort of standardization-heavy approach to admissions: The Use of Background and Ability Profiles to Predict College Student Outcomes. That doesn't sound so friendly to me... I'll just leave you with the abstract, which is fairly fuzzy:

College admissions tests predict college performance well, particularly first-year grade point average (Kuncel, Hezlett, & Ones, 2001, 2004). Noncognitive measures such as those measuring interests, background experiences, and motivational characteristics may add incremental validity to traditional cognitive college admissions tests and high school grade point average, in that they assess a broader range of dimensions reflecting the potential of college students – such as those measuring leadership, interpersonal skills, and ethics.

Oh, and one more thing, from one more paper (Developing a Biodata Measure and Situational Judgment Inventory as Predictors of College Student Performance,

Colleges and universities seek out the best students possible for their institutions, where ‘best’ can be defined in many ways. Traditionally, selection systems for college admissions typically use standardized tests of verbal and mathematical skills, and possibly records of achievement in specific subject matter areas. Such systems have worked well for decades ...

Start laughing and crying whenever you like.

Plus!

Bonus featurette from an AIM chat with Tom Gram about the study:

13:32:07 Tom Gram: u got that paper already?
13:32:56 me: check your email
13:33:47 me: schmitt hit me up at 741 this morning, less than 5 hours after I sent it
13:34:18 Tom Gram: thats hardcore
13:34:37 me: yeah he's chair of the department too
13:34:40 me: guess he has time to respond to emails
13:35:00 me: I thought about signing my email "future psych major" but decided against it
13:35:52 Tom Gram: haha, your email is quite friendly and enthusiastic about his work
13:36:06 me: that's how I got such a quick reply, of course
13:36:29 me: "dude u sux y u gotta be hatin on HS studnets" does not a prompt reply bring
13:41:21 Tom Gram: haha i like this so far, hes basically like....there was evidence that students' liking toward the learning enviornment affects performance
13:41:45 me: satisfaction correlating with a positive change in environmental fit perception, yes

see also this one from Mike Lee:
Mike Lee: I thought the whole personality thing was supposed to be judged by the essays/extracurrics
me: oh god no
me: that's not STANDARDIZED