Thursday, February 21, 2013

The Right Stuff

When I started this blog last fall I had a number of questions and concerns.  Would I find enough to write about?  Could I discipline myself to write and post on a regular basis?  Would anyone want to read anything I wrote? 

I have preliminary answers to those questions, but the question that I am still struggling with is whether I will end up repeating myself.  I am cursed to be inside my own head on a daily basis, and my shtick therefore gets old, and I worry that my writings and my presentations are all variations on the same themes.

I have devoted more energy and space to issues surrounding misreporting of admissions data than I anticipated or wanted, and I am at the point where I may ignore the next scandal of that type because I have nothing left to say.  The other broad theme that I have found myself drawn to repeatedly involves the question, “Are we measuring the right stuff?”  (I use “stuff” rather than “things” in homage to Tom Wolfe, my school’s most famous alumnus.)

Attention to assessment and metrics is pertinent for college admissions in several different respects.  Demographic changes will mean that during the next decade the going-to-college population will change in profound ways.  The NACAC Strategic Plan calls for increased access to post-secondary education, but also recognizes that true access requires the conditions for success.  We need a better understanding of the qualities that lead to success, qualities such as persistence and motivation that have come to be lumped together as “grit.”  For years my goal was to devise a standardized test that measures motivation, and I vowed that if successful, I would share it not in a forum such as this, but rather during an appearance on Oprah.  Unfortunately, I was not motivated (or smart) enough to create such a test, and now my window of opportunity to appear on Oprah has closed.

Measuring the right stuff will also be important for institutions and perhaps even publications that rank colleges.  At some point it will no longer be enough to use input measures that measure popularity rather than educational benefit, especially when those measures are so easy to manipulate.  The Gen X parent is going to demand data and evidence to show educational value, and we can expect both the federal and state governments to pressure colleges to develop metrics that measure outcomes or added value (gainful employment) from a college education.

The other issue is the movement within education toward what have become known as 21st century skills, skills such as creativity, collaboration, global consciousness, and technological savvy.  Ignoring the question of how far into the 21st century we will be before the term “21st century skills” is no longer seen as novel, I haven’t seen much evidence that admissions offices are paying attention.  Several years ago I attended a counselor breakfast sponsored by five of the nation’s very best universities.  When I asked what discussion was taking place regarding 21st century skills on their campuses, the answer was “none.”  There are exceptions, of course.  During his tenure as Dean of Arts and Sciences at Tufts, Robert Sternberg (currently Provost at Oklahoma State University) worked with Dean of Undergraduate Admissions Lee Coffin to institute the Kaleidoscope Project, which added a series of application essays to measure creativity and other skills and to augment the information provided by traditional measures such as high school grades and SAT scores.  Sternberg’s 2010 book, College Admissions for the 21st Century, is an interesting read on both 21st century skills and college admissions.

Recently I have encountered two encouraging signs of progress in thinking about how we measure “The Right Stuff” and communicate what is important about a college education.

The first was a conference held last month at the University of Southern California’s Center for Enrollment Research, Policy, and Practice.  The title of the conference was “Attributes That Matter: Beyond the Usual in College Admission and Success,” and it focused on non-cognitive assessment.  A front-page story in the January 18 issue of The Chronicle of Higher Education talks about the conference in detail, and more information is also available on the Center’s website and in a number of posts on the Chronicle’s Head Count blog.

The second is a new publication produced by Augustana College in Illinois.  “The Augustana Story” is an interesting attempt to present relevant data in a transparent way.  I originally saw reference to it in a post in the Head Count blog and then received an e-mail from Kent Barnds, Augustana’s Vice-President for Enrollment, Communications and Planning.

The publication has data and graphs with information relevant to four different areas:

Students: Four-year graduation rate; first-to-second year retention; student housing data; admission overlap schools; scores on the Collegiate Learning Assessment (CLA) and CAAP Critical Thinking test.   

Faculty:  Full-time and part-time faculty; percentage of tenured and tenure-track faculty; breakdown of class sizes; percentage of faculty who advise and average advising load; three-year data on percentage of faculty engaged in international study, internships, undergraduate research, service learning, and other programs.

Investment:  Pie charts for revenue and expenses; Mid 50% ranges for administrator and faculty salaries; percentage of students receiving various forms of financial aid with average amounts; cost to enroll a student and the freshman class; funds invested and student participation in the Augie Choice program that gives each student participant $2000 toward faculty-led research, study abroad, or an internship.

Alumni:  Placement stats nine months after graduation; average indebtedness; default rate on student loans; alumni survey results.

There is an interesting discussion to be had on what data is meaningful and why, but I applaud Kent and Augustana for their efforts to tell their story in a different way.  Maybe some day we’ll see a new Common Data Set focused on output metrics or a tool on college websites similar to the Net Price Calculator with information about value-added.  It’s time for us to think more broadly about The Right Stuff, both for success in college and for communicating the value of college.


  1. Jim,

    One interesting set of measures that I have tried to follow is Emotional Intelligence or EQ. My understanding from studies I have read is that there is a good match between one's level of EQ and their self-reported happiness scale.

    It has been a few years since I have followed the research around EQ, but I wonder if there are now EQ measures that can better tell us if students have the "Right Stuff". Because, after all, it isn't all about data from colleges, families or job placement. Whether a student is successful in college or not also depends (heavily I would argue) on how well they can regulate their emotions, get along with others, express how they are feeling--all good social/emotional skills. I even wonder if there has been a tool developed that does a better job of predicting how much success a person might have in college when looking at EQ. And, if that prediction factor has been compared to the SAT prediction factor. I also wonder how closely connected are family incomes to a student's EQ and how much could a student's EQ account for any gaps in their education?

    Sometimes I wonder if we are missing the boat when it comes to our focus on trying to encourage more students to go to college. We talk about the importance of it, the opportunities they will have, the level of education jobs need now, the money they will make and so on. We also talk about a shortage of counselors doing college and career education and that we need to start that conversation earlier. All of the above I am not opposed to. However, if a student has trouble regulating his/her emotions, the rest of the stuff doesn't have as much impact.

    So, I wonder--shouldn't we also be focusing on and making noise about teaching social/emotional skills within our schools so students can develop a higher Emotional Intelligence? Would this lead to increasing success in college and in life?

  2. It is in the nature of uneducated to wind up in an arguments leading to harm others and most to himself. Education would rather control your temper towards a more productive and peaceful goal through negotiations and productive here