|
Joe Q. Resident is sitting in front of you. His paperwork says a tempting “yes,” but your gut says “no.” Experts know that in the end, the gut usually knows best.
“There are many things to consider when selecting residents,” states Dr. Robert Pedowitz, Professor and Chair of the Department of Orthopedics and Sports Medicine at the University of South Florida in Tampa. “The person most involved in this process is usually the residency program director. That person coordinates selection in a very competitive field. In 2007 while 568 U.S. students matched in orthopedics, there were 126 who did not.”
With all of the individuals interested in orthopedic residencies, it is critical that programs have a streamlined way of going through the process. Says Dr. Pedowitz, “Most residency programs use The Electronic Residency Application Services (ERAS), a data warehouse that constitutes a huge improvement in the system. We used to get tons of paper and applications mailed to us directly, meaning that things sometimes got lost. Now we have this electronic system that distributes information to residencies so that the program coordinator can organize all of the information into files. ERAS has functions that allow programs to screen applicants by what they think are the most important criteria. For example, maybe a program only wants people who have at least five letters of recommendation or have a certain board score or above.”
As an example of the competitive nature of orthopedics, Dr. Pedowitz notes, “This year we have received nearly 500 applications for the four residency positions we have available. Our initial job is to take that number down to something more manageable in order to do a detailed review of a small group. We typically interview 40 - 50 people and organize all the interviews into a couple of days.”
Getting the big numbers down to a workable few involves strategy and data. Says Dr. Pedowitz, “There is some literature saying that certain things about medical students correlate with their performance as residents. We have learned that performance on clinical rotations in the third year of medical school and the specific number of times students have received honors during their clinical rotations tend to correlate with clinical performance in residency. If your objective is to recruit people who are scientifically oriented you might focus on grades and performance in the first two years. If your goal is to find good clinicians and people who will work well in a hospital, then you should focus more on clinical performance in the third year. When I ran the residency program at the University of California, San Diego, we had a required research year, meaning that our recruitment efforts also focused on research experience and productivity. It all depends on your programmatic priorities.”
Even though residency recruitment has been going on for many years, there are still improvements that can be made to the process. “Take letters of recommendations and deans’ letters,” says Dr. Pedowitz. “Those are not normalized around the country so it’s hard to compare them across the board. In the absence of good objective ways to compare things, we lean on the test scores because they are the only fairly uniform measurement that every medical student has to produce. Although you can compare board scores from a test taken at the end of the second year, it’s not automatically the best strategy because some people may be great doctors, but not good test takers or vice versa. There has been a movement within the last five years to make letters of recommendation and deans’ letters more uniform, with a more consistent format and way of presenting information. Another measure we use is the quality of the school that applicants come from. We use the U.S. News & World Report rankings to help formulate a gestalt of the reputation and quality of the medical schools. One area that needs significant improvement is how we assess technical skills. Generally speaking we are poor at identifying how good an applicant is with his or her hands, as well as that person’s three-dimensional skills. A few programs use some test of psychomotor skills involving a puzzle or having the applicants perform a task with their hands when they are stressed. I am working on developing a virtual reality simulator to be used during surgical training. Although I would hope to use it to assess an applicant’s surgical potential, this would be controversial because it is not yet validated.”
And what are some misconceptions about the recruitment process? Says Dr. Pedowitz, “A new residency director may overemphasize performance on tests or grades as being a good predictor of success as a resident. Another mistake would be to try to fully assess ethics and personality from applications and short interviews. While you certainly want to be on the lookout for teamwork, manners, and other human qualities, it can be pretty hard to get that from paperwork and 15 minutes with the applicant. Personally speaking, I have been shocked at the number of times my impression changes from reading someone’s application to when I have them in the interview. The most important thing is to have, in addition to any orthopedic-related conversation, a general conversation that is on another topic. You should stay tuned in and trust your instincts. I recall an interview with a medical student who bragged that he had spent a year abroad as a bartender. More disturbing than the content was the way he presented the experience—it was immature and made me think, ‘As a rule, does this person do what is most fun or what is good for his career development?’ It showed poor insight and judgment on his part. Other interviewees are maniacally intense, with flights of ideas and pressured speech. Then there are people who are so completely arrogant; contrast those with the applicants who are so painfully shy that they are noncommunicative. Perhaps the most egregious situation is to find that someone has falsified information on his or her application. This becomes evident when, for example, someone lists a research project, but then when you ask about it, they have no idea what it was about.”
Sometimes informal situations can yield the most information. “We observe interactions between applicants and other people during lunches and dinners,” states Dr. Pedowitz. “There is also a party where the residents can talk informally with applicants. The feedback we obtain from the existing residents is invaluable. Also helpful are the comments we hear from our administrative staff. It’s really telling when an applicant is rude to an administrative person, but charming with an interviewer.”
Also intimately familiar with the resident selection process is Dr. Ken Yamaguchi, the Sam and Marilyn Fox Distinguished Professor of Orthopaedic Surgery at Washington University School of Medicine in St. Louis. “There are two issues when trying to obtain the best residents for your programs,” states Dr. Yamaguchi. “One is selection, meaning that when you evaluate medical students you want to be able to select well, thus ensuring that you rank the people most likely to succeed. The other issue is recruitment, i.e., the part when you attempt to convince a student of the merits of your program. The two things are not mutually exclusive, either. For example, by providing the most professional, fair selection process, even if it is rigorous, you tend to recruit the best students. They tend to be impressed by programs that have a professional appearance and give the impression to prospective residents that the program is run in the best way possible. The top applicants often want to go to a program that is the hardest to match into.”
Elaborating further on the selection process, Dr. Yamaguchi notes, “You want to make sure the applicant is both a good one and a good fit for your program. Intelligence is usually not an issue as applicants who warrant an interview are often top students; however, what we call the affective domain or emotional intelligence, usually is. Most applicants come to us without much more information than what is on paper, so we don’t know about their emotional intelligence. We are not concerned about this arena with some applicants because we know them. They may have rotated with us, we may really know the program they came from, or I may have received a call from someone I know who is recommending the person. For the other applicants, the only other way to determine affective domain is the interview. This has been shown to be very inaccurate and perhaps even unfair. We begin with an initial screening based on grades and whether or not the person has achieved honors in medicine and surgery. While board scores are a minor criterion, we do look for a score around 220. But even if someone has low board scores and no honors, they could still get accepted if they have rotated with us, have great letters of recommendation, and people who we know call and tell us that so-and-so is a great applicant.”
The other side of the coin, recruitment, requires that programs put their best foot forward. But then that’s what you do every day, right? Says Dr. Yamaguchi, “Fundamental to recruiting is that you have a good program to begin with. Clean up any problems with your program before you begin the recruitment process. If you have unhappy residents find out why. On the day of interviews, what residents say about your program is probably the most important element of recruitment. And if you ‘plant’ happy residents, medical students are smart enough to see that the residents you are putting forth are giving a biased view of the program. As a rule, we have every resident available on interview day to show that we’re not trying to hide things or present a skewed perspective. The night before interviews we have a dinner so the applicants can meet residents in a social setting. This is particularly important for residencies that aren’t located in the big cities such as New York or Boston. In our case, we want to show them that St. Louis is a fun, interesting place to live.”
Continues Dr. Yamaguchi, “Having spent time with the applicants, the residents are also well positioned to provide opinions on the students. We take those comments very seriously. In fact, the residents have much to gain from participating because they have to work with the people we choose and would feel the consequences of poor selection. My advice would be to have well-organized interviews, have a professional demeanor, and not make the interviews overly rigorous. Applicants nowadays are turned off by interviews that are too grueling. Focus on assessing the affective domain as most applicants are exceptionally bright, meaning that you can leave that issue aside. How are their interpersonal skills? What about the person’s work ethic, sense of compassion, teamwork, and stress management skills? It’s true that certain programs have different personalities which fit best with specific people’s emotional intelligence. And no matter how good someone looks on paper, you don’t want them if they don’t fit from a personal standpoint.”
But how to assess something as intangible as stress management skills? Says Dr. Yamaguchi, “We turned to professional consultants outside the orthopedics field. In the business world, measuring the affective domain is a science and is routinely used as a screening tool. We went to one of the leaders in this industry, Psychological Associates, which told us that contrary to the business world, there were no data on what personality tests and what answers would predict future performance as a surgeon. We then obtained a grant to investigate this and have been collecting data for years now. To date, many of the things we found are predictable. For example, if you are someone who can take criticism well, are more people-centered than self-centered, are internally confident, and can handle stress well, you will perform well as a surgeon. These things can be tested and assessed with these questions pretty accurately. In another year we will have a critical mass of data with statistical significance. As of this year we have begun using these tests as a general screening tool. So as to protect the applicants’ privacy, the tests are blinded. During the selection process when we’re looking at the top applicants we examine the information from the test to help us make a choice.”
“Our day with applicants,” notes Dr. Yamaguchi, “is organized as follows: the lecture series is usually split into three parts, including information about the program and philosophy, information on St. Louis, and a lecture on the personality questionnaires. Students can get nervous about these, so we let them know that we are not testing for clinical psychopathology, but rather to understand their goals and objectives and what motivates them. The questionnaires are essentially standardized interviews. Following this, one group goes on a tour while the other group does the interviews.”
Honing one’s interview skills is an art crafted over time. “To start with, be familiar with legal regulations regarding interviews. For example, though they may sound like innocent questions, it is unlawful to ask about age or marital status. I usually begin with a broad question such as, ‘Tell me about yourself.’ This allows me to see what they choose to mention first and what they want to highlight. I also ask about their strengths and weaknesses and wait to see how candid they are. Someone may say, ‘My weakness is being quiet, but it’s also a strength because it allows me to think.’ That seems canned. I would rather hear complete honesty, such as, ‘I wish I were better at this or that.’ This also shows the person is capable of introspection. Occasionally you have applicants who are exceptional in that they’ve never had any stress in their lives. They have seemingly had nothing but success. In that case I ask, ‘Have you ever felt failure or disappointment? What did you learn and how did you handle it?’ However, if the applicant looks uncomfortable the interviewer should move on to something else. I don’t believe that a short interview format is an appropriate forum to put stress on the applicant. It is unlikely that useful information will result and the applicant will probably not want to come there anymore, anyway.”
Finding the right people for your residency is a balance of selection (‘Who is the best?’) and recruitment (‘Hey, we found the best, let’s make sure we get them.’). It’s also a balance of using the best data and strategies available with trusting your gut instinct. The combination will rarely lead you astray.
|