My research this summer involves a survey. Apparently there’s a secret organization on campus whose mission includes critiquing survey designs (of course there is), and they were very willing to meet with me (of course they were).
(Sometimes this medical center feels like Disney World. First a 24/7 falafal stand, then the realization that I could mint my own money by participating in research studies, and now a magical survey critiquing group appears? Out of thin air? Awesome.)
So we had a meeting today, where I dressed up like a real human being (goodbye, yoga pants! Hello, fancy pants!), outlined my goals, and took notes while we discussed all of the problems with my survey design.
From my perspective, it was incredibly helpful, and I left in a great mood. But I saw several of them later that day, and kept getting comments like:
“Hey! Sorry we sort of picked you to pieces like vultures this morning!” and “Wow, this morning must have been intense for you, huh? You did well, though!”
My first thought was that maybe I had a serious case of tired-face-looking-suspiciously-like-offended-face. My second was that I must’ve missed some mortal insults.
But after discussing it with someone else who was at that meeting, I realized that they’ve all just assumed I actually know something about survey design, and would therefore actually have an ego capable of being wounded by their suggestions.
That concept was a huge revelation. A crazy, nonsensical, huge revelation of untruth. Maybe I should’ve opened the meeting by pointing out that my survey is a pilot that would take years to morph into something publishable. Or, instead, I could have introduced the survey by saying something like:
“Hi. I’m Action Potential, and this is my survey. My guiding rule in writing it was “Fake it ’til you make it.” I literally just clicked around wikipedia until I discovered that the proper phrase for that scale from 1-5 was “Likert scale”, I’m stealing the methods of statistical analysis used by similar surveys, as determined by a PubMed literature review that took all of about 20 minutes. All the survey books in the library had too much math in them, so I just made up my own principles and measurable items and called it good. Thoughts?”
Then again, perhaps not.
Action Potential 
If you want to sound crazy-insider, pronounce it “Lick-urt” and not “Like-urt.” (My thesis advsor knew a guy who knew THE guy…) And make sure to say Likert-TYPE scale unless it’s an honest-to-God Likert scale with 6 points and everything.
Congratulations, you have just gotten the first of my lectures from when I was a Psychometrics (aka Correlational and Experimental Design) TA.* If you ever want to talk about surveys, I am THERE.
*I am also happy to explain in unnecessary detail why comma placement and semicolon use are important.
I should’ve taken your class! SO MUCH KNOWLEDGE. Psychometric scales are really fascinating, but somehow I escaped college without ever considering how much it would help to take a class about them.
Thanks for the Likert info – I will definitely write “Likert-TYPE” scale, then! (By the way, if there’s any chance you would be willing to look over a copy of my revised 30 question survey, I would totally email it to you and would love even the most cursory feedback.)
I know this was posted ages ago, but I thought I’d let you in on a little secret…..90% of psychology research is done just like this! I should know, because I am a researcher in psychology! Except, I get annoyed with likert scales and just throw a psychometric at the problem and viola! Research!
And the hard sciences make fun of us. Pfft.
Seriously, good luck with recruitment!
Actually, I really needed to hear that right now. 🙂 Thanks so much!