"Playing the right tune in VET research" : using cognitive interviewing to improve our instruments

![CDATA[The importance of using research instruments that ask the ‘right’ questions is critical in our VET research. How often have we heard the old adage that a particular piece of research is only as good as its methodology? How often have we examined theses, reviewed journal articles or listened to conference papers where the method section is the area that we really pay attention to and where we stake our verdict as to the worth of the research? How many of us VET researchers, in the agonising throes of writing up, have come to the shattering realisation that it is extremely difficult simply because the material is not producing the goods due to some less than satisfactory data gathering? And how many of us then blame our own (or someone else’s) tools? This paper focuses on the process of developing interview schedules or survey questionnaires and highlights the issues involved in getting the ‘right’ questions. It is based on our own recent research, particularly on our experience in undertaking an NREC project that involved a national survey of private providers of vocational education and training. It is a story of how we were able, eventually, to obtain approval from the Commonwealth Government Statistical Clearing House (SCH), a body which vets survey processes and instruments involving 50 or more organisations run by, or on behalf of, any Commonwealth agency. We used a process of cognitive testing (Willis 1999) – in particular, verbal probing techniques – to fine-tune our instrument before the survey. This is different from conducting a pilot study. The focus is on the cognitive processes that respondents use to answer survey questions rather than on the results received. We will explain what that means, how we used it, the advantages and disadvantages, and what the results were.]]