Backup - Communicating with Recent Graduates: Making Sense of the GDS
The Graduate Destination Survey (GDS), with the Course Experience Questionnaire (CEQ) or Postgraduate Research Experience Questionnaire (PREQ) attached, is sent to all new Australian university graduates every year. The aim is to take a snapshot of what recent graduates are doing in terms of their post-graduation activities and to gather information from them about their higher education experience.
The CEQ and the PREQ
The Course Experience Questionnaire (CEQ)
The CEQ is a primary source of information for those charting quality measures in higher education. However, these figures should always be used with caution and insight. There is a time delay between the collection of such data and the production of results and this can dilute the usefulness of the information. For example, the graduate satisfaction figures produced below were gathered in 2001 from people who had completed their degrees in 2000. Respondents would have based their opinions on their experiences of the three, four or more years previous to that.
In that period, course managers could have been acting on prior CEQ feedback and fine-tuning their programs. So, for some courses at some institutions, the latest CEQ figures might not be a timely reflection of student experience, lacking the detailed items required to allow deeper analysis of issues specific to the institution.
However, it has always been argued by the GCCA that the instrument represents a macro-level overview of teaching quality and should form part of a suite of instruments used within an institution. The other components would be questions and scales developed within the institution that would get at more specific issues and put flesh on the CEQ figures. This gives institutions an incentive to develop internal quality assurance tools that lock the CEQ into their own systems and build on the data in a way that is meaningful for the improvement of teaching for that institution.
In addition to the issue of the timeliness of the figures, their relative complexity can make them difficult to interpret with accuracy. For example, Table 3 shows institutions ranked in terms the percentage of respondents who agreed or strongly agreed with Good Teaching Scale (GTS) items. The figures in this table seem to indicate that BondUniversity and CentralQueenslandUniversity received relatively high ratings in terms of the GTS. However it can be seen that these figures were based on very few responses (as were the figures for the next three institutions). Similarly, the lowest figure for agreement, for UNSW, was also based on few responses only. Figures based on such small response numbers cannot be assumed to be representative of the opinions of the whole group of graduates and should be ignored or treated with caution.
Even in cases where there are a sufficient number of responses, the figures need to be interpreted with caution. For example, while CurtinUniversity had an agreement figure close to the national average, its disagreement figure was very low. Curtin had a lot of cases fall into the middle ‘undecided’ category. Similarly, the agreement figure for the University of Adelaide was low. However, the disagreement figure was very close to the national average. Again, a lot of cases fell into the ‘undecided’ category.
At the other end of the table, Swinburne and VictoriaUniversity rated relatively well in terms of agreement but were near the national average in terms of disagreement. In this case, low percentages are found in the ‘undecided’ category. These examples all indicate that CEQ figures need to be treated with caution and a basic understanding of the intentions of the instrument and the methods of analysis. Users should consult the annual CEQ reports (GCCA 2002c) for background information.
Attempts to condense such figures to single digits or star ratings certainly have the effect of reducing the complexity of interpretation, but this is at a cost to the richness of the information.
The Postgraduate Research Experience Questionnaire (PREQ)
The PREQ is still in development in terms of optimal data analysis methods, however many of the issues discussed in relation to the CEQ are pertinent to the PREQ.
|
