Mixing Quantitative and Qualitative Research
Thomas E. Macnamara, Kenneth E. D'Silva
Faculty of Business, Computing & Information Management
London South Bank University, London, United Kingdom
Abstract: The first segment of the paper describes and evaluates some lessons learned from a recent empirical research exercise conducted by a six member multidisciplinary team of researchers at London South Bank University (LSBU). The researchers used a mixture of qualitative and quantitative methods in order to assess policy issues within a major United Kingdom public sector organisation. The paper highlights some of the benefits and problems of integrating different research paradigms into a single research team. The research led to the development of a useful team method, devised to improve the objectivity of interview data. This team-based research method is likely to be of general utility and application.
Additionally, from a teaching research methods perspective, the second segment of the paper describes and evaluates a simple tutorial teaching exercise conducted at LSBU in order to help highly numerate (quantitative) students adapt to qualitative type thinking. In doing so, the paper provides a further contribution to help bridge the gap between the two major (quantitative and qualitative) research orientations.
Keywords: Multidisciplinary, Paradigms, Objectivity, Teaching, Team-working.
The nature and quality of a student's learning is influenced by the nature and quality of the teacher's teaching style (includes pedagogy, approach and techniques). This suggests that, to the extent possible, teaching styles should take regard for and accommodate students' backgrounds (Dunn & Dunn, 2000).
The set of 18 students who participated in this Research Methods teaching exercise would be considered "mature" by most standards and revealed the following features:
average age of 28 years (oldest 36 and youngest 25 years)
a third of the students female
international group, with citizens of 8 countries represented
all postgraduates (studying for a Master's degree in Accounting with Finance)
many being exposed to research concepts and techniques for the first time
preparation being given to enable them to, in due course, write Masters' dissertations.
Of particular relevance and importance to the exercise was the fact that all students were (United Kingdom) qualified professional accountants in the process of developing proposals for dissertations to be written in Accounting and/or Finance (strands of the Social Sciences). Equally, it is established that the skills demanded of accountants include the ability to interpret not only quantitative information (such as balance sheets) but also qualitative information (such as the Directors Report) contained in a company's financial statements.
Prior teaching to the students had reinforced the view that, because of their accounting qualification and better than average numeracy, they readily appreciated and related to quantitative research techniques. However, this was not always the case when exposing them to qualitative research techniques. This provoked a need for the students to be exposed to qualitative research techniques whose nature and rationale they could more readily appreciate and for which they could see a possible application in their everyday professional (accounting related) lives – i.e. a case of mixing qualitative techniques with the quantitatively disposed.
Thus, a further focus of this paper is in "Teaching Research Methods" and, in part, its genesis lies in ideas suggested by Dey (1993:1) who rightly contends that there "is no one kind of qualitative data analysis, but rather a variety of approaches, related to the different perspectives and purposes of researchers". In order that such methods have as wide an appeal as possible, when teaching research methods, Dey (1993) suggests using examples drawn from "everyday life" possibly including the use of humour. Though one must admit that an exclusive humour focused approach will have some limitations, as humour is often culturally dependent. While developing these ideas, Dey (1993) and the authors acknowledge use of the helpful classification typology for qualitative research methods developed by Tesch (1990:58).
This segment of the paper has five key sections. The first presents the teaching exercise within its context. The second sets out the objectives of the exercise while the third explains the method and procedures employed in its conduct. The fourth section considers select findings generated within and from the exercise. The fifth and final section of this segment of the paper offers some conclusions with possible interpretations and related policy considerations.
Given their backgrounds and prior formation as accountants, students did not appear to readily "take to" information packaged primarily in words or of a more qualitative nature. Additionally, there was some reluctance to accept that such (non-quantitative) information could be relevant to their professional lives.
Thus, in order to bring out a few important aspects of qualitative research, the 18 students were asked to consider, in a prescribed manner, the words of a thought provoking and, for some, poignant song written by Eric Bogle entitled " The Band Played Waltzing Matilda" 1.
In esence, the song deals with war and its consequences. It was selected for particular consideration because, it was judged, most students would have had some exposure to the phenomenon of war - at least in theory. Unexpectedly, later, during general discussion of the somng, it became apparent that a small group of students had had first hand experiences of war by them being subjected to the immediate effects of a war, or, for a smaller group, by them having participated personally in a war. To that extent, the use of this particular song as the basis for the teaching exercise is likely to have provoked even deeper thinking by the first group of students and to have made it more personal for the other.
"The Band Played Waltzing Matilda" tells of the experiences of a happy-go-lucky, carefree Australian youth who, because of the needs of the First World War (1914-1918), was plucked from the "dusty outback" of Australia to serve in the Dardanelles and so became part of the Gallipoli campaign. While taking part in that campaign, the youth is exposed to the harsh realities of (then) war and is permanently maimed by suffering the loss of a leg. Consequently, he is "shipped" home to Australia to live a relatively unfulfilled life (as suggested by the words of the song). The song concludes with the (now) old man's questioning the utility (futility?) of war, reinforced yearly by poignant reference to the "victorious" ANZAC Day parades he witnesses each April 26th - still a public holiday in Australia and New Zealand.
Having regard for all the above, the objectives of the teaching exercise were set out to reinforce within students or to give them a realisation that:
Some types of qualitative research call for an attempt to discover meaning (rather than fact) within a particular presentation or submission.
Consistent with theories of Discourse Analysis (DA), meaning is a function of perception and, if perceptions vary, it is likely that interpretations of meaning will also do so.
An element of subjectivity is therefore always likely to be present in several applications of qualitatively-based social science research.
The selection of (an) appropriate research method(s) within a given research situation can only be done in conjunction with a good understanding of the objective(s) of the research.
1.4Method and procedures
Students were given no prior information that the exercise would be conducted and so it is unlikely any form of non-participation bias was at hand (by students consciously deciding not to be present when the exercise was conducted). Thus, the 18 students who participated in the class exercise were all those present on the day it was conducted.
The tutor first explained to the students that a class exercise would be conducted as part of the day's learning activities and this would involve consideration of the words of a particular song. The tutor then gave a limited contextual explanation of the setting of the song - just so that students had some understanding of its historical context.
On a virtual random basis (though mainly by reference to where they happened to be seated that day), the 18 students were paired into nine pairs. Then, each student was given a sheet with the words of the subject song printed on it and only after reading and considering it through, was requested to discuss views on the text of the song with his/her partner. Students were given about fifteen minutes to individually consider the words of the song and a further ten minutes to freely discuss (i.e. in a non-prescribed manner) them between themselves.
Then, within each pair of students, the first student was asked to thoughtfully re-consider the words of the song to determine the number of occurrences that, in his/her judgement served primarily to convey description. Equally, the second student within each pair was asked to thoughtfully consider the words of the song to determine the number of occurrences that, in his/her judgement served primarily to evoke emotion.
On expiration of a further ten minutes of consideration time, the first student in each of the nine pairs was asked to state the number of occurrences that he/she determined present in the words of the song and which served primarily to convey description. Then, the second student in each of the nine pairs was asked to state the number of occurrences that he/she determined were present in the words of the song and which served primarily to evoke emotion.
A register of the two types of occurrences for each was pair was maintained and a generalised class discussion as to why the number of occurrences for each pair was so few (or many) in relation to other pairs was actively encouraged by the tutor. This discussion proved to be quite animated and elicited a variety of justifiable views. But perhaps, more importantly, it enabled students to assess their own personal views on the occurrences considered by them in relation to the comparable views held by their peers.
The discussion enabled the tutor to reveal that what the students had engaged in was a limited form of DA. And, it was indicated to students, that DA itself is characterised mainly by a way of approaching and thinking about a problem or issue. So, to that extent, DA was neither a qualitative nor a quantitative research method but a manner of questioning the basic assumptions of both research methods. Students were invited to see DA as a means of gaining a higher awareness into the hidden motivations in others (and oneself) so enabling a solution to concrete problems - not so much by providing answers but by prompting one to ask related ontological and/or epistemological questions.
Students were reminded that while the term "deconstruction" is associated with post-modern thinking, DA is in fact an invitation to think critically. On that basis, critical thinking is not really a post-modern approach. For, already in the early part of the last century Dewey (1933:9) defined the nature of critical reflection as an "active, persistent and careful consideration of any belief or supposed form of knowledge in the light of grounds that support it and the further conclusion to which it tends".
Equally, students were invited to see the exercise they had engaged in as a form of DA, which can be seen as nothing more than a de-constructive reading/interpretation of a problem or text. This "deconstructive" approach, i.e. to accept virtually nothing at face value but to "deconstruct" by uncovering the hidden, multifaceted meanings of a text and thereby reveal the codes of power. Such an approach would be consistent with the thoughts of Jacques Derrida and particularly Michel Foucault who saw language as a product of power. As such, truth (which could be seen as the ultimate quest of all researchers) was itself not fixed nor constant. Instead, Foucault would argue, truth is relative and flexible, defined by whoever is in charge.
Finally, after about twenty minutes of such generalised class discussion/instruction, each student was asked to anonymously (i.e. name withheld basis) briefly respond on a pre-printed questionnaire form to the following four questions:
What did you think of the class exercise that required you to work with a colleague in order to consider and then textually analyse the words of the subject song?
What did you learn about research methods (not the contents of the song itself) from the class exercise?
Did you find the exercise helpful and, if so, why?
Would you be able to conduct a similar exercise (with due amendments) to (say) a Director's or Chairman's Report as presented in a set of annual financial statements?
1.5Selected results and comments:
These can be categorised as those arising from the exercise itself and those provided by the students as a result of the exercise.
Selected results arising from the exercise itself:
Views of instances intended to primarily provide description was variable - the minimum being 15 and the maximum being 27 (with an average of 19).
Views of instances intended to primarily convey emotion was highly variable - the minimum being 9 and the maximum being 35 (with an average of 20).
Variability in relation to descriptive occurrences was much less than variability in relation to emotive occurrences.
1.6Selected student comments on the exercise after had been completed:
‘In qualitative research there is really no right or wrong interpretations. Two differing interpretations can be valid, so long as both have justified reasoning behind them.’
Qualitative research methods can be much more difficult than quantitative methods when conducting financial research.’
‘Interpretations of qualitative research depend on the researcher - in particular how the researcher understands the facts and his mind set.’
‘Qualitative findings can often be captured within quantitative terms so the two research approaches mutually support each other.’
‘There is a greater potential for differing outcomes within qualitative research than in quantitative research and, in general, qualitative methods are more challenging.’
‘Discussion can help bring out why others perceive the very same words in a different light.’
‘Qualitative research methods give more room for the researcher’s opinions. Such researchers have a strong influence on research decisions, assumptions and conclusions. Quantitative researchers do not have as much scope for their personal ideas and opinions’.
‘This somewhat totally unrelated (to accounting and/or finance) exercise was indeed relevant to thinking about methods one could use when conducting accounting and/or finance research.’
1.7Benefits stated by participants as a result of participating in the exercise included enhancing or enabling:
Team playing skills.
Clarity of thinking through discussion with other participants and regard for the evidence at hand.
Development of communication skills - a function of paired and class discussion.
Participants to detect underlying themes in parcels of text (as in a Directors' Report).
1.8Conclusion and policy consideration
Overall, the exercise proved successful. One participant described it as ‘very useful’ and stated that he would "highly recommend it for future similar sessions". Discussion conducted while the exercise was in-process and the anonymous feedback (later) confirmed the objectives of the exercise had been achieved.
The exercise evoked evidence to suggest that appropriately selected qualitative texts, even when not highly related to the immediate domains of the researcher, can be of benefit when exposing types of qualitative research methods to more numerate students. Equally, within reason, an appropriately designed teaching exercise grounded in everyday life can be a useful bridge to help adapt more quantitatively disposed students to a more qualitatively predicated mind set. Such exercises should also be considered in terms of assessing whether related theory could be exposed in conjunction with the exercises.
Bamberger, M (1999)
Integrating quantitative and qualitative research: lessons from the field.
Washington, DC: World Bank.
Brannen, J (1992)
Mixing methods : qualitative and quantitative research.
Aldershot, Brookfield, USA: Avebury.
Budd, J and Raber, D (1996)
‘Discourse Analysis: Method and Application in the Study of Information’
Information Processing & Management, Volume 32 (1996) pp.217-226
Burrel, G. and Morgan, G (1985)
Sociological paradigms and organisational analysis: elements of the sociology of corporate life
Carr, W. and Kemmis, S. (1986)
Becoming Critical: Education, Knowledge and Action Research.
Falmer Press, London, England
Dewey, J (1933)
Experience and Education.
Macmillan Press, New York, New York, USA
Dey, J. (1993)
Qualitative Data Analysis: A User-Friendly Guide for Social Scientists.
Routledge Press, London, England
Dunn, R. and Dunn, K. (2000)
The Complete Guide to the Learning Styles In-service System.
Allyn & Bacon, Boston, Massachusetts, USA
Frohmann, B. (1996)
‘Discourse Analysis as a Research Method in Library and Information Science’
Library and Information Science Research, Volume 16 (1994) pp 119-138
Jones B. (2004)
“Perspectives on Management Research Design and Orientation: Quandaries and Choices’
Electronic Journal of Business Research Methods, Volume 2 (2), pp 111-118
Knox, K. (2004)
‘A Researcher’s Dilemma - Philosophical and Methodological Pluralism’
Electronic Journal of Business Research Methods. Volume 2 (2) pp 119-128
Kuhn, T S (1977)
‘The function of measurement in physical science’ Isis, Volume 52 (1961) p161-93.
Kuhn, T S (1977)
The Essential Tension. Chicago UP, Chicago and London. p178
Merton R, K. (1968)
Social theory and social structure: Toward the codification of theory and research.
Glencoe, Ill., Free Press.
Olson, H (2000)
‘Quantitative "versus’Qualitative Research: The Wrong Question’
Ramsden, P. (1992)
Learning to teach in Higher Education. Routledge Press, London, England
Tesch, R. (1990)
Qualitative Research: Analysis Types & Software Tools. Falmer Press, London, England
Thomas, R M (2003)
Blending qualitative and quantitative research methods in theses and dissertations.
Thousand Oaks, Calif.: Corwin Press.
APPENDIX 1 - "The Band Played Waltzing Matild » (Eric Bogle)
Now when I was a young man I carried the pack
And I lived the free life of the rover,
From the Murray's green basin to the dusty outback
I waltzed my Matilda all over;
Then in nineteen fifteen my country said "Son:
It's time to stop rambling, there's work to be done"
And they gave me a tin hat and they gave me a gun,
And they sent me away to the war.
And the band played "Waltzing Matilda"
As the ship pulled away from the quay,
Amid amid all the cheers, flag waving and tears,
We sailed off to Gallipolli.
How well I remember that terrible day,
When our blood stained the sands and the water,
And how in that hell that we called Suvla Bay,
We were butchered like lambs at the slaughter;
Johnny Turk he was ready, he'd primed himself well,
He rained us with bullets and he showered us with shells,
And in five minutes flat we were all blown to hell,
Nearly blew us back home to Australia.
And the band played "Waltzing Matilda"
As we stopped to bury our slain.
We buried ours and the Turks buried theirs,
And it started all over again.
Those who were living, just tried to survive
In that mad world of blood, death and fire,
And for ten weary weeks I kept myself alive,
Though around me the corpses piled higher;
Then a big Turkish shell knocked me ass over head,
And when I awoke in my hospital bed
And saw what it had done, then I wished I were dead,
Never knew there were worse things than dying.
For no more I'll go waltzing Matilda,
All around the green bush far and near,
For to hump tent and pegs, a man needs both legs,
No more waltzing Matilda for me.
They collected the wounded, the crippled
The maimed, and sent us all back to Australia.
The armless, the legless, the blind, the insane,
Those proud wounded heroes of Suvla;
And when the ship pulled into Circular Quay,
I looked at the place where my legs used to be,
And thanked Christ there was no one there waiting for me
To grieve and to mourn and to pity.
And the band played "Waltzing Matilda",
As the carried us down the gangway,
Nobody cheered, they just stood there and stared,
Then they turned all their faces away.
And so now every April I sit on my porch,
And I watch the parade pass before me;
I see my old comrades how proudly they march
Renewing their dreams of past glory.
I see the old men all tired, stiff and sore,
Proud old heroes of a forgotten war,
And the young people ask: "What are they marching for?’
And I ask myself the same question.
And the band played "Waltzing Matilda",
And the old men still answer the call,
And year after year, the numbers get fewer,
Some day no one will march there at all.
Waltzing Matilda, waltzing Matilda, who'll come
a-waltzing Matilda with me?
And their ghosts may be heard
As they march by the billabong;
Who'll come a-waltzing Matilda with me?