NOTES ON RESEARCH METHODS
Michael Wood (email: michael.wood@port.ac.uk )
Portsmouth University Business School, January
2004
These notes are
at http://userweb.port.ac.uk/~woodm/rm/norm.doc
Contents
Introduction to research methodology … 1
Strategies for research
projects
Research aims or questions
… 2
General issues concerning
research: philosophy, etc … 5
Understanding the
present, predicting the future, and improving the future
Positivism and
phenomenology, and similar distinctions
The degree of
generality
Theories: building,
testing, amending, using
Politics and ethics … 7
Research design … 8
Empirical methods
Surveys
Experiments and
quasi-experiments
Case studies and small
sample research
Action research
Modelling
A general design for a
typical Masters degree project
Linking methods to
research aims or questions
Data collection methods … 13
Interviews
Questionnaires
Sampling
Trustworthiness … 16
Validity
Reliability
Objectivity
Triangulation
Statistical hypothesis
tests
Data analysis … 18
Types of measurement
Computer software
Writing the report … 20
The critical attitude
Publishing your
research
Checklist when starting a
project ... and finishing it … 21
References … 22
Appendices … 24
A note on
"theory"
Example to show
analysis of questionnaire data
Introduction
to research methodology
This is an area where there is
considerable disagreement on the definition of concepts, and what is right and
wrong. Accordingly you should read widely and critically; never assume that you
need to accept every concept and every assertion. You will probably be able to
find an exception to every rule (see Feyerabend, 1975, for an extreme version
of this principle).
These
notes are intended as a brief overview of the main issues. It is important that
you read in more depth on the specific issues of particular concern to you. For
example, if you intend to conduct some interviews or a questionnaire survey, it
is important that you consult a suitable source of guidance on surveys,
interviews and questionnaires – eg Saunders et al (2003), Robson (2002),
Easterby-Smith et al (2002).
I
will use the word "method" for a specific research method such as a
questionnaire survey. The word "methodology" refers to the study of
methods in the same way as "psychology" is the study of the psyche. A
research "strategy" is the overall approach to the project - which
may include the use of several methods.
The
word "research" in this context covers everything that academic
researchers do: the gathering of information about the world, the discovery and
creation of theories and models to make sense of this information, reviewing
and collating research done by others, as well as conceptual, mathematical and
computational analysis.
The strategy for carrying out a research project is largely a matter of
common sense. It is important not to let jargon and technicalities obscure
this. (I am using the term strategy here in the sense of a general answer to
the question "How do I go about research?" - taking all aspects into
account. You will find other authors may use the term in a slightly different
sense.)
A
simple basic strategy for any research project is:
1 Decide what you want to
achieve - the aims of the project, or the questions it will answer.
2 Decide how you are going
to achieve these aims or answer these questions - the design of your research
project. (Most aspects of research tend to take longer than anticipated, so it
is important to plan the timescale carefully to take this into account.)
3 Carry out the research,
analyse the results and state the conclusions and (if appropriate)
recommendations.
4 Check that you have in
fact achieved the aims of the project. If you have not, work out your excuses,
try again, or pretend that you were really trying to do something else - ie
change your aims to fit what you actually did.
One difficulty with this is that
you may not know exactly what you want to achieve at the outset. This may only
become clear as the research progresses. Similarly the appropriate methods
(step 2) may only become clear as the research evolves. In general, it is best
to plan your research in advance as far as possible, but it is clearly
important to be flexible.
Sometimes the research aims or questions are quite clear. More
typically, a research project may start from a fairly fuzzy problem or area of
concern; it is then necessary to decide on a clear focus by formulating some
more definite aims or questions - although you may change your mind about these
as discussed above. This process of achieving a focus is often not easy and
deserves care (see Saunders et al, 2003, Chapter 2). It is almost always
better to focus on a limited area so that you can do a thorough job, rather
than having a broad focus with inevitably superficial results.
It
is normal to include a section on the background context of the research
project. As well as details of the real world issues the project tackles, you
may also wish to discuss the academic background and your personal motivation.
(Your personal aims for doing the project - perhaps to pass the course and
acquire a marketable skill - are, of course, distinct from the research aims of
the project.)
The
focus for your research project, its goals, can then be formulated in any of
the following ways:
* Question(s) to be
answered: eg What is the best quality strategy for ABC Company?
* Aim(s) (or objectives)
to the achieved: eg To devise the best quality strategy for ABC Company.
* A hypothesis or
hypotheses to be tested: eg Strategy X is the best strategy for ABC Company.
Aims and questions are more or
less equivalent. Whether you express your goals as a list of aims or as a
series of questions does not matter much.
My
preference would be for questions because questions lead to answers which can
be written down in a research report, whereas aims may be wider than this. For
example, the aim "to increase profits" is not an appropriate aim for
a research project because the output is not research. This is a business aim
not a research aim. The corresponding research aim would be to find out
how best to increase profits. On the other hand, Saunders et al (2003)
recommend objectives because they "lead to greater specificity" (p.
25).
However, in general, I would advise you against
formulating the aims of your project as a series of hypotheses to be tested. Testing hypotheses in management is more difficult
than it may appear, and the results of the research become a simple list of
True/False statements - which may be boring for readers!
Despite
this, it may be useful to have an informal hypothesis - eg TQM is
helpful - to guide your research. Then you can formulate some more detailed
aims spelling out which aspects of the helpfulness of TQM that you wish to
investigate.
You
may also have hypotheses you wish to test as a part of addressing your research
aims. For example, you may wish to test the hypothesis that there is no
difference in effectiveness between two procedures.
It
is often helpful to have a series of questions (or aims), which may be broken
down into a hierarchy - for example:
1
This
diagram shows the fairly vague topic "Strategy to improve X in
organisation Y" broken down into three more specific objectives. This is a
typical general aim for a Masters degree project: X might stand for quality,
profitability, marketing or employee job satisfaction, for example. Each of
these objectives is then applied to two areas of the organisation. There may be
more areas to consider, but the diagram indicates that this project is only
concerned with two of them.
A
diagram such as this (based on Keeney, 1992) should be helpful for clarifying
and structuring your aims (or objectives, or questions). It is also helpful for
checking that your proposed research methods are likely to be adequate for
meeting your aims (or answering your research questions). We'll return to this
below.
The
research aims or questions should
* be unambiguous and
clear;
* be coherent, and
reasonably challenging but not too ambitious;
* make the scope of the
research clear (will it refer to one company or be broader, for example?);
* clarify the meaning of
any key terms used;
* refer to practical or
theoretical outcomes;
* be listed near the start
of the project repory.
Try to envisage the sort of
conclusions which you might expect to arrive at. Then ask yourself:
* Are you likely to be
able to get the evidence to justify these conclusions?
* Are the conclusions
worth the effort. Put yourself in the position of a critic who says, simply,
"So what?".
At the end of the project
report, you should have a clear section explaining how you have achieved the
aims (or answered the questions) laid out near the beginning.
The first point to be made is that the outcomes of a research project
(the answers to the questions posed by the researchers) may be of a wide
variety of different types. The possibilities include:
* Universal laws of the
type which are common in natural science. (Eg E=mc2. TQM always improves profitability.) Such laws are
very rare, or perhaps non-existent, in management. They are not a realistic
aim.
* Statistical conclusions. (Eg 60% of TQM implentations fail. On average,
on-the-job training is more effective that class-room training.) These are
common outcomes of management research. It is obviously very important to
specify the scope of the research (what training in which industries?) and the
exact meaning of key terms (on-the-job, classroom, effective).
* Detailed analyses of
particular situations (case studies). Eg a detailed case study of a TQM implementation which failed might be
useful for understanding the causes of failure and so avoiding them elsewhere.
* Conceptual frameworks.
* Mathematical and other
models.
* Recommended procedures
or methods.
Can you think of any other
possibilities?
Research projects may seek to understand and explain the present and
past situation, to predict the future situation, or to recommend how
to improve the future situation (sometimes called
"prescriptive" conclusions), or a combination of all three.
For
example, consider a research project which aims to find the best quality
strategy for a particular company. This might start by developing an understanding
of the existing problems in the company, and the effectiveness of the various
possible quality strategies in use in the industry. This understanding may
range from a simple catalogue of problems, to a deeper explanation of the
sources of the problems and the effectiveness of the various quality
strategies.
The
next step might be to predict (roughly) the impact of the various
possible strategies. These predictions would be based on the understanding of
the existing problems and the effectiveness of the various possible strategies.
These
predictions can then be used to decide which strategy is likely to be best in
the sense that it will improve the company's performance more than the
others. The research is aimed at understanding, prediction and improvement, but
improvement is, of course, the main goal.
Unfortunately,
most discussions of research methods in management are based fairly closely on
similar discussions about the natural and social sciences - which aim to
understand and predict, but not to improve. This means that the aim of making
improvements tends to be ignored in philosophical discussions. Ulrich (1983, p.
15) claims that "there is no adequate philosophical basis" for this
type of research. This is serious because the logical basis of recommending
improvements is very different from the logical basis of understanding or
predicting.
There
are two important differences. The first is that if a change is made, the new
situation will be different from the existing situation, and so difficult to
research directly. It is difficult to study the impact of a new idea which has
not been tried! There are a number of ways round this difficulty: the use of
experiments, action research and modelling (see below), and studying (for
example) other organisations which have tried the new idea. (This is not
possible, of course, if the idea is really new.)
The
second point about making recommendations about what an organisation ought to
do to improve performance is that this obviously presupposes some value
judgements. These are "subjective estimates of worth" (the
Pocket Oxford Dictionary, Clarendon Press, 1996): ie assertions about how
things are valued, or about what is good and what is bad, and about which goals
an organisation or individual should strive for. Different groups in an
organisation, or different stakeholders, may, of course, arrive at different
value judgements, and different recommendations about what should be done.
It
is important to try to be as explicit as possible about the basis of these
value judgments. Where the value judgments depend on several different
criteria, it may be helpful to indicate how each quality strategy (or whatever)
scores against each criterion by means of an "options by criteria
matrix". (See also Robson, 1993, chapter 7 on "Evaluations", and
Keeney, 1992.)
Both
of these points - the fact that the research has to study hypothetical
situations, and has to be based on value judgments - mean that research which
seeks to improve situations fits uneasily into the crude idea of the scientific
method known as positivism - to which we turn next. (Despite this,
"management science" is perhaps the main source of prescriptive
management research!)
Positivism is the view that research should be scientific in a fairly
crude sense. The reality researched is viewed as external and objective, and
the methods used should be "value-free" and, as far as possible,
quantitative. (There are many different versions of positivism. The confusion
is exacerbated by the fact that much of modern physics is far closer to
phenomenology than positivism as it is usually understood, and some branches of
management science have a lot to say about values.)
Phenomenology
"stems from the view that the world and 'reality' are not objective and
exterior, but that they are socially constructed and given meaning by
people" (Easterby-Smith et al, 1991, page 24, citing Husserl, 1946). This
leads on to a style of research that involves detailed interviews and other
interactions with the actors involved in a situation, and appreciating, but not
necessarily predicting, the different perspectives and choices people adopt. It
typically involves an in-depth study of a small sample of people which attempts
to understand the experience of these people "from the inside" - ie
in terms of their subjective experience. The researcher is inevitably not independent
of the situation under study, which may mean that different researchers come to
different conclusions. (Does this matter?)
A
phenomenological analysis is typically mainly qualitative in character rather
than quantitative, and deterministic or statistical conclusions tend to be
shunned in favour of a thorough analysis of a small number of cases - which
may, of course, illustrate possibilities which could occur elsewhere.
Positivistic research, on the other hand, typically involves larger samples, which
produce more reliable statistical generalisations but at the cost of a
shallower understanding "from the outside" - ie in terms of
externally defined variables.
It
is not helpful to regard this as an either-or choice. Any useful research is
likely to draw on both objective facts and subjective experiences, and
to use both qualitative and quantitative methods of analysis.
There
are other related concepts and distinctions - hard and soft (Rosenhead, 1989);,
and positivism and social constructionism (Burr, 1995, Easterby-Smith et al,
2002). The terms “quantitative" and “qualitative” are often used as
umbrella terms for the two ends of the spectrum.
The
meaning of many of these terms is rather hazy, so it is important to define
what you mean when using them.
Einstein's famous equation E=mc2 refers to all the matter in
the universe at any time. It is perfectly general.
At
the other extreme the aim of devising the best quality strategy for QRW Ltd.,
Emsworth, England in 1997 refers to one particular company at one particular
time.
Obviously,
other things being equal, the more general the research is the more useful it
is. However, other things rarely are equal. In fields like management, general
theories are often too vague to be helpful in specific situations, and they are
also far harder to set up. For this reason, it is usually a good idea to
make your aims fairly specific - ie relating to one organisation or sector or
country. However it may be worth adding a subsidiary aim to generalise your
conclusions more widely (particularly if you are considering getting another
job or want to publish your findings).
The word theory means different things to different people. I
think that anything which goes beyond a straight listing of the facts should be
counted as a theory. This includes explanatory frameworks, generalisations,
recommendations, mathematical models, etc. All useful research involves theory
in some form - there is a note on the meaning and role of theory in the
appendix.
Sometimes,
the aim of the research is to develop theory from scratch. This is the inductive
approach: trying to derive generalisations and explanations from the data you
collect. In its pure form the researcher tries to forget any preconceptions and
just let the data "speak". You will find suggested tactics for this
in books on research methods, and in more detail in Miles and Huberman (1994).
The
other extreme style of research involves starting with a theory, or hypothesis,
and then testing it. The theory may come from other researchers, or it may be a
hunch or a conjecture. This is the hypothetico-deductive approach to research.
Karl Popper is an influential advocate of this style of research (see Popper,
1978, or one of the many commentaries on Popper’s views).
It
is very important that the theory should be very clearly defined. For example
"Women are more intelligent than men" could not be properly tested
without defining intelligence in numerical terms, specifying which women the
hypothesis refers to, and whether it refers to average intelligence levels.
Popper (1978) has stressed the importance of the hypotheses being testable: he
claims that the theories of Marx and Freud are useless because their hypotheses
cannot be tested.
In
practice, the best approach is often in the middle: a bit of induction, and a
bit of testing theory. The result may be an amended theory, or a theory adapted
to a particular situation, or conclusions about the value (or otherwise) of the
theory in a particular context.
Sometimes
a research project will make use of a theory developed by other researchers,
without trying to test or amend it. For example, research into profitability
and employee empowerment might make use of measures of profitability and
empowerment - which are themselves theories.
The theories which play a part in your research are an
important aspect of the project. You should discuss these theories, and their
role in your research, carefully.
The political issues surrounding access to data, and the impact of the
results also need considering. Will you have access to the data you need? Do
you have to give guarantees of confidentiality and if so does this matter? What
if your conclusions are not to the liking of key stakeholders?
Similarly,
there are sometimes ethical dilemmas in research. These are obvious in medical
research where, for example, it is obviously unfair to withhold what is
considered the best treatment in order to set up a controlled experiment. In
management research, withholding benefits from a comparison or control group
may also be considered unfair. More generally, except in very special
circumstances, it is considered unethical to mislead people involved in
research, to subject them to stress, to invade their privacy, and so on. If
interviewees are promised they will not be identified in research reports, it
is obviously unethical to fail to do this.
Having decided on the aims to be achieved the next stage is to design
your research: in other words devise a plan for achieving the aims. Much of the
most successful research uses a variety of different methods. It is best to
start without too many preconceptions concerning the best approach.
There
are three possible sources of information for research:
1 Empirical sources:
gathering information from the real world. This may be primary data that
you have gathered yourself, or secondary data gathered by someone else -
eg published statistics or company documents.
2 Literary sources:
gathering information from published books and papers, and from the internet
(see Stein, 1999).
3 Conceptual analysis:
analysing the meanings of concepts and their implications. (Mathematical
analysis and model building are conceptual in that they are concerned with
working out the detailed implications of assumptions.)
Almost all projects make some
use of all three - but the emphasis is usually on empirical methods.
However, your research report should always include a review of relevant
research by others (the "literature review"); and your research will
also inevitably depend on a framework of concepts ( a "conceptual
framework") which should be carefully analysed and justified. What do you
mean by "quality", "competitive" or whatever other terms
are important for your research?
Empirical research usually involves making choices in
four areas:
1 Are
you going to study the existing situation, or are you going to do an experiment
or a "quasi-experiment" - ie change something and see what effect it
has? Experiments and quasi experiments are particularly useful for gathering
support for recommendations.
2 What sort of sample are
you going to take? Large sample, small sample or study of a single case?
3 Are you going to use a
standard theory or framework (and if so which?), or are you going to develop
your own theory? In either case, theories are important (see appendix).
4 How are you going to
gather the empirical data? The possibilities include: written questionnaires,
interviews, observation, “participant observation”, document and data archive
analysis, the internet, etc. Can you think of any others?
All of these choices deserve
very careful consideration. Don't forget that you will probably use different
approaches for different parts of your research.
There
are also some other possibilities which do not fit neatly into this framework
(eg computer simulation, role plays). The important thing is to be flexible
and use a variety of methods to achieve your aims.
The
following subsections describe five general patterns of research design:
surveys, experiments and quasi-experiments, case studies, action research, and
modelling. These may overlap - a model may be built from a case study or a
survey, or an action research project may make use of a survey - and there are
certainly other possibilities.
You should not be restricted by these: good research
generally uses a combination of these patterns as well strategies which do not
fit neatly into any of them.
A survey involves the collection of information from a (usually fairly
large) number of "units". These units may be people, or
organisations, or towns, or families, or departments, etc; the information
collected may be of any kind - eg financial information or opinions in the case
of surveys of people, or information about numbers of employees and
organisational structures in the case of a survey of organisations. A survey
provides a snapshot of the situation as it is at a particular time, usually
with a view to analysing patterns and trends applying to the group as a whole.
Most surveys are based on a sample of the population of interest
(see notes on sampling below). Surveys often use questionnaires to collect
data, but interviews or observation may sometimes be preferable. Many people
seem to assume that an Masters degree project has to include a questionnaire
survey but this is not so; do not use a questionnaire survey if
it is not the appropriate method for your purposes.
Further
reading in any book on research methods.
Surveys provide a way of finding out about the present situation and
what has happened in the past. However, there are two major difficulties with
simply monitoring what is happening now and what has happened in the recent
past.
The
first difficulty is that it may be difficult to disentangle cause and effect.
There is apparently (Huff, 1973, p. 84) a strong and positive correlation
between the number of babies born into families in Holland and Denmark and the
number of storks' nests on the roofs of their houses. Does this suggest that
the storks are in fact responsible for the babies? Obviously there is a more
plausible explanation - big families have big houses which provide more space
for storks to nest. However, you cannot make reliable inferences about which
factor is the underlying cause from the correlation observed. To test the
hypothesis about storks increasing the number of babies, you would need to do
an experiment - perhaps encouraging more storks to nest to see if the number of
babies born increases. You need to control the relevant variables (eg
size of houses, age and gender of occupants) so the comparison is a fair one.
The
second difficulty is that things that have not happened cannot be investigated.
If the best solution is a combination of circumstances that have never arisen,
no survey will ever find it.
The
best way round these difficulties is to design an experiment. This involves
changing something and then measuring the effect that this change has. The
simplest design for an experiment is the "post-test only two group
design" (Robson, 2002):
1 Set up an experimental and a
comparison (control) group using random assignment.
2 The experimental groups gets the
"treatment"; the comparison group gets the "comparison
treatment". It is important to ensure that the two groups get roughly the
same amount of attention - otherwise there is a possibility that any observed
difference may be due to the “Hawthorne effect”. This is named after a famous
experiment in which it was discovered that any treatment - including
reversing a previous treatment - brought improvements because it indicated that
the experimenter was taking an interest in the people involved.
3 Give "post-tests" to see
what the effect of the treatment is.
More complex designs are of
course possible (see Robson, 2002). The random assignment is important to
reduce the likelihood of some factor other than the "treatment" being
responsible for any observed improvement. The results of an experiment are then
usually analysed by means of a statistical hypothesis test (see below).
Experiments
are widely used in medicine, psychology, and to a lesser extent in education.
In management, it is often impossible to follow a rigorous experimental design
so quasi-experiments are often used instead. Quasi-experiments are
defined by Campbell and Stanley (1963) (cited by Robson (2002), p. 133) as
"a research design
involving an experimental approach but where random assignment to treatment and
comparison groups has not been used."
For example, the success of an
organisation using a particular type of quality management system may be
compared with an organisation which does not use this type of system, or with
the same organisation before the system was implemented. In either case the
"treatments" (quality system or no quality system) were not allocated
at random, so there is the (strong) possibility that some other, uncontrolled,
variable is responsible for any differences found. For this reason, Robson (2002)
does not recommend either of these designs, preferring more elaborate designs
(see Robson, 2002, pp. 136-146 for details). The important thing is to be as
sure as possible that the lack of randomisation in quasi-experiments is not
likely to affect the validity of the results.
It often seems more useful
to undertake a detailed study of an individual case, or of a small sample of
cases, than to do a superficial study of a larger sample. The cases may be individual people, organisations,
neighbourhoods, projects, events of various types, etc. It is important to be
clear about the purpose of the case study. Is it intended to be typical of
something more general, or to be a case of particular interest from some
specific point of view?
It
is also important to make sure that your approach is systematic, and that you
give adequate attention to developing a suitable conceptual framework and list
of research questions. Case studies normally use multiple sources of evidence
(eg interviews, observations, document analysis, etc), and should aim for a
detailed (“in depth”) understanding of the chosen case(s).
Further
reading: Yin (1994).
Traditional science seeks to keep the researcher separate from the
researched and their aims and values in the interests of
"objectivity". Action research is the name given to research which
seeks to integrate theory development and data collection with action in the
sense of improving the process being studied. The action researcher would
typically be an active participant in the process. The obvious danger here is
that the particular interests of the researcher / actor will encourage a biased
perspective: clearly you must try to reduce the likelihood of this happening.
(A counterargument to this starts from the assertion that there is no such
thing as an unbiased perspective, just different biases ....)
There
are different variants and interpretations of action research. One simple
possibility would be:
1 Study the existing
situation
2 Plan how improvements
could be made.
3 Carry out these
improvements and analyse their effects and success. (This step may be a quasi
experiment.)
4 Study the new situation.
5 Go back to step 2, etc,
etc.
It is obviously important to
ensure that the researcher's involvement in the process does not compromise the
validity of the results.
Management science researchers often seek to set up a model of,
for example, a stock control system, or a series of cash flows, or a project.
Models are also important in many other areas including, for example, finance
and "softer" disciplines such as marketing. Harding and Long (1998)
summarise 45 of these management models. Modelling is not treated as a standard
type of research in most texts on research methods - you will need to consult
books such as Pidd (1996 or 2003).
Pidd
(1996, p. 15) defines a model as
an external and
explicit representation of part of reality as seen by people who wish to use
that model to understand, to change, to manage and to control that part of
reality.
Models may be physical,
mathematical or computer based. They are useful if experimenting directly with
reality is too difficult, costly or time-consuming. They are typically set up
on the basis of empirical data and a "common sense" analysis of how
the situation "works". Models are always simpler than reality: it is
important to consider the appropriate degree of simplification.
The
steps in a typical modelling project are:
(1) build the model;
(2) check its accuracy
and/or usefulness and adjust if necessary;
(3) use the model to
understand, change, manage, control...
Further
reading: Pidd (1996) chapter 1.
Many (but by no means all) projects fit the following
pattern:
Aim: To
find a good strategy to "improve" X in org Y
Method
1 Survey/case
studies of Org Y to investigate problems and opportunities
2 Survey/case
studies to see how other organisations do X and which approaches work well
3 Based
on (1), (2), the literature, and perhaps creative inspiration and consultations
within the organisation, devise a strategy likely to improve X
4 Try/test/pilot/monitor
the proposed strategy
To ensure that your methods are firmly linked to your research questions
(or aims), it is a good idea to draw a diagram which links each research
question with the methods you plan to use to answer it.
In
the diagrams below, the lines without arrows indicate the breakdown of the
research aims. The arrows indicate that the box at the start of the arrow is a
means to help achieve the box at the end of the arrow. The arrows only indicate
that a method will help with the aim or method it points to, not that it
will solve the problem completely. The dotted arrow is intended to signify that
the help involved is likely to be slight. (This notation is due to Keeney,
1992).
2
This diagram should help you to
ensure that the methods you are proposing are likely to be sufficient. This is
a matter of judgement, obviously. You need to check each aim carefully. In this
example, the lack of methods drawing on data from Organisation Y for assessing
the improvements from the proposed strategy, and for devising and justifying
the implementation strategy, suggests that this plan is not adequate. You are
likely, for example, to need some input to the implementation strategy from
Organisation Y. The next diagram shows a possible improvement.
3
There are many sources of data
which you should consider - see the section on Empirical methods above). This
section contains very brief notes on interviews and questionnaires, and also on
sampling, which is important whatever you decide to do. For more
detailed help, is essential to consult a textbook or other source of
advice.
Interviews
These could play a part in surveys, or case studies, or experiments, or
action research. They usually allow you find out about the topic of interest in
more depth than a questionnaire, because people are likely to give more detail
when talking than when writing, and it is possible to ask questions to probe
points of particular interest. It is however necessary to be organised: use a
list of questions and prompts and decide how you are going to record the
answers. Telephone and group interviews are other possibilities to bear in
mind. Above all, remember that the idea is to get a deep understanding of the
issues in question.
* Write a plan or schedule
for the interviews, but treat it flexibly and be prepared to modify it if
appropriate. What are you going to ask and how? Don't forget that interviews
are particularly useful for open-ended questions.
* You will probably want
to probe some responses for more detail. Some such probes can be in the
interview plan, but obviously as you do not know what the interviewees will
say, you cannot cover all eventualities.
* It is a good idea to
record the interview so that you can quote interesting bits in the write-up.
You must ask interviewees for their permission, of course. If a recording is
not possible, you will obviously need to make very detailed notes.
* Don't forget to think
about putting interviewees at ease.
* With interviews there
may be a danger that the interviewer influences the interviewee. Does this
matter and what can you do about it?
Many books and articles give
advice on questionnaires: you should consult one at an early stage because
designing good questionnaires is far more difficult than it may look. It is
essential to test the questionnaire and the proposed method of analysis by
means of a pilot survey before the final questionnaires are sent out.
When designing
questionnaires consider:
* Exactly what do you want
to find out?
* Why should people fill
it in? (Anonymity, confidentiality? Reward for returning it?)
* Will they tell the
truth?
* Length and sequence of
questions
* Wording: avoid leading,
long, complicated questions asking several things, incomprehensible,
unanswerable, silly, rude, annoying questions....
* The covering letter
explaining who you are and what the research is for.
There are three main types
of questions you can ask in a questionnaire:
* Closed questions asking
for a category. (Which department are you in? - tick the appropriate box.) Be
careful to ensure you have thought of all the categories; you should usually
have a box at the end for Other - please specify.
* Closed questions asking
for a number. (How old are you? Questions asking respondents to rate their
agreement with a series of statements on a 1 to 7 scale.)
* Open ended questions.
(What do you think of … ?) The responses may either be coded for analysis (in
which case it may be better to use a closed question in the first place), or
simply read and used for quotations and as a means of coming to understand the
respondents.)
Particularly with closed
questions you need to cater for respondents who do not know the answer. You
don't want to force them to make up an answer!
Remember that designing a
good questionnaire is much more difficult than it looks.
Common Problems with
questionnaires:
* Low response rate (What
should you do about this?)
* Too much information to
analyse
* Inconclusive answers
* You only find out what
people want to (and can) tell you
Finally, ask yourself, are
you sure you need a questionnaire? Would you fill it in yourself? If not, why
not think again?
We often talk about analysing data (figures, etc) and drawing graphs of
data as if we were interested in the data for its own sake. Usually this is not the case. Usually we are interested in our data because
of what it tells us about a wider situation.
So, for example, an opinion poll might ask 1000 voters how they are
going to vote in the next election: the assumption being, of course, that the
voting pattern of the electorate as a whole will be similar.
The
first step is to decide exactly where our interests lie. Population or universe
are terms used by statisticians for the group comprising all the instances in
which we are interested. It is important to be very clear about the exact
nature of the population. For example:
* employees in an
organisation
* employees in all similar
organisations
* all the transactions
which may be carried out by a software system (now and in the future)
If
the population is large or infinite we will need to use a sample: ie a
subset chosen as far possible to be representative of the population as
a whole. It is important in all investigations, quantitative and
qualitative, large scale and small scale, to be careful about the choice of a
sample.
Even
when it is apparently possible to look at every member of the population - ie
to carry out a census, the benefits may not be real. In one survey of
applications of "100% inspection" (Oakland, 1986, p 50), 17% of
defects on PCB's were missed, and 25% on
chest X-rays (where a defect may represent a case of TB). The problems in each case were that the
necessity to check everything meant that the job was done quickly and
carelessly. It is often a good idea to
take a fairly small sample and investigate this carefully.
In
addition, populations are often slightly wider than is apparent at first
sight. We might, for example, consider
all the transactions performed by a computer system in the past week as our
population; however a more useful perspective might be to think of these
transactions as a sample of the possible transactions for which the system is
designed. This raises the question of
whether the past week's performance is likely to typical or representative.
From
the point of view of ensuring representativeness, two problems may arise in
sampling:
1 The method of selecting
the sample may lead to an inevitable bias (even with large
samples). It is often surprisingly
difficult to obtain an unbiased sample.
2 Even if the method of
selection does not lead to bias, inevitable random variations may mean that the
particular sample chosen is unrepresentative in some way. This is known as sampling
error, and its size can be analysed by statistical methods: eg the 3% error
often quoted for surveys of electors' voting intentions with samples of around
1000 is based on a 95% statistical confidence interval (Wood, 2003).
Conversely, the theory can be turned round to tell you how large a sample is
necessary for a given degree of accuracy.
Methods of sampling can be
divided into probability sampling (where the idea is to try to ensure
that the sample is representative by controlling the probability of each
individual being chosen), and non-probability sampling (which does not
use this principle). Four important methods of sampling are:
Probability sampling:
1 Random
sampling: sample chosen so that every member of the population has an equal
chance of being selected, and every member of the sample is chosen
independently of every other member. It also means that the sample is chosen
without allowing the investigator's (possibly subconscious) preferences to
influence the choice. This is the standard on which most statistical theory
is based. To produce a random sample it is necessary to have a numbered
list of the population - this list is known as a sampling frame. Then
the sample is chosen by drawing random numbers (see below) and selecting the
corresponding members of the population as the sample.
2 Stratified sampling:
population divided into "strata" and a random sample of appropriate
size taken from each of the strata. If done properly this should yield a
slightly lower sampling error but the difference is often very small. It is
generally only worth doing if it easy to do or you want to compare results by
stratum. You should also bear in mind that your sample will suffer if you only
take a few of the strata. For example, if you base a sample of workers on just
three companies, this sample will obviously not encompass as much variety as it
would if you took a wider sample of companies.
Non-probability sampling:
3 Purposive
sampling: the researcher's judgment is used to choose individuals which are
thought to be typical or of special interest. It is often a good idea to choose
small samples (eg for case studies) in this way; for larger samples, the random
or stratified methods are likely to produce more representative results.
4 Opportunity or
convenience sampling: taking the sample that you can get. This is
effectively working backwards: the problem then is deciding on the population
to which the results can be generalised.
As a general principle
random sampling is best for large samples (say 12+), whereas purposive sampling
is suitable for small samples. Remember that the final sample may be smaller
than you anticipate because of non-return of questionnaires, etc.
Random numbers (produced by a spreadsheet)
2569 9114 7079
2209 3867 0793
6977 3720 1510
2765 4074 5878
5759 2317 4575
6224 1399 7161
6903 6414 1792
5956 1543 3127
4895 2861 6714
0676 0635 7399
3420 7827 2116
7672 1573 0632
5594 1149 9320
2288 7634 6464
2378 6759 9738
1734 1063 2848
6489 8750 1189
5490 7826 0818
9196 5858 4586
4792 1260 6522
1039 9930 7971
2092 8076 5686
8511 2598 8687
7479 9436 0699
8264 1735 6532
0860 6313 6132
7005 7045 1183
5183 6472 8021
5716 7222 7773
5886 7473 3033
8900 2384 8255
9014 1209 8897
2828 1461 7399
0623 8927 3789
2030 1993 1094
7274 3554 2439
4360 1900
What makes research trustworthy? Why should you believe or accept
the conclusions? The concepts of validity, reliability, objectivity,
triangulation and statistical hypothesis testing are all relevant to
this issue. The most general concept is validity.
As
well as being trustworthy, research should, of course, also be relevant
and useful. Readers should not be left asking "So what?".
Validity refers to the
extent to which the results are valid - ie true or well grounded. Gill and
Johnson (1991, p 161) distinguish three types of validity:
1. Internal validity is the
extent to which the conclusions regarding cause and effect are warranted.
2. Population validity is
the extent to which conclusions can be generalised to other people, or other
organisations, or other sampling units. This is a matter of ensuring that samples
are likely to be representative (see the notes above).
3. Ecological validity is
the extent to which conclusions might be generalised to social contexts other
than those in which data has been collected.
There is also ...
4. The extent to which
operational definitions or indicators (eg defect rates as a definition of
quality; IQ tests as a measure of intelligence) reflect the concept they are
trying to capture.
This refers to the consistency of the research method. For example would
you get the same answer if you repeated the research with a different sample,
at a different time, or with different observers or judges? Suppose, for
example, your research involves coding responses to an open-ended question on a
questionnaire. You should check a sample of codes by bringing in a second
researcher. You could then indicate the reliability of the coding scheme by
saying that the two coders agreed on the code given to 95% (or whatever) of
responses. This provides the reader with a simple assessment of how reliable
this aspect of the research is.
This term refers to the extent to which research reflects the reality of
the "objects" (including people) under study, as opposed to the
subjective experience of the researchers or observers. In practice, the method
for checking whether an observation or assessment is objective is to see if
different observers agree: if they do it is objective, if they do not it is
subjective in the sense that it depends on the subjectivity of particular
people. Physical measurements like weight or time are objective because
different observers will agree readily, whereas assessments of the quality of a
meal are more likely to be subjective.
Some
would say objectivity is essential; other would say that it is meaningless or
impossible in many contexts. Do you think it is sensible to talk about the
objective quality of a meal? On the other hand if you are interested in the
amount of scrap produced, it seems sensible to get as objective a measure as
possible.
Checking your conclusions by other methods. For example, if
questionnaire results suggests that particular managers are not motivated by
money, this could be checked by interviewing the managers, and by observing
their behaviour (or records of their behaviour) when offered financial
incentives.
These provide a way of deciding if the evidence is strong enough.
Examples are the "t test", analysis of variance (ANOVA) and the
"Chi square test". These tests are mathematically complex, and are
very frequently misunderstood and misinterpreted. Despite this they are useful
and widely used. Statistically significant means that the data cannot
reasonably be attributed to chance alone (ie to the accident of the particular
sample chosen). A significant result signifies a real effect (and
not just a sampling accident). The significance level tells us how strong the
evidence is - with the lower levels indicating stronger evidence.
For example, the
results below (McGoldrick & Greenland, 1992) come from a survey on the
service offered by banks and building societies:
|
Aspect of service
|
Banks' mean rating
|
Building Society's mean rating
|
Level of significance (p)
|
|
Sympathetic/understanding
|
6.046
|
6.389
|
0.000
|
|
Helpful /friendly staff
|
6.495
|
6.978
|
0.000
|
|
Not too pushy
|
6.397
|
6.644
|
0.003
|
|
Time for decisions
|
6.734
|
6.865
|
0.028
|
|
Confidentiality of details
|
7.834
|
7.778
|
NS
|
|
Branch manager available
|
5.928
|
6.097
|
0.090
|
The data was obtained from a
sample of customers who rated each institution on a scale ranging from 1 (very
bad) to 9 (very good.). The above six dimensions are a selection from the 22
reported in the paper. The evidence is strongest in relation to the first two
variables and weakest in relation to the least one. The p values in the
final column of the table give the estimated probability of obtaining the
results which were actually observed, or more extreme ones, if there is
really no difference between banks and building societies. (There is a
fuller explanation at http://userweb.port.ac.uk/~woodm/nms/test.doc
.)
NS
means not significant - which in this table means that the p value is
greater than 0.1. The lower the p value the more convincing the evidence
for a real difference between banks and building societies./
In
many contexts (including the example above) “confidence intervals” provide an
alternative method of analysis - which may be more useful and user-friendly
(Gardner and Altman, 1986; Wood, 2003).
There are many methods of analysing data. You should read up those that
are appropriate to your particular study.
At
one extreme is statistical analysis. The steps here are:
1 Decide what you are
going to measure. Check that the proposed measurements are valid and sensible.
If appropriate check the reliability of your measurements.
2 Produce diagrams and/or
tables to show the values of your measurements and the relationships and
differences between them. It is more difficult than it might appear to design
tables and diagrams which are clear and unambiguous - ask someone else to
check!
3 If appropriate, do
statistical hypothesis tests or work out confidence intervals to indicate the
likely effects of sampling error. (You may need help here.)
At the other extreme, the
analysis of tapes of interviews, or open-ended questions in questionnaires,
might simply consist of listening to the tapes, or reading the questionnaire
responses, to try to understand the situation. The report of the research would
then include direct quotations (in “…”) from the interviews, or the
questionnaires, as evidence for the assertions put forward.
The
weakness of this last style of research is that the particularly passages
quoted may give an unrepresentative impression. The suspicion may be that the
researcher has chosen the quotes that confirm her (or his) prejudices. Clearly
this type of analysis needs to be backed up by some further evidence. It is,
however, a very useful method of providing a detailed analysis of certain possibilities.
For example, a researcher investigating the use of a software package might
find one individual using it in a particularly innovative manner: a detailed
analysis of this one instance may be interesting because it illustrates what is
possible - although it is in no sense representative of the population as a
whole.
To
use interview data, or data from open-ended questions on questionnaires, to
obtain more quantitative information about the frequency with which phenomena
occur, or the strength of relationships, it is usually necessary to devise a coding
scheme (see Saunders et al, 2003). This can be used to give quantitative
results on the percentage of individuals in each category, or the number of
times particular things are mentioned. These results can then be analysed
statistically like any other quantitative results.
One
issue to consider when analysing "softer" data from interviews and
participant observation studies is the extent to which the conclusions should
"emerge" from the data without the researcher imposing his or her
preconceptions. This is the grounded theory approach (see Saunders et
al, 2003; Robson, 2002). Various methods have been proposed for achieving this
– eg analytic induction (Saunders et al, 2003, 397-8; Robson, 2002, p. 322).
Whatever
you do it is important to consider the validity and reliability (see above) of
your final conclusions.
Further
reading: Miles and Huberman (1994).
Variables may be numerical (eg salary), ordinal (ie a rank
- eg Manchester United's position in the league was 2nd), or category
variables (eg male or female, make of car, etc). Take care not to manipulate
results in ways that do not make sense. For example there is little point in
coding a category variable (eg make of car) by the numbers 1, 2, 3, 4, etc and
then taking the average - it won't mean anything.
Numerical
scales can be further subdivided into ratio and interval scales.
Ratios make sense in ratio scales but not interval scales. For example it makes
sense to say that one man earns twice as much as another (earnings is a ratio
scale), but it does not make sense to say that a temperature of 20 degrees
Celsius is twice as hot a temperature of 10 degrees since temperature is not a
ratio scale; the zero point is arbitrary - the equivalent Fahrenheit
temperatures are 50 and 68 which are not in the same ratio.
The most useful type of package is a spreadsheet. Excel is particularly
good because of the wide range of statistical functions and procedures which it
incorporates. Put each record (individual from a sample) in a separate row with
field headings at the top. For example:
|
NAME
|
SEX
|
HEIGHT
|
WEIGHT
|
|
Bill
|
M
|
1.47
|
132
|
|
Susan
|
F
|
1.91
|
|
|
Mandy
|
F
|
1.45
|
38
|
Avoid the temptation to include
fancy formatting, to leave rows to improve spacing, etc. Any frills you include
may cause problems when you try to analyse your data.
If
any data is missing (eg Susan's weight) leave the cell blank. Do not enter 0.
Yes/no is best coded as 1 for yes and 0 for no; then the average of the column
will give you the proportion answering yes.
You
may be able to do all your analysis with a spreadsheet. Don't forget that
spreadsheets will sort data. If you want to see how males differ from females,
you can sort the data on this field. Spreadsheets are also good for working out
averages, drawing bar charts and other diagrams, etc.
However,
if the statistical analysis you need is at all complex, it may be worth
transferring the data to a statistical package such as SPSS (Statistical
Package for the Social Sciences).
Further
reading: Wood (2003) contains brief notes on the use of Excel and SPSS for
analysing data.
A standard layout is:
* Abstract
* Acknowledgments (if any)
* Contents
* Introduction (including
background and context - this would normally lead on to the aims in the next
chapter)
* Aims of the project
(what you intend to achieve)
* Literature review (briefly
and critically reviews relevant previous research and discusses
its relation to your study)
* Research design or
method (what you did and why)
* Investigation results
and analysis (may be split into several chapters)
* Conclusions and
recommendations (possibly two chapters). You should also discuss the
limitations of the research and possibly include suggestions for future
extensions.
* References (must follow
one of the standard formats)
* Appendices (supporting
material to which readers may want to refer – eg questionnaires, examples of
interview transcripts)
However, many projects are not
standard so you should feel free to adjust this pattern if appropriate.
Whatever
the structure of your report, you should, as far as possible, ensure that
readers can check your analysis to see if they accept your conclusions (put
details in appendices). Above all, please ensure that the report is clear,
concise and does not exceed the permitted length.
It
is important to describe and discuss all important aspects of your empirical
research: details of questionnaire surveys and interviews, software used,
methods of analysis, and so on. The reader should be able to follow what you
did, and how you derived your conclusions. This should enable the reader to
decide how trustworthy your research is, and perhaps repeat it in another
context. Remember that if your research is well designed and competently
carried out, this should be clear from the report.
All
books and other sources should be clearly referenced using one of the standard
styles. There is a leaflet on this available from the library, but you may find
it easier to copy the style used in a particular academic paper. In my view the
easiest style is to refer to works in the text by the author's name and the
date of publication only - for example, Plato (1956) - and then to list the
publications in alphabetical order of authors' names at the end. Every
reference you give in the text should appear in the list of references at the
end - check for Plato (1956) in the references at the end of this document.
(The date is the date of the publication of the version to which you
referred; obviously Plato did not write in 1956.) Notice that books and journal
articles are mixed up in this list of references; otherwise you would not know
which list Plato (1956) is in. Note also the style of book and journal articles
(eg Thorpe and Moscarola, 1991) in this list of references.
One of the distinguishing characteristics of good research is that as
much as possible is subjected to critical analysis. You should question as much
as possible. If the objective of the project is to derive a "good"
strategy for a particular purpose, what does "good" mean? Who says
and how do you know? Why is this method appropriate? What are the potential
flaws with this method and how did you try to overcome them? What are the main
weaknesses of your research, and other research in the field? Try and
anticipate and answer all possible criticisms of your research.
If you think your project deserves a wider audience you should consider
publishing it in a journal or in some other format. Ask your supervisor for
advice.
These are my suggestions for checking your initial
project proposal:
1 What outputs do you expect? Write down some examples of the
sort of conclusions and recommendations you might expect at the end of your
project.
2 So what? Is the world -
or at least part of it - going to be a better place once these conclusions and
recommendations have been reached?
3 Are you likely to be
able to get the right data, and enough data, to justify these conclusions. What
if a key stakeholder doesn't like your results, conclusions or recommendations?
Have you access to all the information you need? Will the information be
sufficiently accurate and reliable?
4 Are the aims challenging
but not so ambitious as to be impossible with the limited resources (time, etc)
at your disposal? It is often a good idea to have a fairly restricted focus
that is analysed in depth.
5
Are your research methods
appropriate to achieve the aims? If you have, say, three aims, you must make
sure that you have considered the methods for achieving all three of them.
And at the end of the
project you should check that:
1
Your research aims,
literature, analysis and conclusions are clearly linked together. It is
important to be very clear about how your conclusions and recommendations
follow from your analysis, and achieve the aims you set yourself at the start.
2
Remember that you are
reporting a research project. It should be clear from your write-up that you
have done some useful, systematic and rigorous research. Make sure that you
give enough detail for this to be clear.
General texts on research methods include Saunders et
al (2003), Robson (2002), Easterby-Smith et al (2002).
Burr, V.
(1995). An introduction to social constructionism.
London: Routledge.
Easterby‑Smith, M., Thorpe, R.,
& Lowe, A. (2002). Management Research: an introduction
(2nd edition). London: Sage.
Easterby‑Smith, M., Thorpe, R., & Lowe, A. (1991). Management
Research: an introduction. London: Sage.
Feyerabend, P. K. (1975). Against method: an outline
of an anarchistic theory of knowledge.
London: New Left Books.
Gardner, M. J., & Altman, D. G. (1986). Confidence intervals rather
than P values: estimation rather than hypothesis testing. British Medical
Journal, 292, 746-750.
Gill, J., & Johnson, P. (1991). Research Methods for
Managers. London: Paul Chapman Publishing Ltd.
Harding, S., & Long, T. (1998). MBA management models.
Aldershot: Gower.
Huff, D. (1973). How to lie with statistics. Penguin.
Husserl, E. (1946). Phenomenology in Encyclopaedia Britannica,
14th edition, Vol 17, 699‑702.
Keeney, R. L. (1992). Value‑focused thinking: a
path to creative decisionmaking. Cambridge,
Massachusetts: Harvard University Press.
McGoldrick, P. M., & Greenland, S. J. (1992). Competition between
banks and building societies. British Journal of Management,
3, 169‑172.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data
analysis (2nd edition). London: Sage.
Oakland, J. S. (1986). Statistical process control.
London: Heinemann.
Oakland, J (1989). Total Quality Management. Oxford, Heinemann
Professional.
Pidd, M. (1996). Tool for thinking: modelling
in management science. Chichester: Wiley.
Pidd, M. (2003). Tool for thinking: modelling
in management science (2nd edition).
Chichester: Wiley.
Plato (1956). Meno (trans: Guthrie, W K C). Harmondsworth:
Penguin.
Popper, K. (1978). Conjectures and Refutations. London: R.K.P.
Quinn, J B; Mintzberg, H; James, R M (1988). The strategy process:
concepts, contexts and cases. Prentice Hall.
Robson, C. (2002). Real World Research (2nd
edition). Oxford: Blackwell.
Rosenhead, J. (1989). Rational analysis for a
problematic world: problem structuring methods
for uncertainty, complexity and conflict.
Chichester: Wiley.
Russell, Bertrand (1961). History of Western Philosophy. George
Allen & Unwin.
Saunders, M., Lewis, P., & Thornhill, A. (2003). Research methods
for business students (3rd edition). Harlow:
Pearson Education.
Stein, S. D. (1999). Learning, teaching and researching
on the internet: a practical guide for
social scientists. Harlow: Addison Wesley Longman.
Thorpe, R., & Moscarola, J. (1991). Detecting your research
strategy. Management Education and Development, 22(2),
127‑133.
Ulrich, W. (1983). Critical heuristics of social
planning. Bern and Stuttgart: Haupt.
Wood, M. (2003). Making sense of statistics: a non-mathematical
approach. Basingstoke: Palgrave.
Yin, R. (1994). Case study research: design and
methods (2nd edition). Thousand Oaks, CA: Sage.
A “theory” is defined by the Concise Oxford Dictionary as a
“supposition or system of ideas explaining something, esp. one based on general
principles independent of the particular thing to be explained.” This clearly
hinges on the meaning of “explain” -
which is defined as “make intelligible”.
According
to Russell (1961, p. 52), the word theory is derived from an Orphic word which
can be translated as “passionate sympathetic contemplation”; at first sight
this is very different from the modern meaning but in fact it fits well with
the ethos of, for example, the research method of participant observation.
Theory
is often contrasted with “facts” and what happens “in practice”. A fact is “a
thing that is known to have occurred, to exist or be true”, and “in practice”
means “when actually applied, in reality”. A theory is thus a system of ideas
which explains something, or makes it intelligible, whereas facts
and practice are simply the reality of what happens. (However, the physicist,
Sir Arthur Eddington, dismisses the common assumption that facts are more
certain than theory in physical science: "You should never believe any
experiment [fact] until it is confirmed by theory" - quoted in The Guardian,
January 7, 1993).
To
give a concrete example, it might be a fact that a firm's sales have increased
by a particular amount. A theory to explain this might be the assertion that
the increase in sales is the result of improved quality in the products sold.
The system of ideas which forms this theory is the fact that quality levels have
improved, and the assertion that, in these circumstances, improved quality is
likely to lead to increased sales. The theory is useful because it gives us a
means of predicting when sales are likely to rise and so of increasing sales in
new situations. A list of facts and of what happens in practice may be
interesting; however to predict and control in new situations, theory is
needed. This reason for going beyond facts and a simple description of
practice, to theory, seems, to me, unanswerable.
According
to Quinn, Mintzberg and James (1988) "theories are useful because they
shortcut the need to store masses of data ... it is easier to remember a simple
framework ... than to remember every detail you ever observed" (p. xviii).
However, this misses the most important function of theory which is to help
cope with new situations which you have not yet observed.
However,
even apart from this reason for using theory as a means of going beyond the
given facts, theory is necessary for defining the “facts”. The above example
depends on a way of measuring quality. This can be done in various ways - by
reported defect rates, by customer satisfaction, or by some other means.
Obviously, we need a system of ideas defining quality before we can even claim
to detect an increase. The required theory might be formal academic theory, or
it might be provided by “common sense”. But in either case it is still a
theory. The same is true of many other “facts”: profitability can only be
defined by reference to theories of accounting, facts about organisational
structures can only be defined by reference to the appropriate theories. Even a simple questionnaire designed to
elicit an attitude or an opinion depends on the theory that people give true
(or valid or meaningful) answers to such questions. (This is often a rather
dubious theory.) In all these cases the facts are defined by the underlying
theory. The facts cannot even exist without the theory, and different theories
are likely to give rise to different facts. Whether this applies to all facts,
or just some facts, is an issue which need not concern us here. The important
thing is that it applies to many facts of interest to management researchers.
This
means that the use of theory is inevitable and it is clearly important to use
the best theory for the purpose in hand.
Types and levels of theory
Part of the difficulty in discussing theory is that the single term
encompasses a very broad range. Examples of theories are the simple assertion
that an improvement in quality led to an increase in sales (see above),
theories about how quality can be measured and monitored, mathematically based
theories such as the model for calculating the economic order quantity, the
theory that specifying objectives clearly increases the chances of a project
succeeding, the theory that there are particular categories of organisation,
and, on a much more ambitious scale, the theory of total quality management
(Oakland 1989). These are all theories in the sense above. They are all useful
for defining the facts and for providing explanations about, for example, what
to do in given situations.
Theories
may differ in their source: some come from academic publications, while others
may be derived from common sense. They differ in their level of generality.
They differ in the sense in which they “explain” things: sometimes the
explanation leads to a prediction (following the TQM way will lead to
improvements in quality which will lead to increases in sales); sometimes it
merely categorises the possibilities - which is an essential prerequisite for
understanding and managing a situation. Theories may be stated in formal
mathematical terms or in informal terms, which allow or even encourage
differing interpretations. Theories differ in many other ways. But they are all
theories.
The
problem for the researcher is that of choosing, creating, or adapting, the best
theory for the purpose in hand. It is important to investigate all the
possibilities and make the selection carefully.
Theories may be wrong or inadequate
Scientists tend to think of the current theory as the “truth”. However,
even the history of physical science indicates that this is likely to be a very
limited perspective: there are many old “truths” - the earth being the centre
of the universe, atoms being unsplittable, matter indestructible - which have
been replaced by contradictory new “truths”. In management, few, if any,
theories command respect from everyone. Theories of management are much more
obviously fallible and for this reason should not be taken too seriously.
Conclusions
What is the relationship between theory and management research? I think
that the discussion above demonstrates that:
1 Theories are necessary
as a background for a research project to define the concepts and terms in
which the research is phrased. Denying this does not make it less true; it just
means that the implicit theories underlying the research will be
unacknowledged, uncriticised, and, very likely, quite unsuitable for the job.
2 The only useful aim for
research is to make a contribution to theory, since a simple list of facts or
practices is of little use. The following seem to me to be the possible types
of contribution:
(a) Demonstrating that an existing theory
applies to a particular situation and showing how it can be used in this
situation: for example an application of TQM theory X to Organisation Y.
(b) Modifying, elaborating or extending an
existing theory: for example demonstrating that TQM theory X, when applied to
organisations of type Y, needs modifying in a particular way.
(c) Creating a new theory.
(d) Demonstrating that an existing theory is
wrong or useless.
(The reader should bear in mind
that the theory presented here, about the role of theory in management
research, is as fallible as any other theory and should be not accepted
uncritically. It represents my analysis; others may disagree.)
The questionnaire was to obtain feedback from students on a course. It
comprised one question asking for the student's tutorial group (GP - an
"independent variable"), 21 questions asking for ratings of different
aspects of the course on a 1-7 scale (the "dependent variables"), and
two open ended questions which were analysed separately. The data was entered
in a spreadsheet, and then the analysis was carried out using SPSS (Statistical
Package for the Social Sciences). Refno was a reference number written on each
questionnaire to identify it.
SPSS
was used to produce histograms, means and standard errors for each of the 21
questions, and a breakdown of the scores by tutorial group and an analysis of
variance to assess the significance of these results. It could also give other
statistics such as standard deviation, skewness, kurtosis, minimum, maximum,
etc. There were a total of 66 pages of
output of which one is below. (A lengthier questionnaire or a more detailed
analysis can easily result in hundreds of pages of output.)
It
would also be possible to use a spreadsheet to do some, if not all, of the
analysis.
Top left of data
spreadsheet
|
Refno
|
GP
|
Q1
|
Q2
|
Q3
|
Q4
|
Q5
|
Q6
|
Q7
|
Q8
|
|
1
|
11
|
1
|
4
|
4
|
4
|
2
|
|
3
|
1
|
|
2
|
|
2
|
1
|
3
|
1
|
4
|
4
|
4
|
4
|
|
3
|
|
5
|
4
|
4
|
4
|
3
|
|
5
|
3
|
|
4
|
4
|
4
|
5
|
5
|
6
|
6
|
|
5
|
5
|
|
5
|
|
3
|
2
|
4
|
2
|
4
|
|
3
|
2
|
(Note that missing data is
indicated by leaving the cell blank.)
Analysis
of Variance
|
|
|
Sum of
|
Mean
|
F
|
F
|
|
Source
|
D.F.
|
Squares
|
Squares
|
Ratio
|
Prob.
|
|
Between Groups
|
10
|
49.3789
|
4.9379
|
2.4055
|
0.0227
|
|
Within Groups
|
43
|
88.2693
|
2.0528
|
|
|
|
Total
|
53
|
137.6481
|
|
|
|
|
Group
|
Count
|
Mean
|
95 Pct Conf Int for Mean
|
Minimum
|
Maximum
|
|
Grp 1
|
3
|
6.333
|
4.8991 To
7.7676
|
6.0000
|
7.0000
|
|
Grp 2
|
3
|
4.333
|
2.8991 To
5.7676
|
4.0000
|
5.0000
|
|
Grp 3
|
5
|
3.400
|
.5415 To
6.2585
|
1.0000
|
6.0000
|
|
Grp 4
|
3
|
4.333
|
‑.8379 To
9.5045
|
2.0000
|
6.0000
|
|
Grp 5
|
3
|
3.000
|
‑1.9683 To
7.9683
|
1.0000
|
5.0000
|
|
Grp 6
|
11
|
3.636
|
2.7722 To
4.5005
|
2.0000
|
6.0000
|
|
Grp 7
|
8
|
4.500
|
3.5008 To
5.4992
|
2.0000
|
6.0000
|
|
Grp 8
|
4
|
3.000
|
.0949 To
5.9051
|
1.0000
|
5.0000
|
|
Grp 9
|
7
|
2.142
|
1.3107 To
2.9750
|
1.0000
|
3.0000
|
|
Grp10
|
4
|
3.500
|
1.9088 To
5.0912
|
2.0000
|
4.0000
|
|
Grp11
|
3
|
3.666
|
‑.1280 To
7.4613
|
2.0000
|
5.0000
|
|
Total
|
54
|
3.685
|
3.2453 To
4.1251
|
1.0000
|
7.0000
|
No comments:
Post a Comment