EDUCATIONAL MATERIAL

have an awareness of social research and why it is undertaken;
understand the benefits of generating knowledge through scientific inquiry as compared with knowledge based on personal experiences;
be able to distinguish between basic and applied research;
have familiarized themselves with three main categories (or levels) of social research, namely exploratory, descriptive, and explanatory;
have developed an appreciation of the philosophical underpinnings of social research, especially the opposing paradigms of positivism and postmodernism;
understand the organic relationship between research and theory, as well as the processes of inductive and deductive reasoning;
differentiate between quantitative and qualitative research approaches;
begin to critically appraise the ideology and ethics of research, and their dynamic (changing) nature.

Introduction

This chapter is an introduction to empirical research in the social sciences. It starts with conceptualising research as a natural process of human learning about the world. It then goes on to differentiate this type of everyday life research from the so-called “scientific method” which is based on specific standards agreed upon by an international community of researchers. The chapter proceeds with explaining the difference between basic and applied research, with special reference made to action research. Next, it discusses how social studies may differ in the depth (or level) of understanding they seek to attain and distinguishes between exploratory, descriptive, and explanatory research. The chapter also analyses the relationship between research and theory, highlighting Wallace’s circular model of science which includes an iterative process of inductive and deductive reasoning. The philosophical foundations of research are then explored, focusing on the paradigmatic stances of positivism and post-modernism. Finally, the chapter ends with a discussion of quantitative and qualitative research methods, which are defined on the basis of the types of data they generate and use.

Research and the “scientific” method

From the moment we are born, we, humans, have an innate drive to discover and understand the world in which we live. The desire to learn about, and create knowledge of, our world is almost instinctive. But how is such knowledge generated? This is done through naturally observing our environment in our everyday lives, generating information (data) which we then analyse using our innate cognitive capacities to identify patterns of what happens, how it happens, and why it happens. By patterns, we mean “phenomena that occur repeatedly in life” (Walter, 2019, p. 8). Such knowledge helps us predict what will (or may well) happen under certain circumstances and decide how we should behave to have desired outcomes. This systematised – yet natural – process of knowledge generation constitutes what is known as “research”, seeking to answer questions of three main types: “what”, “how” and “why”? The first aims to describe “what is” or “what exists”, the second focuses on the way something happens, and the third seeks to explain why something happens (looking for the causes behind it). This is a process in which people engaged long before sciences (such as physics, anthropology, sociology, etc.) were established.

Let’s take a familiar example (Figure 1.1). We often see babies dropping their food over the highchair table, watching it hit the floor with delight (to the dismay of their infuriated mother who spent the whole morning preparing their favourite meal…). They often do this to discover how different food items or cutlery (of varied shapes, weights, or substances) produce different outcomes as they reach the floor. These are spontaneously planned experiments that systematically generate data which babies analyse (albeit unconsciously) to reach certain conclusions or generalisations, such as “heavy items will be loud when they reach the floor”, “glass objects may break on the floor”, or “when my soup falls on the carpet my mum starts panicking”.

Even though the above example may bring to mind the well-defined and highly structured experiments that quantitative researchers conduct, everyday research is often far less discernible and more organically embedded in daily activities. Let’s think of what happens when we join a new work setting. In the first days or weeks of taking up a new post, we are likely to feel hesitant about our actions, spending much time just observing the behaviour of, and language used by, our colleagues in an attempt to identify what (unseen) norms guide their interactions, what is that they value the most, and what sense (understandings) they make of their work. Such observations generate data which help us learn the culture of this (new to us) work setting and adapt our behaviour accordingly to become accepted members; we may work out how and when to be formal or informal, when to be serious and when not, what constitutes humour, the relative status of different people, new technical terms or commonly used turns of phrase, etc.[1] (Holliday, 2016). This natural exploration of a new culture resembles the rather flexible, unstructured, designs of qualitative research in which it is often difficult to distinguish the act of “researching” from that of actually “living in” a field (as happens, for example, in ethnography).

Having explained what research is, a reasonable question that may arise at this point is the following: if doing research is part of human nature, what is, then, so special about the “scientific” method? In fact, if we were to determine the primary components of the research we do in our daily lives, we would (most likely) realise that these make part of the very definition of “science”, namely: (a) it is a focused process (it has a purpose), (b) it is empirical (based on real-world data collected through experience), (c) it involves the analysis of data to make generalisations, (d) it attempts to explain such generalisations (i.e. builds theory), (e) it is based on logical thinking, (g) it is systematic (follows specific steps in a sequence, even though these may not be explicit or obvious to us) and (e) it accounts for existing knowledge (e.g. in our previous example, the new employee may ask another coworker if s/he thinks it would seem “disrespectful” to call their boss by their first name in order to check if his/her own conclusions agree with those of others who have worked there for longer) (Adler & Clark, 2015; Punch, 2005, Black, 2019).

So, is all research “scientific”? The answer is “yes…and no”. Walter (2019) aptly observes that a core defining feature of the scientific method is its agreed character. For our research to be considered “scientific”, it must adhere (in all its phases) to specific standards agreed upon, and shared by, an international community of established researchers. Acceptance of, and adherence to, these standards must be made explicit by the researcher, and this is what sets scientific research apart from everyday research or other ways of learning about the world (e.g. vicarious experiences, expert advice, introspection, etc.). These standards have multiple layers of detail and may differ between disciplines or schools of thought, yet they define some universal guidelines:

Careful formulation of research aims and questions (Adler & Clark, 2015).
Having an explicit structure. What is meant by “structure” is knowing what the different parts of the research are, how they connect with each other, and in what sequence. This does not imply that everything (e.g. research questions, data collection methods, analysis techniques, etc.) must be worked out in advance. Many research projects adopt loose designs in which the structure gradually unfolds as the research progresses (as happens in ethnography). What is important is that structure becomes explicit at some point between the start and the completion of a project (Punch, 2005).
Explaining the reasons for our choices: (a) why is our topic worth investigating and (b) why we believe that our chosen methods are a good fit for our research questions (a central criterion for the validity of research) (Punch, 2005). For example, it would make little sense to use self-completion questionnaires to examine toddlers’ experience of a childcare programme (they are unlikely to know how to read or write), whereas observing their behaviour during the programme would be a more suitable approach.
Sceptical re-examination of our own conclusions: the scientific method requires that the researcher is always cautious about their findings, actively searching for counterevidence against their own claims, and open to the possibility that others may challenge what initially appeared as “rock solid conclusions”. A basic assumption is that no statement can be proven beyond any doubt; between facts (data) and results always lies “interpretation” (Gannon, 2004; Black, 1999).
Transparency and replicability: this refers to the clear and open (honest) communication of the methods and procedures used to obtain results so that the same research can be replicated under identical conditions by other investigators who wish to verify the findings (Bakken, 2019).
Publication: scientific research is expected to be published in recognised journals after being reviewed (assessed) by other scientists in the field. It thus becomes subject to public scrutiny, whilst a knowledge base is created which is accessible to future researchers and provides a foundation for their projects (Walter, 2019).
Adherence to ethical principles that protect the subjects of research from undue harm and, in the case of human participants, ensure that they freely give their informed consent before they are included in a study (Walter, 2019).
Drawing on a set of “tried and tested” methods for the collection and analysis of data. This does not mean that researchers are expected to apply predefined methods precisely, as these are described in textbooks, but that they craft their own approach by adjusting those methods to the particularities of their study (Jensen & Laurie, 2016).
Making the familiar strange: this refers to the process of de-familiarising everyday things we know well and take for granted. Putting oneself at a reflexive distance from the object of a study is a key feature of the scientific method, as it is difficult to see things clearly from within a context or situation. The popular saying “culture is to humans as water is to fish” describes how the most essential things in life can be difficult to see and articulate. A fish lives its entire life in water, without necessarily seeing it or even realising its existence. Researchers must make a conscious effort to adopt the position of a stranger in the contexts they choose to study to be able to explore reality with fresh eyes without overlooking things they take for granted or (unconsciously) focusing on data that support their preconceptions. This reflexive approach is necessary in all types of research, but particularly so in qualitative research ( in the study of culture) (Holliday, 2016).

Having explained what is special about the scientific method, it becomes apparent that a key feature of this type of research is that it is extensively communicated within a community of “experts” who assess and scrutinize it based on agreed criteria to ensure “rigour”. Jensen and Laurie (2016, p.30) describe such intracommunity communication as a “collaborative way of creating knowledge” that involves “researchers critiquing and building on each other’s work”. Members of this community are theoretically of equal status, yet, in practice, hierarchies do exist (either visible or hidden) with those accumulating most power promoting certain types of research – through disproportionate funding or publishing – that (unilaterally) serve specific interests. This reflects the political nature of research which is further discussed later.

What is social research?

If research is the systematic search for answers to questions we ask about the world, then social research is concerned with questions about the social world, that is, about people and their behaviour in a social context. This broad category of research includes topics in varied fields, such as psychology (which focuses on the individual’s mental characteristics and how these affect behaviour), anthropology (which studies human cultures across time and space) or economics (that focuses on the way people produce and exchange goods or services) (Punch, 2005).

Researching the social (constructed) world is often more complicated than researching the physical world. Social phenomena are not stand-alone events; they are interwoven with human belief systems and different people may understand these in completely different ways. That is why social research does not only aim to identify social patterns (and explain these in a supposedly “objective” way) but also seeks to uncover the social meanings embedded in these patterns. “Social meanings” refer to the way people make sense of, and understand, aspects of their social lives. Such understandings differ between individuals, groups, or cultures, and therefore, social phenomena cannot be truly comprehended if such sense-making is not taken into account (Walter, 2019). For example, in the so-called “western” or “westernised” world, children working to make a living can be viewed as a factor of inequality (at the very least) or even as a form of abuse. On the other hand, in non-western societies, it can be a respected and valued way of contributing to the maintenance of one’s own family or community. Such contribution is often a source of pride and fulfilment for children in those contexts rather than being considered a symptom of oppression as happens in the so-called “liberal” and deeply individualistic societies of the West (Punch, 2004).

Walter (2019) outlines some further particularities of doing research in the social sphere:

Interaction and communication with human participants are key elements of social research. Effective people skills are, therefore, essential for social researchers, such as ease in verbal and written communication, active listening skills, and ability to relate to, and show genuine interest in, others.
Ethical constraints are much more intense in research with humans than in other types of inquiry. Such ethical constraints work towards protecting human participants from undue harm and include the challenging requirement that informed consent must be gained before any participant can take part in a study. Despite the absolute and undebatable importance of such a principle, it can clash with method selection, creating issues of validity and/or reliability, and may even lead to the cancelation of a research project.
Human participants are not always predictable. Although researchers may ask simple and straightforward questions, respondents can be dishonest or ambiguous in their answers. People are not always prepared to be frank in discussions about their behaviour, attitudes, or beliefs. They may over-report “good behavior” and under-report “bad behaviour” so as to be liked and accepted by others. It would not be surprising if, in a study of drug use, researchers found a discrepancy between participants’ self-reported frequency of smoking marijuana and the recorded number of hospitalisations as a result of consuming it.
Unlike inanimate objects, people are usually aware that they are being studied. Hence, they develop feelings and attitudes about being studied, which can, in turn, influence research outcomes. More than often, results are affected by participants’ interpretations of what a study is all about and the value they attach to it.

Purposes and levels of social research

We earlier described social research as a systematic process of discovering, and gaining deeper understandings of, the social world. It is often conducted out of a sense of curiosity about the unknown or due to an inner need for shedding light on one’s personal experiences. Research, in this case, has as sole objective the generation of knowledge for knowledge’s sake and is known as basic research. Basic research contributes to the creation of new, or the refinement of existing, theories that explain social phenomena, that is, how and why people behave, interact, or organise themselves in the ways they do (Turner, 1991, as cited in Adler & Clark, 2015). Such theories may not be of immediate practical benefit (such as offering solutions to existing problems) yet they form a foundation of knowledge on which future practical developments can be based (Black, 1999). For example, many businesses of today develop innovative strategies of staff motivation based on Maslow’s Hierarchy of Needs which is a widely known theoretical model developed decades ago (Northouse, 2016).

On the other hand, there are studies conducted with action in mind, aiming to produce knowledge that is used immediately to inform the practices of individuals, groups, or organisations. As opposed to the idea of inquiry for its own sake, this type of research is triggered by a specific practical problem or question and its whole purpose is to lead to action that solves this practical problem or answers this practical question. For example, a school may decide to conduct a study on the learning preferences of its students in order to inform its curriculum planning for the following year. This type of research is known as applied research. Even though applied studies often have a theoretical foundation, their central focus is on enhancing a real-world process, outcome or service (e.g. teaching practice or student performance) rather than refining a theoretical model or explanation (Punch, 2005; Black, 1999). Applied research includes, among others, action and evaluation research (Adler & Clark, 2015).

Action research is often treated as synonym for applied research. Yet, it would be more accurate to consider it as one type of applied research. What sets action research aside from other applied research designs is that (a) it is far more situational in nature (focused on the practices of specific individuals or groups in specific contexts), (b) it does not clearly separate the researcher from the researched (those who live in the context are often those who conduct the study), and (c) it follows a cyclical path that reflects people’s tendency to work towards solutions to their problems in cyclical (iterative) ways. For example, a collaborative community project aimed at addressing teen depression in a poor suburban district could be classified as action research, whereas a study conducted by university researchers to develop a new treatment for teen depression with a wide scope of applications worldwide is applied (but not action) research (Jensen & Laurie, 2016; Sarafidou, 2011; Punch, 2005).

Figure 1.2. presents the four main phases of the action research cycle. To illustrate how it works, let’s take an example. Let’s say an archaeological museum wants to better understand visitors’ experiences so as to identify changes that can be made to improve these. For this reason, a research team is set up (composed of staff members and patrons) to design and conduct a series of focus groups with different types of public visitors and to analyse the data generated. This is the first “research” element in the action research cycle. The team then presents results to other members of staff so that they review, together, what has been learned from the study and its implications for practice (this is the “review” element in the cycle). The next steps include creating a change plan for improving visitors’ experiences in the museum (the “plan” element in the cycle) and trying it out through implementation (the “action” element in the cycle). The cycle can start again if the museum wishes to undertake further research, with new focus groups, to assess how visitors respond to the changes made, review results, and inform new planning for new further action, and so on. As Reinharz (1992, as cited in Punch, 2005, p. 138) points out, action researchers “intervene and study in a continuous series of feedback loops”.

Figure 1.2. Action research cycle (Jensen & Laurie, 2016)

The cycle need not start with research; it can start at any phase. For example, the museum may decide to start with action by implementing a new “Augmented Reality” programme which is then followed by research evaluating its impact on the quality of visitors’ experiences. Alternatively, it can start by reviewing existing knowledge on how museums can stay relevant and attractive in contemporary cities before developing (planning) a new programme for attracting a wider range of ages, which will then be implemented (action), subsequently evaluated (research), and so on.

Evaluation research is a type of applied research that aims to assess the effectiveness of certain actions in specific contexts; it seeks to determine whether (or not) the initial objectives or intended outcomes of particular interventions or change initiatives have been achieved, how, and why. For example, an evaluation study might assess the effectiveness of a peer education programme implemented in a secondary school to improve students’ writing skills by organising them in small ‘alike’ groups in which they help each other produce different types of text (Jensen & Laurie, 2016; Adler & Clark, 2015). In fact, evaluation research could also be seen as the “research” element in an action research cycle, seeking to appraise a specific “action”.

Social research can also be classified according to the depth (or level) of understanding it seeks to attain. Hence, a study may be exploratory, descriptive, and/or explanatory (Walter, 2019).

Exploratory research focuses on a relatively under-researched, or completely new, topic so that the researcher gains familiarity and develops some initial ideas about it. It relies heavily on generating “thick” empirical data of a qualitative nature rather than on structured quantitative measures that presuppose the existence of some theory or conceptual framework. This type of research does not offer conclusive answers to research questions but lays the groundwork for more conclusive research in the future (Adler & Clark, 2015). An example of exploratory research could be the study of patients’ experiences of a new treatment through unstructured interviews and/or participant observations of their daily routines. A research strategy that is typically applied in exploratory studies is grounded theory (discussed in chapter 6.3.).

Descriptive research aims to provide a very detailed, and accurate, picture of a particular group, event, process, or situation with which the researcher is already familiar. The focus is merely on describing what is being studied without seeking to explain it. Descriptive studies often rely on the collection of large volumes of quantitative data. A population census is a typical example of descriptive research that maps the socioeconomic and cultural profile of a particular country by gathering information on every single person living within it, whilst revealing changes over time (Adler & Clark, 2015).

Explanatory research is not limited to describing what happens but is primarily aimed at explaining why it happens (building theory). Many consider this type of research as the only one that is truly “scientific”. It is much more powerful than descriptive research; the ability to explain is a core defining element of science and what differentiates “observation” from “knowledge”. Descriptive studies enjoy lower status within the scientific community than studies which aim to explain (Punch, 2005). An example of explanatory research was a series of experiments conducted by American professor Stanley Milgram in the 1960s who examined why people were ready to obey unreasonable requests from authority figures even if these were utterly unjust or cruel (Adler & Clark, 2015).

In practice, social research does not fall neatly into one category of research or another. It usually embodies different purposes and levels of investigation. For example, an exploratory study may also be used to describe the social phenomenon under investigation as well as to provide tentative explanations of it (Walter, 2019).

Theory and research

Thus far, we have referred to the concept of “theory” in a rather intuitive way, without explicitly defining it. But what exactly is theory? Put simply, theory is the attempt to explain an observable event, with the explanation being couched in more abstract terms than those used to describe it (Punch, 2005). A good example illustrating the process of theory development is presented by Field (2009). Let’s say you have noticed that your cat climbs up and stares at the TV when it is showing birds flying about but not when jellyfish are on. Your curious mind will probably urge you to come up with a plausible explanation of this behavioural pattern. You could say that fast-moving birds spark your cat’s prey drive making her jump to the screen unlike slow-moving jellyfish that fail to catch her attention. For this explanation to acquire the value of a theory it must be phrased more abstractly so that it covers a wider range of situations (other than birds, jellyfish, or just your cat). So, you could perhaps say that swiftly moving images on TV spark a cat’s hunting instinct – especially if there is an auditory component to them such as chirping – and make it cling to the screen.

The process just described is known as theory generation or inductive reasoning. That is, one starts with specific observations and infers a general conclusion from them. In the previous example, you first observed (generated data about) your pet, you then analysed those data to identify a general pattern of behaviour, and after that, you developed an explanation of that pattern (a theory explaining initial observations). It is important to highlight that theories are probabilistic in nature which means that they rarely have universal applications; they provide explanations only in terms of tendencies in groups rather than predicting accurately individual conduct. They are also dynamic in character; they are expected to change and improve, whilst they may be overturned by completely new evidence or perspectives (Black, 1999).

But how do we test the “correctness” of a theory once we have generated it? This is a process known as theory verification or deductive reasoning in which one starts from a general conclusion and applies it to specific cases or situations to see if it applies. So, once a theory has been generated, we can make predictions about what is likely to happen under certain circumstances according to what the theory maintains. These predictions are called “hypotheses”. To test our hypotheses, we need to collect new empirical data and check whether they fit our initial predictions (theory) (Black, 1999).

Let’s expand on Field’s (2009) hypothetical example of “cats watching TV”. One logical prediction (hypothesis) deduced from our proposed theory might be that if a given film is played in the homes of a randomly selected sample of cats, the majority will start watching attentively once fast-moving images appear on the screen while they will remain rather apathetic during slow motion scenes. To test this hypothesis, we can ask cat owners to keep notes of their cat’s behaviour during the film by filling in a structured observation schedule. Data gathered in this way may or may not support our initial hypothesis. If most cats behave in the predicted way, we can assert that our theory is verified. Yet, if the data contradict our hypothesis, we will inevitably conclude that our theory is “falsified” or “disproven”, or that it needs to be modified to become a better fit for the data.

In practice, research follows a cyclical path of inductive-deductive reasoning as shown in Wallace’s (1971) circular model of science depicted in Figure 1.3.

Figure 1.3. Wallace’s circular model of science (adapted from Adler & Clark, 2015)

According to this model, a theory produces specific hypotheses which the scientist must test by generating empirical data through carefully planned processes (research methods). These data are then analysed to check whether they bear out initial predictions, and ultimately, to determine if the theory provides a useful explanation of the phenomenon under study or if it requires revision. In either case, the scientist is ready to generate new hypotheses for testing and s/he may go through the cycle several times. Our observational study of cat behaviour might produce data that shows cats being attracted to fast-moving TV images only the first time these appear on the screen and that, once a cat gets used to them, she loses interest. This can lead to the modification of our theory so that it accounts for how new a given TV showing is for the cat, thus, initiating a new inductive-deductive cycle.

The philosophical foundations of social research

So far, social research and the “scientific method” have been discussed in a relatively unproblematic way. Yet, there is a crucial aspect of doing “science” that has been blatantly disregarded so far: the researcher’s philosophy of the world and of humans’ relation to it. This philosophy manifests itself when one answers two basic questions:

What is reality?
How do we know reality?

The first question is an ontological one; it refers to the nature of being (existence). The second one is an epistemological question concerned with the human mind’s relation to reality, i.e. can we know things, and if yes, how do we come to know these things, and what exactly do we know? There are different ontological and epistemological positions in relation to the above questions – known as paradigms[1]– which give rise to diverse approaches (or methodologies) to doing social research. An extensive presentation of these is beyond the scope of this book. We will limit our discussion here to two paradigmatic stances that largely shape contemporary thinking about the world and how humans relate to, and come to know, it: positivism and post-modernism (Holliday 2016; Punch, 2005).

Positivism holds that social reality is an objective entity existing independently of human perception. It also asserts that this reality is inherently ordered despite its seemingly chaotic surface. That is, social phenomena are governed by universal cause-and-effect laws in exactly the same way as natural phenomena are. Hence, human behaviour, as well as the behaviour of human systems, can be predicted and controlled if these laws are discovered. For positivists, researchers can decipher these laws through systematic (sense-based) observations of their social environment. Care must be taken so that these observations are not distorted by the observer’s prejudices but that they represent pure and accurate “facts” about the world as it really is (Coyle, 2016; Sarafidou, 2011). Positivism has been criticised for failing to acknowledge that no human representation of reality can be truly objective (valid). The social “facts” that positivists claim to provide are themselves products of human consciousness; they do not exist “out there”, in the objective world. In response to this criticism, an “amended” post-positivist philosophy emerged asserting that, whilst researchers need to pursue objectivity, they must recognise the inevitable influence of human biases. Post-positivism retains the idea of an objective reality, external to the researcher, but holds that this reality cannot be accurately represented; one can only come to know it through informed guesses or conjectures. Post-positivist researchers aim at achieving intersubjectivity (instead of objectivity) by comparing their results to those of others in an attempt to identify points of agreement about what this objective reality might be (Holliday, 2016; Adler & Clark, 2015).

Post-modernism emerged as a reaction to positivism. It straightforwardly rejected the destructive illusion that there is an external, objective reality, knowable through the application of the supposedly impersonal norms and procedures of “science”. Contrary to this nomothetic view, postmodern researchers assert that no meaningful social worlds can be discovered “out there” until they have been constructed by people. They maintain that reality cannot be studied independently of human sense-making, because what we call “reality” is the very product of human action, interaction, history, and culture. The meaning people give to their circumstances is what explains why they do whatever they do. Hence, postmodern researchers are not particularly concerned with identifying cause-and-effect relationships between variables in order to explain (or predict) observed phenomena, but they focus on uncovering people’s subjective interpretations of these phenomena and the ways in which they actively shape these. In other words, humans are considered active makers of their social worlds rather than passive pawns governed by universal laws (Holliday, 2016; Punch, 2005).[2]

From a post-modernist perspective, it is not just research participants who construct their social settings; researchers come to re-construct those settings in the course of their studies. They bring to the setting a certain ideology and a given language which bind what they “see” and what they “say”. The language researchers use does not just describe (in a transparent way) the reality being studied, but it re-creates this reality. Language is a creator of meaning; it conditions the data generated in a study by imposing limits on what can be said and how it can be said. Furthermore, the social setting is moulded during a study as the researcher interacts, and forms relationships, with participants. So the researcher and the researched co-construct the social setting, which is “doomed” to change every time a study is conducted. This is why “reflexivity” in research acquires importance in post-modern thinking; researchers are expected to identify in what ways their actions, interactions, and interpretative frameworks influence the settings being studied, their research processes, and their outcomes. Post-modernist philosophy is closely associated with other paradigms (such as relativism, interpretivism, social constructionism, or feminism) and could be viewed as an overarching term that encompasses all the others (Coyle 2016; Holliday, 2016; Punch, 2005).

Let’s take an example to illustrate how the two paradigms affect the way social research is conducted. A positivist researcher who wants to explore the smoking of marijuana by teenage boys living in a working class neighbourhood of a big city will probably aim to identify the (independent) factors associated with a high probability of an adolescent getting involved in this illegal activity by carefully operationalising the variables of interest, developing appropriate instruments to measure these variables, collecting (mainly numerical) data from large representative samples of young boys, and statistically analysing these to determine significant associations or causal links. The ultimate objective might be to propose policy measures for reducing (controlling) this “negative” (unwanted) behaviour. On the other hand, a researcher with a post-modernist perspective will probably focus on generating rich (non-numerical) data – in the form of talk, text, images, sound, etc. – by immersing herself (or himself) in a given working class neighbourhood for an extended period of time so that s/he can uncover the meanings that teenage boys attach to the act of smoking marijuana and the non-visible ways in which it shapes their lives. Uncovering what “smoking marijuana” means to research participants will ultimately explain why they do it. Such a study may not be concerned with putting forward any suggestions for policy and practice to “amend” (control) boys’ behaviour.

Quantitative and qualitative research

From the above discussion, one may infer that quantitative research is always associated with positivistic thinking, whereas qualitative research is grounded on a post-modernist philosophy. Such conclusion is neither entirely correct, nor necessarily wrong. Even though it applies to many cases, quantitative research may also be part of a post-modernist approach to doing social science, whereas qualitative research may be applied and interpreted in a positivistic way (Adler & Clark, 2015).

Technically speaking, what differentiates quantitative from qualitative research is not so much their philosophical underpinnings but the type of data they generate and use. Put simply, quantitative methods involve the generation and analysis of data that can be codified into numbers and subjected to statistical analysis, whereas qualitative research involves the generation of data that are not numerical (e.g. sounds, images, text, etc.) and which are analysed as such, using methods other than statistics (Walter, 2019; Jensen & Laurie, 2016; Adler & Clark, 2015).

Furthermore, quantitative research allows the generation of data from large representative samples of people, giving room for generalisations to be made to entire populations. Common quantitative methods of data generation include self-completion questionnaires, structured interviews, or structured observations (see chapter 5.2.). Qualitative research mainly aims at gaining deep understandings of people’s experiences in specific contexts, a process that requires rich contextual (but not large-scale) data. It focuses on smaller units of people to draw the meanings and understandings they attach to social phenomena. Common qualitative methods of data generation include unstructured interviews, focus groups, participant observations, taking photographs, creating drawings, etc. (discussed in chapter 6.2.) (Walter, 2019; Jensen & Laurie, 2016)

As both approaches have strengths and weaknesses, many studies combine these to get the best out of each one, leading to what is known as “mixed methods designs” (discussed in Chapter 8).

Concluding remarks

This chapter described social research as a natural (almost spontaneous) process of human learning about the social world. It showed that the main components of such everyday life research – i.e. focus, collection of empirical data, analysis of this data, explanation, logic, a systematic approach, and accounting for previous knowledge – coincide with the core defining features of the so-called “scientific method”. Yet, for research to be considered “scientific”, it must also adhere to a set of additional standards agreed upon, and shared by, an international community of established researchers within which certain power relationships and micropolitics also exist. Throughout the chapter, it has become apparent that researching the social (constructed) world is often more complicated than researching the physical world, because social phenomena are interwoven with human belief systems and different people may understand these in completely different ways. In an attempt to identify different types of social research, we referred to basic research, which aims at generating knowledge for knowledge’s sake, and to applied research which is conducted with action in mind. The organic relationship between research and theory was also examined, highlighting the complementary processes of theory generation (inductive reasoning) and theory verification (deductive reasoning). Furthermore, the importance of philosophy in guiding social research was emphasised and two opposing paradigms were presented, namely positivism and postmodernism. Positivism asserts that social reality is an objective entity existing independently of human perception; hence, researchers must depict this reality as accurately as possible through systematic observations, taking care so that these observations are not distorted by their prejudices. Post-modernism, on the other hand, rejects the positivistic assumption that there is an external, objective reality, maintaining that reality cannot be studied independently of human sense-making. Finally, the chapter differentiated quantitative from qualitative research, by asserting that quantitative methods involve the use of data that can be codified into numbers, whereas qualitative research involves the use of non-numerical data.

Self-assessment questions/quizzes

What is the “scientific method” and how does it differ from everyday life research? Give a practical example.
Choose one journalist’s report that is currently topical on the media. Write down what characteristics make it a piece of journalism and what you would need to do differently if you wanted to research the same topic empirically.
We hear, or read, about many different studies on a daily basis (from newspapers, professional bulletins, our bosses, friends, TV programmes, etc.). What are we supposed to believe, and how can we judge if the results are useful?
Explain the difference between basic and applied social research by giving a specific example.
Let’s assume you wanted to study the new types of romantic relationships that have been developed as a result of the increasing use of the internet and online dating. What type of research would that be? Exploratory, descriptive, or explanatory? Justify your answer.
In what ways does your intended doctoral research have the postmodern features described in this chapter?
Which of the following would be regarded as qualitative data?
Interview transcripts
Social media posts
Photographs
All of the above
Cyber bullying at work is a growing threat to employee job satisfaction. Researchers want to find out why people do this and how they feel about it. The primary purpose of such a study is:
1. Description
2. Prediction
3. Exploration
4. Explanation
Which of the following is a form of research typically conducted by managers and other professionals to address issues in their organisations and/or professional practice?
Action research
Basic research
Professional research
Predictive research

References

Adler, E. S., & Clark, R. (2015). An invitation to social research: How it’s done (5^th ed.). Stamford, CT: Cengage Learning.

Bakken, S. (2019). The journey to transparency, reproducibility, and replicability. Journal of the American Medical Informatics Association, 26(3), 185–187.

Black, T. R. (1999). Doing quantitative research in the social sciences: An integrated approach to research design, measurement and statistics. London: Sage Publications.

Coyle (2016). Introduction to qualitative psychological research. In E. Lyons & A. Coyle (Eds.), Analysing qualitative data in psychology (2^nd ed.) (pp. 9-30). London: Sage.

Field (2009). Discovering statistics using SPSS (3rd ed.). London: Sage Publications.

Gannon, F. (2004). Experts, truth, and scepticism. EMBO Reports, 5(12), 1103.

Holliday, A. (2016). Doing and writing qualitative research. London: Sage Publications.

Jensen, E. A., & Laurie, C. (2016). Doing real research: A practical guide to social research. London: Sage Publications.

Northouse, P. G. (2016). Leadership: Theory & practice (7^th ed.). London: Sage Publications.

Punch, K. F. (2005). Introduction to social research: Quantitative and qualitative approaches (2^nd ed.). London: Sage Publications.

Sarafidou, G. O. (2011). Συνάρθρωση ποσοτικών & ποιοτικών προσεγγίσεων: Η εμπειρική έρευνα [Reconciliation of quantitative & qualitative approaches: The empirical research]. Athens: Gutenberg.

Wallace, W. (1971). The logic of science in sociology. Chicago: Aldine.

Walter, M. (2019). Social research methods (4^th ed.). Oxford: Oxford University Press.

Social research strategies, research design, planning a re-search project and formulating research questions and research hypoth-eses

Learning outcomes

Upon completion of the study of this chapter the students will be able to understand the notions of social research strategies and research design, to plan a research project and to formulate research questions and research hypotheses. By comprehending and identifying the basic features of the two basic research strategies (qualitative and quantitative) and the basic steps of planning and implementing a research project, students will be able to plan their own research project and state their research question(s) and hypotheses.

Research strategies

As already mentioned, social research is the scientific study of society. It examines attitudes, beliefs, trends, stratification, and the rules of society and seeks to provide answers to or even solve social problems. Based on existing approaches, social research can be considered the systematic process of collecting, analyzing, and interpreting information and data to prove a hypothesis or answer a specific question that may help understand a social phenomenon. Social research is applied to a wide range of disciplines, such as sociology, political science, psychology, economics and the media (Bryman, 2012; Armenakis, 2021; Babbie, 2021).

By the term research strategy, we mean a general orientation to the conduction of social research (Bryman, 2012:35). In the realm of social research there are two basic strategies to follow (and their combination): quantitative research strategy, through the implementation of quantitative research methods, and qualitative research strategy, through the implementation of qualitative research methods (Adler & Clark, 2011; Armenakis, 2021, Tsigganou et al., 2018; Babbie, 2021).

Quantitative research is a research strategy that emphasizes quantification in the collection and analysis of data and that entails a deductive approach to the relationship between theory and research, in which the emphasis is placed on the testing of theories; has incorporated the practices and norms of the natural scientific model and of positivism in particular; and embodies a view of social reality as an external, objective reality.

By contrast, qualitative research is a research strategy that usually emphasizes words rather than quantification in the collection and analysis of data and that predominantly emphasizes an inductive approach to the relationship between theory and research, in which the emphasis is placed on the generation of theories. Qualitative research puts an emphasis on the ways in which individuals interpret their social world and considers social reality as a constantly shifting emergent property of individuals’ creation (Bryman, 2012:36).

Quantitative and qualitative research represent different research strategies. Each carries with it differences in terms of the role of theory, epistemological issues, and ontological concerns. However, the distinction is not an absolute one: studies that have the broad characteristics of one research strategy may have a characteristic of the other (Bryman, 2012:37). In addition, one could actually pinpoint several “common” characteristics that permeate both strategies:

They are both theory-related -literature review is a necessary step for both of them.
They require the formulation of research questions and/or research hypotheses.
It is necessary to collect data, although with different research tools and methods.
After the collection of data, the next step for both strategies is the process and analysis of the gathered information through different analytical tools.
And, of course, in both cases the last steps of the research are the presentation of the results and the relevant discussion/conclusions (Tsigganou et al., 2018).

As it has become obvious, theory is important to the social researcher because it provides a meaningful ground for the research that is being conducted, since it provides a framework within which social phenomena can be understood and the research findings can be interpreted (Bryman, 2012:20; Adler & Clark, 2011; Armenakis, 2021; Babbie, 2021). Deductive theory represents the most common view of the nature of the relationship between theory and social research. The researcher, on the basis of what is known about in a particular domain and of theoretical considerations in relation to that domain, deduces a hypothesis (or hypotheses) that must then be subjected to empirical scrutiny. In order to do that, the researcher must “translate” the hypothesis into operational terms. This means that the social scientist needs to specify how data can be collected in relation to the concepts that make up the hypothesis. Theory and the hypotheses deduced from it come first and drive the process of gathering data. In the discussion of her/his basic findings (after the research has been conducted) the researcher is engaged in a process of induction, as (s)he infers the implications of her or his findings for the theory. This deductive approach is usually associated with quantitative research (Bryman, 2012:24; Babbie, 2021).

Deductive process appears very linear-one step follows the other in a clear, logical sequence (Tsigganou et al., 2018). However, there are many instances where this is not the case: a researcher’s view of the theory or literature may change as a result of the analysis of the collected data; new theoretical ideas or findings may be published by others before the researcher has generated her/his findings, or, the relevance of a set of data for a theory may become apparent after the data have been collected (Bryman, 2012:25), especially in exploratory research (Armenakis, 2021; Babbie, 2021).

An alternative position is to view theory as something that occurs after the collection and analysis of some or all the data associated with a project (Bryman, 2012:24), the so-called inductive approach. The inductive approach emphasizes moving from more specific kinds of statements (usually about observations) to more general ones and is, therefore, a process called inductive reasoning. Many social scientists engage in research to develop or build theories about some aspect of social life that has previously been under-researched. Theory that is derived from data in this fashion is sometimes called grounded theory (Adler & Clark, 2011:33). According to the inductive rationale, theory is the outcome of research. In other words, the process of induction involves drawing generalizable inferences out of observations (Bryman, 2012:26; Armenakis, 2021).

To a large extent, deductive and inductive strategies are possibly better thought of as tendencies rather than as a clear-cut distinction (Bryman, 2012:27; Adler & Clark, 2011:32). It is probably most useful to think of the real interaction between theory and research as involving a perpetual flow of theory building into theory testing, and back again (Adler & Clark, 2011:34).

Choices of research strategy, design, or method have to serve answering the specific research question(s) under scrutiny. If we are interested in teasing out the relative importance of a number of different causes of a social phenomenon, it is quite likely that a quantitative strategy will fit our needs. Alternatively, if we are interested in the world views of members of a certain social group, a qualitative research strategy that is sensitive to how participants interpret their social world may be more appropriate. If a researcher is interested in a topic on which no or virtually no research has been done in the past, the quantitative strategy may be difficult to employ, because there is little prior literature from which to draw leads. A more exploratory stance may be preferable, and, in this connection, qualitative research may serve the researcher’s needs better, since it is typically associated with the generation rather than the testing of theory and with a relatively unstructured approach to the research process.

Factors influencing the choice of a research strategy

A factor influencing the choice of the different research strategies and the subsequent implementation of the research method(s) is the values of the researcher. Values reflect either the personal beliefs or the feelings of a researcher. According to the Weberian rationale on the “neutrality” of the researcher, we would expect that social scientists should be value free and “objective” in their research. Such a view is held with less and less frequency among social scientists nowadays, since there is a growing recognition that it is not feasible to keep the personal values “out of the research”. These values can intrude at any or all stages of social research (e.g., choice of research area, formulation of research question, interpretation of data, conclusions). It is quite common, for example, for researchers working within a qualitative research strategy, and in particular when they use participant observation or very intensive interviewing, to develop a close affinity with the people whom they study to the extent that they find it difficult to disentangle their stance as social scientists from their subjects’ perspective (Bryman, 2012). Nowadays, most researchers acknowledge that research cannot be value free, but one must ensure that values do not dominate the scientific implementation of the research method(s), even though values can serve as motivation for the conduction of specific research projects (e.g., in feminist studies/feminist social research) (Bryman, 2012:39-40).

Another dimension influencing the decision on the research strategy to be followed has to do with the nature of the topic and of the people being investigated. For example, if the researcher needs to engage with individuals or groups involved in illegal activities, such as gang violence or drug dealing, (s)he would preferably use a qualitative strategy where there is an opportunity to gain the confidence of the subjects of the investigation or even in some cases not reveal their identity as researchers, albeit with ethical dilemmas, as we shall discuss in the next chapter of the educational material at hand.

Research methods are associated with different kinds of research design. The latter represents a structure that guides the execution of a research method and the analysis of the subsequent data, it actually guides the implementation of the decided strategy (Bryman, 2012:45). A research method is a technique for collecting and/or analyzing data. It can involve a specific research instrument/tool, such as a self-completion questionnaire or an interview guide, or participant observation whereby the researcher listens to and watches others, or a technique for analyzing the data gathered such as thematic analysis, (critical) discourse analysis etc. (Bryman, 2012; Babbie, 2021).

All in all, social research is a coming-together of the ideal and the feasible (Bryman, 2012:41), and the researcher has to make her/his strategic and on-the-spot decisions in order to overcome any obstacles that might encounter in her/his way. On top of that, one should bear in mind that there is no “perfect” research strategy or research project. Thus, any researcher should be aware and acknowledge the limitations of her/his research.

Evaluation of social research

Three of the most prominent criteria for the evaluation of social research are reliability, replication, and validity. Reliability is concerned with the question of whether the results of a study are repeatable. The term is commonly used in relation to the question of whether the measures that are devised for concepts in the social sciences (such as poverty, racial prejudice, deskilling, religious orthodoxy) are consistent. Reliability is particularly at issue in connection with quantitative research. Ιf, for example, IQ tests were found to fluctuate, so that people’s IQ scores were often wildly different when administered on two or more occasions, we would be concerned about it as a measure. We would consider it an unreliable measure (Bryman, 2012:46).

The idea of reliability is very close to replicability (though in social research replication is very rare as a practice). There may be a number of different reasons for trying to replicate the findings of a research, such as a feeling that the original results do not match other related evidence (counter-intuitive results). In order for replication to take place, a study must be replicable, hence the research procedures followed must be documented in great detail (Bryman, 2012:47).

A further and in many ways the most important criterion of research is validity. Validity is concerned with the integrity of the conclusions that are generated from a piece of research and can be divided in three ‘categories”: measurement validity, internal validity, and external validity.

Measurement validity applies primarily to quantitative research and to the search for measures of social scientific concepts. Measurement validity is also often referred to as construct validity. Essentially, it is to do with the question of whether a measure that is devised of a concept really does reflect the concept that it is supposed to be denoting. Does the IQ test really measure variations in intelligence? Measurement validity is related to reliability: if a measure of a concept is unreliable, it simply cannot be providing a valid measure of the concept in question (Bryman, 2012:47).

Internal validity is concerned with the question of whether a conclusion that incorporates a causal relationship between two or more variables is valid. If we suggest that x causes y, can we be sure that it is x that is responsible for variation in y and not something else that is producing an apparent causal relationship? In discussing issues of causality, it is common to refer to the factor that has a causal impact as the independent (control) variable and the effect as the dependent (interest) variable. Thus, internal validity raises the question: how confident can we be that the independent variable really is at least in part responsible for the variation that has been identified in the dependent variable[1]? (Bryman, 2012:47; Armenakis, 2021). We will further elaborate on that relationship in the quantitative research section, and more specifically when we discuss the statistical tests.

External validity is concerned with the question of whether the results of a study can be generalized beyond the specific research context (Bryman, 2012:47; Babbie, 2021).

Various kinds of research

Research projects, according to their scopes can be classified in several categories:

Exploratory research constitutes groundbreaking research on a relatively unstudied topic or in a new under-researched area (Adler & Clark, 2011:13) (e.g., researching the effects of quarantine due to a pandemic).
Descriptive research, a researcher describes groups, activities, situations, or events, with a focus on structure, attitudes, or behavior. Researchers who do descriptive studies typically know something about the topic under study before they collect their data, so the intended outcome is a relatively accurate and precise picture. Examples of descriptive studies include the kinds of polls done during political election campaigns, which are intended to describe how voters intend to vote (Adler & Clark, 2011:14).
Analytical–Explanatory-Interpretive, research to analyze, explain and interpret “cause-effect” relationships that permeate the characteristics of a situation, a subject, or a phenomenon (e.g., research on the reasons for using social media) (Armenakis, 2021).
Basic/fundamental research is designed to add to our fundamental understanding and knowledge of the social world regardless of practical or immediate implications (Adler & Clark, 2011:11; Armenakis, 2021) (e.g., the ideological orientation of Media outlets).

[1] Dependent variable, a variable that a researcher sees as being affected or influenced by another variable (contrast with independent variable). Independent variable, a variable that a researcher sees as affecting or influencing another variable (contrast with dependent variable) (Adler & Clark, 2011:24).

The fact that two variables are associated with each other doesn’t necessarily mean that change in one variable causes change in another variable (Adler & Clark, 2011:25).

Applied research is intended to be useful in the immediate future and to suggest action or increase effectiveness in some area (Adler & Clark, 2011:11, 382; Armenakis, 2021) (e.g., which measures do citizens prefer to implement in terms of tackling climate change).
Evaluation research is research designed to assess the impacts of programs, policies, or legal changes. It often focuses on whether a program or policy has succeeded in effecting intended or planned change, and when such successes are found, the program or policy explains the change (Adler & Clark, 2011:16, 383). The most common evaluation research is outcome evaluation, which is also called impact or summative analysis. This kind of evaluation seeks to estimate the effects of a treatment, program, law, or policy and thereby determine its utility. These research projects typically begin with the question “Does the program accomplish its goals?” or the hypothesis “The program or intervention (Adler & Clark, 2011:383) (independent variable) has a positive effect on the program’s objectives or planned outcome (dependent variable).”
Cross sectional research is research that is conducted only once at a specific point in time (Armenakis, 2021).
Longitudinal research is research repeatedly conducted at regular or irregular intervals (e.g., ESS- European Social Survey, EVS- European Values Survey, Eurobarometer) (Armenakis, 2021).

A research plan (steps to conduct a research)

There is a range of possible ways to choose a research project, such as personal scientific interest, interest in improving the researcher’s knowledge and skills on a specific issue, theory, research literature, puzzles/theoretical debates, new developments in society, social problems, discussions with colleagues, accessibility of data, willingness to collaborate with competent authorities or research populations, eagerness to resolve issues that may rise in relation to the uncertainty of the existence / collection of data (Tsigganou et al., 2018; Bryman, 2012:88).

In any case, not all ideas, no matter what has triggered them, are adequate to become research projects. Thus, a researcher should try to avoid extremely large “panoptic” subjects, common issues that have been already extensively scrutinized, issues for which there is no research material or access to the material is extremely difficult or impossible (e.g., research that has to do with content that can be found only in archives, very technical and highly specialized topics for which it is difficult to gather a significant number of participants/units of analysis (sample), issues whose course depends on the completion of other research projects (especially when our time-frame is short), issues that might violate the codes of ethics and ethics of social research (Tsigganou et al., 2018).

The basic steps towards the implementation of a research include (in most cases) the following:

an overview of the theory through the review of the relevant literature,
the formulation of research questions and research hypotheses,
the data collection,
the data analysis and interpretation (seeking to answer the research questions and hypotheses)
and the discussion including the limitations of the research and new research ideas (Tsigganou et al., 2018).

Research questions-Research hypotheses

Usually, researchers begin their research with a research question, or a question about one or more topics that can be answered through research (Adler & Clark, 2011:2). Whereas research projects that have explanatory or evaluation purposes typically begin with one or more hypotheses, most exploratory and some descriptive projects start with research questions (Adler & Clark, 2011:73). Research questions should be clear, in the sense of being intelligible. Research questions are similar to hypotheses, except that a hypothesis presents an expectation about the way two or more variables are related, but a research question does not. Both research questions and hypotheses can be “cutting edge” and explore new areas of study, can seek to fill gaps in existing knowledge, or can involve rechecking things that we already have evidence of (Tsigganou et al., 2018).

Several steps are needed to turn a research question into a researchable question, a question that is feasible to answer through research. The first step is to narrow down the broad area of interest into something that’s manageable. You can’t study everything connected to cell phones, for example, but you could study the effect of these phones on family relationships. You can’t study all age groups, but you could study a few. You might not be able to study people in many communities, but you could study one or two. You might not be able to study dozens of behaviors and attitudes and how they change over time, but you could study some current attitudes and behaviors. While there are many research questions that could be asked, one possible researchable question is: In the community in which I live, how does cell phone use affect parent-child relationships; more specifically, how does the use of cell phones affect parents’ and adolescents’ attempts to maintain and resist parental authority? (Adler & Clark, 2011:88) The research questions we choose should be related to one another. If they are not, our research will probably lack focus and we may not make as clear a contribution to understanding as would be the case if research questions were connected (Bryman, 2012:90).

In brief terms, research questions:

Should be researchable—that is, they should allow you to do research in relation to them. This means that they should not be formulated in terms that are so abstract that they cannot be converted into researchable terms.
Should have some connection(s) with established theory and research. This means that there should be a literature on which you can draw to help illuminate how your research questions should be approached. Even if you find a topic that has been scarcely addressed by social scientists, it is unlikely that there will be no relevant literature (for example, on related or parallel topics).
Should be linked to each other. Unrelated research questions are unlikely to be acceptable, since you should be developing an argument in your dissertation. You could not very readily construct a single argument in relation to unrelated research questions.
Should be able to make an original contribution -however small- to the topic.
Should be neither too broad (so that you would need a massive research project to study them), nor too narrow (so that you cannot make a reasonably significant contribution to your area of study) (Bryman, 2012:90; Tsigganou et al., 2018).

On their behalf, research hypotheses aim to interpret the relationship between theory (as described/presented in the essay) and research findings. Research hypotheses describe the questions, making judgments about their answer (according to the theory). Consequently, the results of a research are intended to verify or not the assessments made by research hypotheses. Rejecting a research hypothesis does not necessarily mean that it is wrong or that the implementation of the research is wrong. It may mean that a theoretical approach does not (fully) apply/correspond to the case of our research (Tsigganou et al., 2018).

Research hypothesis assessment starts with a theory or a claim about a specific population parameter. In order to examine if the claim will be confirmed by the research results, we state a pair of research hypotheses:

The null/working hypothesis (H0) usually represents the existing situation as described in our theory. This assumption is true to the point that we have sufficient evidence to reach the opposite conclusion. Whenever a null working hypothesis is specified, an alternative hypothesis is also specified and must be true if the null is false (the two hypotheses are mutually exclusive).
The alternative hypothesis (H1/HA) is the opposite of null hypothesis and covers all other cases not covered by zero (Tsigganou et al., 2018). Thus, the pair of research hypotheses is “exhaustive” in the sense that either the H0 or H1/HA will be true and there is no other (third) option.

It is not necessary to describe both hypotheses (null and alternative) in the context of a research text.

Let’s assume that we want to conduct research on the TV viewing habits of Greek people. According to our theory “the average TV viewing time in Europe for 2017 was 2 hours and 56 minutes”. Having this information as a reference point, we could formulate our null and alternative hypotheses as follows:

(H0): Based on the data available for television viewing in Europe, we expect the average viewing time in Greece to be 2 hours and 56 minutes (so μ = 2 hours and 56 minutes).

(H1/HA): Average view time isn’t 2 hours and 56 minutes (so μ ≠ 2 hours 56 minutes).

Conclusion

The choice of research strategy(ies) and the subsequent research method(s) to be implemented are the first steps towards the implementation of a research project. On top of that, the researcher should be able to formulate adequate research questions that actually “drive” the whole research procedure, and research hypotheses (if needed) that operationalize the theoretical background -needed in both qualitative and quantitative research- and provide the basic directions/parameters of study to the research project.

Exercise

You are interested in finding out which teaching methods are more “attractive” to university students. How would you formulate your main research question? What research strategy would you follow to answer your main research question? Which are the basic steps you would implement towards the completion of your research project?

Questions

What is the difference between a research strategy and a research method?
Name some of the similarities and differences between the qualitative and quantitative research strategies.
Which are the two basic characteristics of the H0 and H1/HA?
In which ways as the working hypotheses related to theory?
Which are the different categories of validity?
Which are the main aims of exploratory and descriptive research?

References

Adler, S. E. and Clark R. (2011). An Invitation to Social Research: How It’s Done (4^th Ed.). Wadsworth Cengage Learning.

Armenakis, A. (2021). Research Methods. Learning Material for Postgraduate students of the Department of Communication and Media Studies of the National and Kapodistrian University of Athens.

Babbie, E. R. (2021). The Practice of Social Research (15^th Ed.). Cengage.

Bryman, A. (2012). Social Research Methods (4^th Ed.). Oxford University Press.

Tsigganou, I., Iliou, K., Fellas, K. N, Balourdos, D. and Giakoumatos, S. (2018). Research Methods. Learning Material for the National Center for Public Administration and Local Government.

Further Study

Bailey, K.D. (1978). Methods of social research. Free Press.

Crotty, Μ. (1998) The foundations of social research. Sage Publications.

Flick, U. (2009). An Introduction to Qualitative Research. 4th Ed. Sage.

Gray, D. E. (2004). Doing Research in the Real World. Sage Publications.

Kumar, R. (1999). Research methodology. A step-by-step guide for beginners. Sage Publications.

Mertens, D. (1998). Research methods in education and psychology: integrating diversity with quantitative & qualitative approaches. Sage Publications.

Pickering, M. (2008). Research Methods for Cultural Studies. Edinburgh University Press.

Theory and research: Literature review- Writing up social research

This chapter examines various methods of documenting social research to offer fundamental concepts for structuring our own written work, particularly if tasked with producing a dissertation.

Learning outcomes

By the end of this chapter, students will be able to understand the importance of writing, particularly effective writing, in social research; identify how quantitative, qualitative, and mixed methods research are written, with examples; recognize the expectations and conventions of writing for academic audiences by identifying the elements of a research report and the various formats in which research might be written.

Introduction

Regardless of the size of a research project, it’s easy to forget that writing it up is a critical part of the process. Writing not only presents our findings but is also vital for persuading our audience of the research’s validity and significance. If findings of studies are not appropriately presented, all the various approaches mentioned in the previous sections will serve nothing. This means that proficient English, or whatever is the usual language one uses, is a minimum requirement for a good social report. When we use extremely complicated terms and constructions, communication is hampered (Babbie, 2021).

Scientific reports have several purposes. According to Bryman (2016), first, our report must clearly convey a specific set of data and ideas, providing enough detail for others to evaluate it thoroughly. We should view our report as a contribution to the collective body of scientific knowledge. While maintaining humility, we must also recognize that our research adds to the broader understanding of social behavior. Lastly, our report should inspire and guide future investigations.

Bryman (2016) emphasizes on the following points:

On time

With the important tasks of gathering and analyzing one’s data, writing is often overlooked and undervalued. While writing up one’s findings cannot occur until after the data have been analyzed, it is advisable to begin writing much earlier. Getting a head start on writing can help the researcher clearly organize the research questions, set the literature in context, and avoid the common mistake of underestimating how long it takes to make revisions to the work. Procrastination is one of the factors that lead to writing at the last minute, which is not ideal. It is very crucial for success to write carefully and early since a full presentation of the results and conclusions of the research is needed to give credit to the study.

Persuasion

Researchers need to persuade readers that their conclusions are believable, important, and plausible. It is not sufficient to present findings and fail to convince the audience of their importance. Writing up research involves more than reporting findings and conclusions; relevant literature needs to be integrated, the research process has to be accounted for, and the analysis has to be described.

Language

There should be no discriminatory, sexist, or disablist language in researcher’s writing.

Feedback

Seek as much feedback on your writing as possible, particularly from your supervisor – make sure to give him/her enough time to comment thoughtfully. You may also want to ask peers, but your supervisor’s feedback will likely be the richest.

Key Dissertation Components

Title Page

This page includes your institutional requirements as it may also contain the dissertation title, the name of the author, the degree, and the date of submission.

Acknowledgements

These are credits for individuals who may have guided or assisted you in your work-for example, gatekeepers, colleagues who may have commented on the work, or your supervisor for guidance and advice.

List of Contents

A standard list of your dissertation’s chapters and sections.

Abstract

A summary of the whole dissertation, including your questions, methodology, results and conclusion.

Introduction

Tell the reader what this is about and why it matters. State your theoretical position, your research questions and give a clear direction for your study. Try not to make general statements at the outset but give a robust definite direction to your reader.

Literature Review

A review of the literature and relevant theories, related to your research question or topic. A research question may often be developed within this section, along with locating and positioning your study within the broader academic landscape.

Research Methods

This provides the details of the design of your research, the methods of sample selection, data collection, and analytical techniques. You are expected to justify the choices you made as well as the minor and/or major problems which occurred in the process of conducting the research.

Results

This chapter contains the key results of your findings. Keep in mind:

Only report findings relevant to your research questions-even if this means omitting other results-so that the clarity of argument is maintained.
Graphs, Tables and Analyses: Just relate important features of tables, graphs or analyses to your research questions. Do not summarize but give the key message on behalf of data.
Always comment on graphs, tables or transcripts since their importance needs to be explained rather than leaving the reader to second guess it.
Quantitative results shall be diversified and represented in different formats: figures, tables, coupled with the most appropriate method of analysis.
Avoid irrelevant information; in qualitative studies, this is necessary if one wants to maintain focus. Try not to use descriptions that are too complicated; bear in mind your research questions and informing literature.
In theses, structure result chapters with clear signposting, linking research questions to the content of the chapter, and provide conclusions that connect to subsequent chapters.

Discussion

Consider how the results answer your research questions. Comment on whether a hypothesis was proved or disproved; if not, give some speculation as to why and what it could mean.

Conclusion

Key considerations for writing an effective conclusion:

Conclusion is not a summary; however a brief recapitulation of your argument and findings in light of your research questions may bring out their significance.
Clearly state what your findings imply for your research questions.
Highlight how your findings might affect theories related to your research area.
Acknowledge the limitations of your research, but without overestimating.
Suggest areas for further research that emerge from your findings.
Avoid making claims that are speculative beyond data or that introduce new ideas not discussed previously.

Appendices

Material to go in appendices includes, for example: your observation schedule, coding frame, questionnaire, letters/correspondence sent to sample members, or to gatekeepers where organisational collaboration was necessary.

References

List all sources cited in the text here.

Writing up Quantitative, Qualitative and Mixed Methods Research

Based on Bryman (2016), this section examines research articles in quantitative, qualitative, and mixed methods to identify helpful writing features. A common question is whether researchers in these strategies use different writing approaches. While it is often assumed that qualitative and quantitative researchers write differently, one notable difference is that quantitative researchers tend to provide more detailed accounts of research design, methods, and analysis in journal articles compared to their qualitative counterparts, who typically offer more thorough explanations in books. Wolcott (1990) also observed this tendency, noting that qualitative researchers often fail to provide sufficient detail about data collection procedures, leading to credibility issues. However, aside from this point, the similarities in writing between quantitative and qualitative articles are often more significant than the differences. In addition to comparing quantitative and qualitative writing, this section will explore how mixed methods research can be effectively written and discuss emerging guidelines for this approach. While the examination of mixed methods research writing takes a slightly different approach, the discussion highlights key practices for clear and credible presentation across different research strategies.

Writing up Quantitative Research

In both qualitative and quantitative research, the articles maintain a similar structural form. They start with an abstract that briefly outlines the importance of the research, the main research question, and state the conclusions in about 200 words. Quantitative articles, therefore, proceed with the introduction containing attention getters, statement of the reason for the research, research questions or hypothesis, and sets used in methods and data collection. Then, the results are brought in relation to the hypotheses and, finally, implications for the theoretical framework. Quantitative research explicitly details the steps for data collection, measurement, and sampling, and all parts of each are labeled for clear identification.

A typical quantitative article includes the following sections:

Introduction: The topic and its importance are identified within the first couple of sentences. It places the article within the greater context of the literature, describing the contribution of the article.
Theory: Description of theories relevant to the subject. The article may also go ahead and state the hypotheses that have been tested.
Data: Sampling procedure, including sample size, data collection, and response rate. If secondary data are used, then how they were collected and applied to the research.
Measurement: Explains how the key concepts are measured, for instance, how social capital is measured by survey questions.
Methods and Models: Explains the relationships between key variables and how the data are analyzed.
Results: Lengthy discussion of results and if hypotheses are supported.
Conclusion: Discussion of broader implications of the findings relating back to the introduction and theory, suggesting additional research.
Bibliography: Should be accurate, use a recognized style (ASA, APA, etc.), and include access dates for online references.

In general, quantitative articles present the purpose of the research, research questions, data, results, and conclusions in that order. Sometimes sections like Data, Measurement, and Methods are combined, and at other times the Results section is followed by a Discussion section before Conclusions.

Writing up Qualitative Research

A qualitative research article is similarly structured to a quantitative article in many respects. The author should engage the reader early on, followed by the research rationale, the research questions, the research design and methods, the findings and concepts, and finally an interpretation regarding the theoretical contribution of the study within the greater context of existing knowledge. The research questions in qualitative research are also generally broader than in quantitative studies. In qualitative research, less attention is given to elaborations of the research design or methods.

The main elements of a qualitative article include:

Introduction: The first few sentences clarify exactly what the article focuses on and tries to create interest on behalf of the reader in the study at hand.
Literature Review or Background: A review of relevant theories and previous research on the topic.
Design and Methods: Gives the rationale for a qualitative methodology, describes the research procedure, participant sampling, setting, and the analytical procedures used, including thematic analysis.
Findings: Presents results and what they mean and why they are important.
Discussion: The findings are related to the research questions and theory presented. Often findings will be divided into different sections; suggestions might be made as to how initial concepts were modified or if new theory was developed.
Conclusions: This section links back to the introduction and literature review, discussing implications for the field, discipline or public policy.
Bibliography: Same as in quantitative articles.

In a nutshell, qualitative articles have broader and more open-ended research questions, have less detail in the methods section, and often expand or refine initial concepts in the discussion. This reflects the inductive nature of qualitative research where theories and concepts evolve through the study.

Writing up mixed methods research

As mixed methods research is relatively new, conventions in writing mixed methods articles are not well established; it is very difficult to determine how an exemplary mixed methods journal article should look. A clear research problem or question will usually begin the articles, then model once from quantitative and qualitative research writing conventions. Two criteria for high-quality mixed-method articles, according to Creswell and Tashakkori (2007), are (1) that both the quantitative and qualitative components of the work need to be well developed and competently executed and (2) the two strands of the research need to be integrated rather than presented separately. This integration should help provide a more comprehensive understanding of the research problem. They further add a third criterion—that the article should also contribute to the literature of mixed methods—which probably sounds like a lot to most researchers.

The first is that the mixed-methods study should meet minimum requirements of both quantitative and qualitative research in stipulating clear specification of research questions, sampling strategy, data collection, and data analysis approaches for each component. The second criterion highlights the significance of connecting both datasets for maximum utility to be derived from the study. According to Creswell and Tashakkori (2007), the attempt is to combine the conclusions of the two strands in order to provide a more complete explanation of the research problem or phenomenon. To this effect, integration could be easily realized if the researcher made it clearer why the inclusion of both approaches was used. Creswell and Plano Clark (2011) also add on the structure of the mixed methods journal articles in which the integrated findings are clearly presented.

Introduction: In the Introduction, the statement of the problem or issue studied should be given. It should contain a literature review concerning the study in hand and describe gaps discovered in earlier research. This may be discussed by the fact that most of the earlier studies had either adopted a quantitative or qualitative approach and how a mixed-methods method is advantageous. The research questions shall also be introduced.

Methods: In the Methods section, the rationale for using a mixed methods design should be provided, including identification of the design type. Further, describe the collection techniques and methods of data analysis that are to be used, discussing how the quality of the data will be assessed.

Results: The results section may combine quantitative and qualitative results, or there may be two consecutive reports; in this case, the two results sections should be discussed together.

Conclusion: The Conclusion Section summarises and interprets the findings, and emphasis is on added value from the integration of both quantitative and qualitative data Emphasis must be put on the mixed-method approach employed and how it adds up to a better explanation of the research problem. So, the mixed methods approach of the study is reaffirmed in the Conclusion section, highlighting its goal of offering a thorough understanding of the subject. The research’s primary contribution is then emphasized, with special attention to its applicability in light of the current situation. The study’s limitation is then addressed, but the attention is soon drawn back to the findings’ extensive detail and depth. Thereafter, prospective directions of inquiries for further research are mentioned. The section concludes with a key takeaway.

Academic Writing

It is crucial to understand that academic writing is a specific type of technical writing with its own set of rules when you get to the point of writing up your research. Even though this chapter has addressed many of these standards, it’s crucial to familiarize yourself with the wider conventions of academic discourse in addition to adhering to your university’s rules. Clarity and precision are required in academic writing because readers frequently take offense at ambiguity or the exclusion of necessary details.

Consider reading a range of scholarly publications to comprehend these conventions in practice in order to produce high-quality work. Keep an eye out for the way literature reviews are presented and connected to research questions, the way arguments are built and sustained throughout the text, and the kind of content that works best when it comes to presenting findings, discussions, and conclusions. Keeping an eye on these components can give you important tips for producing quality academic writing.

Keep in mind that the core purpose of writing is to persuade. Your objective is to persuade readers of the value, importance, and contribution your work provides to the field. Writing for this purpose implies capturing and retaining the reader’s interest. Writing that engages the reader goes beyond simply providing facts; it also tells an appealing story about the goal and conclusions of your study.

Students frequently make the mistake of writing too quickly. Writing quickly increases the likelihood that the development and upkeep of a cogent argument as well as writing style will receive inadequate attention. No matter how important your findings are, such mistakes might lessen the effect of your study.

It’s also critical to understand that academic writing conventions are independent of methodological differences. Following these guidelines will make your work trustworthy and accessible to a broad scholarly audience, regardless of whether your research is mixed methods, quantitative, or qualitative. By keeping your presentation consistent, readers are less likely to become confused from your contributions by uncommon presentation.

Finally, remember that research requires a sustained commitment. Select a subject that you are truly interested in whenever at all possible. Getting involved with a topic you are enthusiastic about improves the quality of your writing and adds enjoyment to the research process. Readers will be able to see your excitement, which will increase the persuasiveness and appeal of your writing.

In short, it is imperative to approach the writing step with meticulous organization and attention to detail. You put yourself in the best possible position to write a well-written dissertation or research paper by becoming familiar with academic standards, giving yourself plenty of time for writing and revisions, and choosing a topic that interests you. This methodical technique improves your capacity to make a significant contribution to your profession while also fortifying your case. Recall that good research requires effective writing since it serves as a medium for disseminating your thoughts, discoveries, and contributions to the public.

Key Remarks

Why arguments are important

Writing up their research, students find it very hard to bring the logical flow of their argument across. The central argument is the backbone that holds your research write-up together-from the statement of the problem and the literature review to the specification, presentation of your research methods followed by discussion and conclusion. Students simply write up a list of points without explaining how each of these contributes to the main argument they are trying to make. They need to understand their main claim or contribution of knowledge and write in a manner that sets out to show and emphasize this claim, which then becomes their argument. This way, the story can outline the applied nature of your research.

By placing your work within a narrative, you have an arc that then takes the reader through the study. In this way, it prevents the digressions to irrelevancies that only subtract from your argument and obscure what you were trying to argue in the first place. If you have digressed, such asides of this type might dilute an otherwise tight argument or even mask for the reader the sheer weight of your research. What would that central argument be, and then you get the “so what?” question of relevance. Good insurance against this possible pitfall is to start with this question: “What, in fact, is the central point or message that I want my readers to walk away with when they finish my work?” If you can’t answer that question, you probably haven’t developed an argument. It will also be a good chance to share your point with others and see what they can understand and appreciate from this message.

Whether it is a dissertation or research work, an argument is always some sort of thread throughout the whole work. This thread makes coherent all its components; that is to say, beginning with the main claim, each one of them develops and supports. You make the research more persuasive and interesting for the audience by keeping a close focus on the argument. At the end, a sharply focused argument makes both your writing itself and the great value of your contribution stronger.

Why research questions are important

Research questions are very important throughout the course of research and writing as they summarize the focus and direction of the piece. They enable the selection of findings to be included or those to be emphasized in maintaining the relevance and coherence of writing. The research questions, combined with the underlying theory and literature, allow one to refer to them regularly in organizing work and keeping the thread constant throughout.

Besides, structuring a thesis or dissertation with specific questions of research allows for logical flow of ideas and themes throughout. Further, this allows structuring the chapters in addressing each main research question, thereby making sure that the contents are directly related to the objectives of the study. This keeps the focus and relevance to the subjects in question and inhibits irrelevant material from finding its way into the research document.

Before writing, it is effective to plan in detail what the major points and ideas to be communicated are, considering how findings relate to the research questions. This will ensure that the discussion remains on target and that every piece of data will answer the main questions of the research.

Similar questions and responses can be combined, helping to develop thematic chapters or parts of the work. These allow the themes that emerge from the data and that are consistent across responses to be combined to present a comprehensive perspective on the topic. It is better in terms of giving cohesion to the writing and noticing how different parts of the research link together.

Also, the integration of relevant theoretical debates and frameworks related to the research area is necessary. Theoretical framing of the study forms a basis on which one could discuss empirical findings and show how the research contributes to available knowledge. This link between theory and data strengthens the overall argument and underlines the importance of the study.

In short, research questions are crucial in that they guide the selection and emphasis of findings, writing in a well-organized framework, and keeping the focus of the study on its real aims. They enable researchers to evolve an intelligible argument connecting all parts of the research from the literature review to the conclusions, hence making their work more easily understood, persuasive, and influential.

How to cite bibliographic resources

Proper citation is important not only in your literature review, but in any parts of your paper, and so is proper form of the citation itself. The good news is that proper citation is quite easy. The not-so-good news is that there are several styles now in wide use. We’ll illustrate here some of the most common formats (Babbie, 2021):

Book Information

Author: Alan Bryman

Title: Social Research Methods

Publisher: Oxford University Press

Year of Publication: 2016 (5th Edition)

Citations

APA Style (American Psychological Association), 7th Edition:

Bryman, A. (2016). Social research methods (5th ed.). Oxford University Press.

ASA Style (American Sociological Association), 6th Edition:

Bryman, Alan. 2016. Social Research Methods. 5th ed. Oxford: Oxford University Press.

APSA Style (American Political Science Association), 17th Edition:

Bryman, Alan. 2016. Social Research Methods. 5th ed. Oxford: Oxford University Press.

MLA Style (Modern Language Association), 9th Edition

Bryman, Alan. Social Research Methods. 5th ed., Oxford University Press, 2016.

Article Information

Author: Mark Gottdiener

Title: “Disneyland: A Utopian Urban Space”

Journal name: Urban Life

Year of publication: 1982

Volume: 11

Number: 2

Pages: 139–162

Citations

APA Style (American Psychological Association), 7th Edition:

Gottdiener, M. (1982). “Disneyland: A Utopian Urban Space”, Urban Life, 11: 139–62

ASA Style (American Sociological Association), 6th Edition:

Gottdiener, Mark. 1982. “Disneyland: A Utopian Urban Space.” Urban Life 11(2):139–62.

APSA Style (American Political Science Association), 17th Edition:

Gottdiener, Mark. 1982. “Disneyland: A Utopian Urban Space.” Urban Life 11 (2): 139–62.

MLA Style (Modern Language Association), 9th Edition

Gottdiener, Mark. “Disneyland: A Utopian Urban Space.” Urban Life, vol. 11, no. 2, 1982, pp. 139-62.

Why proofreading is important

The clarity and professionalism of your writing can be greatly impacted by language and punctuation issues, thus it is imperative that you proofread your dissertation to make sure that these faults are absent. Carefully reviewing your work before submitting it helps you identify errors that could otherwise damage your research’s credibility. Making use of the many useful guides and instructions available will help you spot frequent mistakes and raise the standard of your work overall.

Furthermore, because they are so familiar with the content, even the most careful writers are prone to forgetting their own errors. Thus, getting a friend or family member involved in the proofreading process can offer another viewpoint. This is because the most cautious writer will overlook his own mistakes, having become overly familiar with what he has written. Getting your friend or family involved in proofreading can provide fresh eyes as well. They may be able to catch oversights, mistakes, and inconsistencies that you do not see, and are thus valuable in the cleanliness of your dissertation. The help of this colleague in such regard contributes not only to correcting the overlooked mistake but also to the readability and coherence of the argument.

Not proofreading means your ideas can be misunderstood, or their meaning may be mixed up, thereby reducing the impact of your research findings. Grammatical and punctuation errors distract readers, which also include examiners, and can give the impression of carelessness or lack of attention to detail. By fully proving and editing your dissertation, you indicate respect for academic standards and respect for your audience.

This is, therefore, an important stage that one should not overlook in a dissertation writing process. This helps your work to improve in clarity, professionalism, and quality, thus strengthening the presentation of your research.

Keypoints

Effective writing is arguably just as vital as sound research techniques. In fact, it is most likely best viewed as an element of reliable research methodology.
Writing up research requires a clear format and an explanation of our research topics.
We should examine writing styles to learn that social scientists do more than just report findings.
We should be aware of the ways in which authors try to convince us of their points of view. Writing is intended to persuade and convince.
The purpose of writing is to persuade. We all aim to convince our readers that our ideas are valid and clearly communicated. The question we must ask is: do we achieve this effectively? Are we making the strongest argument possible? It is our shared responsibility to present our analysis convincingly, and the key is to do so with skill. Therefore, always consider the significance of our writing approach when crafting an essay or dissertation.

Checklist

According to Bryman (2016), the following points should be checked before submitting your dissertation or theses:
Ensure the following when preparing your research project:
Clearly specify your research questions to ensure focus and clarity.
Make sure to explicitly show how the literature you have reviewed connects to your research questions.
Ensure your discussion of the literature is critical and well-organized.
Clearly outline your research design and methods, including:
Why you selected a specific research design.
Why you chose a particular research method.
How you selected your research participants.
Any challenges related to participant cooperation (e.g., response rates).
Why you conducted your research in a certain way (e.g., how interview questions align with research questions, why observations were made in specific situations, or the rationale for the order of questions in a focus group guide).
If your research required organizational access, explain how and on what terms agreement was obtained.
The steps you took to ensure ethical responsibility in your research.
How you analyzed your data.
Any difficulties encountered during the implementation of your research approach.
Ensure that your data is presented in a way that directly relates to your research questions.
Clearly demonstrate in your discussion how the findings connect to your research questions and how they illuminate the literature you reviewed.
Support your data interpretations with tables, figures, or transcript excerpts where necessary.
Properly label all tables and figures with titles and numbers, and refer to them in your discussion.
Make sure your conclusions clearly indicate your research contributions to the literature, and explain any limitations of your study.
Avoid simply summarizing your findings in the conclusion; instead, ensure they answer your research questions clearly.
Present your findings and discussion in a way that forms a coherent argument and narrative.
Break up your chapters with appropriate subheadings to improve readability.
Use inclusive language, avoiding sexist, racist, or disablist terminology.
Include all necessary appendices, such as interview schedules, access requests, or communications with participants.
Verify that your reference list includes all sources cited in the text and follows your institution’s required style precisely.
Incorporate your supervisor’s feedback on your draft chapters.
Have others, beyond your supervisor, read your drafts for additional perspectives.
Avoid excessive use of jargon to maintain clarity.
Use clear signposts throughout your writing to guide the reader on what to expect next and why.
Ensure you meet all institutional requirements, such as word count, abstract, and table of contents, if needed.
Avoid over-quoting the literature and properly acknowledge the work of others. It’s crucial to refrain from plagiarism, which is the practice of using someone else’s words or ideas as your own. Make sure to cite sources correctly by using quotation marks or another indicator to indicate that you are using someone else’s words. When you paraphrase someone else’s words or ideas, you have to give the source’s complete bibliographic citation.
Ensure your project title aligns with the content.
Appropriately acknowledge anyone who assisted you, including your supervisor or others involved in your research process.

Self-assessment questions/quizzes

In this part, short self-assessment questions/quizzes will aid in the comprehension of the issues presented in the chapter, challenging the readers to check on their understanding capabilities.

Question 1

Why is it better to start writing your research sooner rather than later?

A) So that you can devote more time to collecting and analyzing data.
B) You must not be surprised by the length of time that revisions and feedback take.
C) So that all study results will be complete before writing is begun.
D) So your supervisor does not have so much work to do.

Question 2

Which of the following do you need to consider when you are communicating the Results section of your thesis?

A) You are required to include all the data you have gathered, even if that data has no bearing.
B) You need to show your awareness through the use of big words.
C) You need to outline the outcomes of research that directly relates to the questions you have asked.
D) For simplicity you must include only numerical data.

Question 3

What is the purpose of building a solid and coherent argument throughout your research in the context of academic writing?

A) In order to offer an array of general, unrelated findings.
B) To ensure that readers trust that your work is important and reliable.
C) This way you can meet your institution’s word count requirements.
D) In order to minimize the need for an extensive literature review.

Question 4

Which of the following most accurately describes the role of research questions in a dissertation?

A) They can be added after the data collection has been done and are optional.
B) To include as much info as possible, they should be wide and open-ended.
C) They don’t have any relation to qualitative research; only quantitative does.
D) They bring coherence and focus by guiding the selection and emphasis of the findings.

Answers: Question 1: B / Question 2: C / Question 3: B / Question 4: D

Indicative References

Babbie, E. R. (2021). The practice of social research (Fifteenth ed.). Cengage.

Bryman, A. (2016). Social research methods (5th ed.). Oxford University Press.

Creswell, J., and Plano Clark, V. L. (2011). Conducting and Designing Mixed Methods Research (2nd ed.). Thousand Oaks, CA: Sage.

Creswell, J., and Tashakkori, A. (2007). Developing Publish- able Mixed Methods Manuscripts, Journal of Mixed Methods Research, 1: 107–11.

Denzin, N. K., & Lincoln, Y. S. (2018). The SAGE Handbook of Qualitative Research (5th edition). Sage Publications.

Ellison, C. (2010). McGraw-Hill’s Concise Guide to Writing Research Papers. McGraw-Hill Education. http://public.eblib.com/choice/PublicFullRecord.aspx?p=6255414

Phillips, R., & Kara, H. (2021). Creative Writing for Social Research : A Practical Guide. Policy Press, an imprint of Bristol University Press.

Silvia, P. J. (2015). Write it up: Practical strategies for writing and publishing journal articles. American Psychological Association. https://doi.org/10.1037/14470-000

Wolcott, H. F. (1990). Writing up Qualitative Research. Newbury Park, CA: Sage.

Media

Gibbs G. R. (2013). Writing Up Social Research Part 2 of 3 on Practical Issues and Ethics. Available at: https://youtu.be/ReyqZE6T8Es?si=LM5sF4xxvZq163gX. Accessed on 21 September 2024.

Research Tube (2023). What is research? For beginners. Available at: https://www.youtube.com/playlist?list=PLrW78oj_xD8pJ7n3eeD31enMRkU2MQxrh. Accessed on 21 September 2024.

Weiss N. (2015). Essay writing in social studies. Available at: https://youtube.com/playlist?list=PL96_6089u0OWHB5KhnHRp7MTU2fNa-dP8&si=U7F6-9aneMRZVtY1. Accessed on 21 September 2024.

Learning Literature Review

Learning Outcomes

By the end of this chapter, students will be able to:
1. Understand the significance of the literature review in research.
2. Identify the stages involved in conducting a literature review.
3. Assess various sources of literature and determine their relevance.
4. Formulate research questions and hypotheses based on existing literature.
5. Critically evaluate and synthesize sources to reveal knowledge gaps and inconsistencies.

Introduction to the Chapter

Literature review is a crucial element in the process of conducting research. According to Denney and Tewksburry (2012), literature reviews are essential for several reasons. First, they require the author to thoroughly research the chosen topic, thereby improving both their understanding and the quality of their writing by highlighting what has already been studied. Secondly, they demonstrate the author’s in-depth knowledge of the subject, which adds credibility and strengthens the argument. In addition, by analysing previous research, literature reviews help to identify gaps or weaknesses, which may justify the need for further investigation and refine the research focus. A literature review involves the systematic identification, evaluation and synthesis of previous work in order to understand what has already been explored in a particular area, the knowledge gaps that exist and the relevant frameworks that can shape a new research agenda. As the number of articles has increased dramatically in recent years, the need for a structured approach to reviewing the existing literature has become an essential process. The production of a comprehensive and insightful literature review is crucial as it not only provides the basis for students’ own research, but also demonstrates their ability to synthesise and critically analyse the work of others (Chigbu et. al. 2023; Kucan 2011). Chigbu et. al (2023), argue that there is no conventional way of conducting a literature review, as the research questions may influence the methodology or approach, and this is where knowledge of some notable approaches to literature review is essential for doctoral students. Knowledge of the different types of literature reviews allows for the selection of the most appropriate approach to achieve specific goals (Snyder, 2019). This chapter aims to provide a guide on how to navigate this process.

Real-World Examples

Consider the following example (Denney and Tewksburry, 2012): In a study examining whether alcohol abuse contributes to the likelihood of committing violent crimes, it would be important to first provide an overview of substance abuse in general (not limited to alcohol) and its potential impact on various types of crime. The literature review should begin by addressing the broader topic of substance abuse and its relationship to crime in general. Then, it would narrow down to the different forms of substance abuse, such as prescription drug misuse and alcohol abuse. Following this, the review should explore the effects of substance abuse on various types of crime, including petty theft, property crimes, and violent crimes. Lastly, the focus would shift to alcohol abuse specifically, examining its psychological and behavioral effects, and how these factors directly relate to violent crime. Overall, the literature review moves from a broad perspective to a more focused examination, guiding the discussion toward the specific research question the author aims to address.

Stages of a Literature Review

Formulating a Research Question

The first step in conducting a literature review is formulating a clear and concise research question. The research question serves as the guiding framework for the review and must be specific, researchable, and original contributing to the existing body of knowledge. This question may be formulated based on prior research, observation, experience or theoretical knowledge. According to Snyder (2023) the contribution of a research depends on the question covered and the researcher must always do a thorough job knowing their research tradition, the discipline, and the state of knowledge on the research question they want to investigate. This should then guide the rest of the design and the conduct of the review, the choice of method for analysis, and the presentation and discussion of the findings. Furthermore, Snyder (2023) considers for the researchers to ask the following questions before designing a literature review:

Why and for whom would this review be useful? Is this review needed?
What other review articles (if any) are published on this topic? How does my review article extend the knowledge generated by these already published reviews?
What constitutes a theoretical/managerial contribution to my field?
What is the nature of my topic? Mature/immature? Narrow/broad?
What will be the actual outcomes of this research?

Searching for Sources

After formulating the research question, the next step is locating relevant literature sources. Berg (2009) listed the potential type of sources and their relative value in the order of:

1.	Scholarly empirical articles, dissertations, and books.
2.	Scholarly, nonempirical articles and essays.
3.	Textbooks, encyclopedias, and dictionaries.
4.	Trade journal articles.
5.	Certain nationally and internationally recognized “good” newsmagazines.

The most reliable sources are those that have undergone rigorous peer review by other scholars before publication. These include academic journal articles, conference proceedings, and academic book chapters (excluding textbooks) with editorial oversight. While textbooks can provide foundational information, they are generally less suitable for citation in literature reviews since they lack a scholarly review process, though they may reference significant or classic studies. Additional sources can include government publications (grey literature), as well as newspaper and magazine articles. These types of alternative sources – newspapers, magazine articles, encyclopedia mentions – should be used sparingly (Denney and Tewksburry, 2012) and when no other information can be found, as over-reliance on these sources may signal to readers that either the research topic lacks sufficient depth or the literature review is poorly constructed. It is important to strike a balance and prioritise academic journal articles whenever possible.

Targeted search of reliable and widely known sources is the key to success. There are several sources of scientific articles. Some of them are the following:

ScienceDirect: There are articles published in more than 2,500 scientific journals. Some of them are paid but most of the articles are offered for free.
Google Scholar
Microsoft Academic Research: Over 200 million publications offered. The modern environment of this particular search engine informs the user about the ‘top authors’ and ‘top journals’ in the field he chooses.
PubMed: It is essential for medical school students.
Directory of open access books
Periodicals Archive online
Doaj
JSTOR
gr

Academic databases such as ScienceDirect, Google Scholar, and JSTOR are valuable resources. During this stage, students must be critical in their choice of sources, ensuring that they are scholarly and peer-reviewed. Accessing a broad range of primary, secondary, and tertiary sources is essential for a comprehensive review (Snyder, 2019). The bibliographic search should be structured to avoid overwhelming amounts of data while ensuring relevant sources are included.

Identifying Keywords

The identification of appropriate keywords is a critical step in effectively searching for sources. Keywords must align with the research question and represent the core themes of the study. Combining keywords using Boolean operators (AND, OR) can help refine searches, making them more targeted and reducing irrelevant results. For instance, in a study on ‘social media influence on political participation,’ keywords might include ‘social media,’ ‘political engagement,’ ‘participation,’ and ‘voter behavior.’

Evaluating the Literature

Once the literature has been collected, the next phase involves critically evaluating each source. The evaluation will help the researcher to identify and refine his research questions and objectives, highlight research gaps that are missed in the research that has been done so far, formulate suggestions for further research and help avoid repetition of work that have already been implemented. A critical literature review enables students to identify gaps in the research, inconsistencies in findings, and areas requiring further exploration. Evaluating literature entails considering factors like the relevance of the source to the research question, the publication date (to ensure the research is current), and the reliability of the methodologies used (Palmatier et al., 2017). During this step, students must also document their findings, taking note of key arguments, methodologies, and conclusions drawn by other researchers.

Writing the Review

After evaluating the collected sources, researcher should organize the information and synthesize it into a coherent narrative. A good literature review not only summarizes the existing research but also offers critical analysis and highlights relationships between different studies. In writing, it’s crucial to group similar studies together, compare methodologies and findings, and identify any contradictions in the literature. Torraco (2005) emphasizes that the literature review is not merely a summary but an opportunity to create new insights by integrating various research outcomes. The references of the review must be written in a specific format based on instructions from the committee or the respective board of the journal. These guidelines may specify the citation style (eg, APA, MLA, Chicago), the formatting of the elements (eg, author names, titles, publishers), and the layout of the citations. Regarding organization, references may appear alphabetically by the last name of the first author or according to the order of appearance in the text of the review. Bibliographic reference management software, such as Zotero, Mendeley or EndNote, can be used for better organization. These programs allow data to be automatically imported from various sources, with the correct formatting style.

Types of Literature Review

Literature review can be published as a section of a scientific paper/article or as a concrete standalone publication. Literature reviews published as standalone and independent studies usually called review papers, where Bem (1995; p. 172) defines that “review papers are critical evaluations of prior studies that have already been published”.

A key aim of a review article is to highlight significant research gaps by examining which constructs, theories, and methods have been commonly used across various contexts. Based on this analysis, review authors offer guidance for future research, suggesting innovative ideas, theories, measures, methods, and research questions. In this way, a review article acts as a foundation for future studies. A well-developed literature review article can serve as a valuable foundation or perspective for future research, as it thoroughly synthesizes existing knowledge, pinpoints research gaps, and proposes promising new directions for further investigation, particularly in terms of methodology, constructs, theory, and contexts (Paul and Criado, 2020). Additionally, theoretical models created through these reviews can be utilized by both researchers and practitioners as frameworks or guides in their studies, whether using quantitative or qualitative methods, or in practical applications.

A detailed summary of literature review types, focus, and their applicable disciplines is provided in Chigbu et. al (2023). Among them the Narrative/traditional literature review, systematic literature review, the scoping review, the bibliometric review and meta-analysis reviews are more common in use by researchers and described below:

Narrative/traditional literature review: The narrative literature review, often referred to as a traditional literature review, focuses on producing a critical, comprehensive analysis of the current state-of-the-art (or science) on a given topic/subject and it is an everyday part of scientific writing because it is essential when establishing a theoretical framework or focusing on contexts (Boyd and Solarino, 2016).

Systematic Literature Review (SLR): Systematic Literature Reviews (SLRs) are reviews carried out in a systematic way using an adopted or adapted procedure or protocol to guide data curation and analysis, thus enabling transparent disclosure and replicability (Lim et al. 2022; Kraus et al. 2021; Siddaway et al 2019). Systematic literature reviews (SLRs) use an inductive reasoning approach, where specific criteria are set and applied to compile a collection of scholarly works for analysis. This process allows researchers to present a comprehensive overview of the current state of a field and suggest future research directions for a domain, theory, or method. These criteria are typically outlined in SLR procedure guides (Kraus et. al 2022; Paul and Criado 2020; Snyder 2019) and protocols, e.g., PRISMA (Page et al, 2021) and can be adapted with appropriate justifications. A summarised list of criteria to be considered in a SLR approach is provided by Kraus et. al 2022 and listed below:

Search database A search database (such as “Scopus” or “Web of Science”) can be selected based on valid reasoning, such as these being the largest scientific databases offering access to scholarly articles and bibliographic data (Pranckutė 2021). To minimize biased results caused by the limitations of a single database, researchers may consider using two or more databases (Dabić et al. 2021).
Search keywords may be developed by reading scholarly documents and subsequently brainstorming with experts.
Boolean operators used in developing the string of search keywords (e.g., “engagement” AND “customer” OR “consumer” OR “business”).
Search period (e.g., from 2000 to 2020 or up to the most recent full year, such as 2021) can be determined based on the specific scope of the study, whether it focuses on contemporary developments or traces historical trends.
Search field (e.g., “article title, abstract, keywords”) can be defined based on assuming that the focus is in the article title, abstract, and/or keywords.
Subject area (e.g., “business, management, and accounting”) can be defined based on justified principles.
Publication stage (e.g., “final”) can be defined based on justified grounds
Document type (e.g., “article” and/or “review”), which reflects the type of scientific/practical contributions.
Source type (e.g., “journal”) can be defined based on justified reasons.

Language(e.g., “English”) can be determined based on justified limitations.
Quality filtering(e.g., Scopus Q1 and Q2) can be defined

Document relevance (i.e., its alignment with the review’s focus) can be determined through informed judgment. For instance, in a review centered on customer engagement, articles that briefly mention the topic without thoroughly exploring it would be excluded.
Others: screening process should start by removing duplicate results from other databases, followed by abstract screening to eliminate irrelevant studies, and conclude with full-text screening of the remaining documents.
Others: Exclusion-inclusion criteria for abstracts and articles is essential when determining if the articles address the research topic. This process may result in the removal of a significant portion of the initially identified articles.

Scoping literature review: Scoping review is similar to systematic literature review. The difference is that there are no restrictions on the materials resourced. Researchers may conduct scoping reviews instead of systematic reviews where the purpose of the review is to identify knowledge gaps, scope a body of literature, clarify concepts or to investigate research conduct. Although conducted for different purposes compared to systematic reviews, scoping reviews still require rigorous and transparent methods in their conduct to ensure that the results are trustworthy (Munn et. al, 2018). Clear guidance regarding the definition of scoping reviews, how to conduct scoping reviews and the steps involved in the scoping review process are described by Colquhoun et. al (2014) and Peters et. al (2015), whereas Munn et. al (2018) provide a practical guidance for researchers on when to perform a systematic review or a scoping review, supported with some key examples.

Bibliometric review: This approach systematically evaluates the literature on a specific subject, topic, or research field by quantitatively assessing indicators like authorship, citations, journals, countries, publication years, and methodologies. The goal is to determine the scope of existing research, identify gaps, and suggest a future research agenda is a scientific approach to literature review. A bibliometric review is a scientific method that quantitatively analyzes bibliographic data to uncover trends in previous studies on a specific subject, topic, or discipline (Merigo et. al, 2017). The Scopus and Web of Science databases are essential tools for conducting bibliometric analyses across various disciplines, helping to identify trends in the development and application of knowledge on specific subjects and research fields (Munn et al 2018).

Meta-analysis literature review: A meta-analysis literature review involves examining the findings from selected scholarly publications and analyzing them using standardized statistical methods (Coughlan et al 2007; Field and Gillet, 2010).

Activities

Activity 1: Conducting a Literature Search

Using an academic database such as Google Scholar or ScienceDirect, search for literature related to your research interests. Identify at least five peer-reviewed articles on your topic, and critically evaluate their relevance to your research question. Present a summary of your findings, focusing on the methodologies used and the gaps you have identified.

Activity 2: Identifying Keywords

Formulate a research question on a topic of your choice and identify at least five keywords that you would use to search for relevant literature. Reflect on the appropriateness of your keywords and whether they accurately capture the scope of your research question.

Concluding Remarks

A literature review is an essential element in doctoral research as it helps to frame the research question and contributes to knowledge development in a specific field. By systematically reviewing and synthesizing existing literature, researchers can identify gaps, contribute to ongoing debates, and establish a foundation for their own research. This chapter provided an in-depth guide to the stages involved in conducting a literature review and offered practical exercises to enhance students’ understanding of the process.

The conclusion at the end of the literature review should act as a clear and concise transition from summarising the key findings and methods of previous research to the significance and/or novelty and/or impact and/or contribution of the author’s research and findings. It should strike a balance between highlighting important findings and showing the need for further research or new methods. Any gaps or weaknesses in previous studies will be explicitly stated, setting the stage for the new research focus that the researcher will present in the rest of the article/thesis or in his research implementation plans.

Self-Assessment Questions

What is the primary purpose of a literature review in doctoral research?
2. How can identifying gaps in the literature contribute to your research?
3. Why is the formulation of a clear research question crucial in conducting a literature review?
4. What criteria should be used to evaluate the quality of a literature source?
5. How do conduct a systematic literature review?

Glossary

Literature Review: A systematic examination and synthesis of previous research related to a specific research question.
Research Question: A clear, focused question that guides the scope of the literature review.
3. Keywords: Terms used to search for relevant literature in academic databases.
4. Systematic Review: A structured approach to identifying and analyzing relevant studies on a specific topic.

References

Bem, D. J. (1995). Writing a review article for psychological bulletin. Psychological Bulletin, 118(2), 172–177.

Berg , B. L. 2009 . Qualitative research methods for the social sciences , 7th ed. , Boston , MA : Allyn & Bacon.

Boyd, B.K.; Solarino, A.M. Ownership of Corporations: A Review, Synthesis, and Research Agenda. J. Manag. 2016, 42, 1282–1314.

Chigbu, U.E.; Atiku, S.O.; Du Plessis, C.C. (2023). The Science of Literature Reviews: Searching, Identifying, Selecting, and Synthesising. Publications, 11, 2. https://doi.org/10.3390/ publications11010002.

Colquhoun HL, Levac D, O’Brien KK, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol. 2014;67(12):1291–4.

Coughlan, M.; Cronin, P.; Ryan, F. (2007) Step-by-Step Guide to Critiquing Research. Part 1: Quantitative Research. Br. J. Nurs. 2007, 16, 658–663.

Dabić M, Vlačić B, Kiessling T, Caputo A, Pellegrini M (2021) Serial entrepreneurs: A review of literature and guidance for future research. Journal of Small Business Management,1–36

Denney, A. S., & Tewksbury, R. (2012). How to Write a Literature Review. Journal of Criminal Justice Education, 24(2), 218–234. https://doi.org/10.1080/10511253.2012.730617.

Field, A.P.; Gillett, R. (2010) How to Do a Meta-analysis. Br. J. Math. Stat. Psychol. 2010, 63, 665–694.

Kucan, L. (2011). Approximating the Practice of Writing the Dissertation Literature Review. Literacy Research and Instruction, 50(3), 229–240. https://doi.org/10.1080/19388071.2010.514037.

Kraus, S., Mahto, R. V., & Walsh, S. T. (2021). The importance of literature reviews in small business and entrepreneurship research. Journal of Small Business Management, 61(3), 1095–1106. https://doi.org/10.1080/00472778.2021.1955128.

Kraus, S., Breier, M., Lim, W.M. (2022) Literature reviews as independent studies: guidelines for academic practice. Rev Manag Sci 16, 2577–2595. https://doi.org/10.1007/s11846-022-00588-8.

Lim WM, Kumar S, Ali F (2022) Advancing knowledge through literature reviews: ‘What’, ‘why’, and ‘how to contribute’. Serv Ind J 42(7–8):481–513.

Merigó, J.M.; Blanco-Mesa, F.; Gil-Lafuente, A.M.; Yager, R.R. (2017). Thirty Years of the International Journal of Intelligent Systems: A Bibliometric Review. Int. J. Intell. Syst., 32, 526–554.

Munn, Z., Peters, M.D.J., Stern, C. et al. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol 18, 143 (2018). https://doi.org/10.1186/s12874-018-0611-x.

Page, M.J.; Moher, D.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E. (2021). PRISMA 2020 Explanation and Elaboration: Updated Guidance and Exemplars for Reporting Systematic Reviews. BMJ, 372, n160.

Palmatier, R.W., Houston, M.B. & Hulland, J. (2018) Review articles: purpose, process, and structure. J. of the Acad. Mark. Sci. 46, 1–5. https://doi.org/10.1007/s11747-017-0563-4.

Paul, J., Criado A.R. (2020). The art of writing literature review: What do we know and what do we need to know?, International Business Review, Volume 29, Issue 4, https://doi.org/10.1016/j.ibusrev.2020.101717.

Pranckutė R (2021) Web of Science (WoS) and Scopus: The titans of bibliographic information in today’s academic world. Publications 9(1):12.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015;13(3):141–6.

Siddaway, A.P.; Wood, A.M.; Hedges, L.V. (2019) How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses. Annu. Rev. Psychol. 2019, 70, 747–770.

Snyder, H (2019). Literature Review as a Research Methodology: An Overview and Guidelines. Journal of Business Research, 104, 333–339.

Snyder, H. (2023). Designing the literature review for a strong contribution. Journal of Decision Systems, 1–8. https://doi.org/10.1080/12460125.2023.2197704.

Torraco, R. J. (2005). Writing Integrative Literature Reviews: Guidelines and Examples. Human Resource Development Review, 4(3), 356-367. https://doi.org/10.1177/1534484305278283.

Research ethics

Learning outcomes

Upon completion of the study of this chapter students will be able to understand the key debates regarding ethical issues in social research, as well as the most important ethical issues arising when planning and conducting a social research project. In addition, students will comprehend political issues that can influence in a significant way the planning and conduction of social research.

A brief overview of the discussion on ethics

Research that is likely to harm participants is regarded by most people as unacceptable. Harm can entail a number of different aspects: physical harm; harm to participants’ development; loss of self-esteem; stress; and ‘inducing subjects to perform reprehensible acts’, (Bryman, 2012:135; Armenakis, 2021). Therefore, among the most heated and diachronic debates in social research is the one concerning research ethics. Social research takes place in a social context. Researchers must bear in mind many ethical and political considerations alongside scientific ones in designing and executing their research. Often, however, clear-cut answers to thorny ethical and political issues are hard to come by (Babbie, 2021:60). Ethical principles in research can be -rather loosely- defined as the set of values, standards, and principles used to determine appropriate and acceptable conduct at all stages of the research process (Adler & Clark, 2011:40), as ethical concerns might rise at all different stages of a social research project (Bryman, 2012; Adler & Clark, 2011).

First and foremost among ethical standards is the obligation to protect study participants from harm, which means that the physical, emotional, social, financial, legal, and psychological well-being of those who participate in a research project must be protected, both during the time that the data are collected and after the conclusion of the study. Overall, there should not be negative consequences as a result of participating in a study (Adler & Clark, 2011:51). Even though these principles seem “obvious” in the first place, the sheer volume of contrasting opinions that frame this discussion, reveals the complexity of the issue. E.g., writers often differ quite widely from each other over ethical issues and questions. In other words, they differ over what is and is not ethically acceptable (Bryman, 2012).

This debate is -among others- revolving around the question if there is a set of universally applicable ethical principles or if it’s better to take a case-by-case approach and consider each project’s specific circumstances when deciding what is, and what is not, ethical. To that end, the researcher should bear in mind that unforeseen circumstances and unintended consequences can take place at every stage of research -from planning to publication- and can create ethical problems (Adler & Clark, 2011). In addition, given that social research is not conducted outside society, but within it, social norms and ethical principles that differ between the various societies influence the planning and conduction of any research projects. Thus, researchers should be well aware of the various restrictions, since ignoring these parameters could seriously undermine or even cancel a research project.

Historically, the discussion around ethics did not become a concern of social science organizations as a whole until the middle of the twentieth century (Adler & Clark, 2011:40). In addition, the main arguments in the debates do not seem to move forward a great deal. One thing that has changed is that ethical issues are nowadays more central to discussions about research than ever before. This may be due to a greater sensitivity to ethical issues, but it is also to do with a greater concern among representatives of universities, research funding bodies, and professional associations to exhibit good ethical credentials, to the point that this increased focus on ethical issues has obtained on several occasions the characteristics of a ‘moral panic’ (Van den Hoonaard 2001).

The debates on ethical issues on social research are usually related to specific research methods (e.g., disguised/covert observation and the use of deception in experiments) (Armenakis, 2021). The problem with the association of ethics with certain methods is that it implies that ethical concerns only or even primarily reside in some methods but not others, but this is not the case. For example, conducting questionnaire or overt observation research with children will raise a lot of ethical issues that may not be the case when the research is on adults (Bryman, 2012:131), as in the case of a study of the sexuality of children (Babbie, 2021:71).

There are different approaches in terms of how ethical principles in social research should be established and implemented in order to avoid any kind of ethical transgression. To begin with, there is the “deterministic” strand of “universalism”. As its name implies, universalists stand for specific ethical rules that should never be broken. On a significantly different path one can find the relativists of “situation ethics”, who argue that moral transgressions should be considered on a case-by-case basis, especially under the rationale that the end justifies the means. Some writers argue that, unless there is some breaking of ethical rules, we would never know about certain social phenomena. Fielding (1982) argues for this position in relation to his research on the National Front, an extreme right-wing organization that was becoming politically influential in the 1970s. Without some kind of covert observation, this important movement and its appeal would not have been studied (Bryman, 2012).

Several authors argue that ethical transgression is pervasive to almost all kinds of research. This occurs whenever participants are not given absolutely all the details on a piece of research, or when there is variation in the amount of knowledge about research. This selective information giveaway aims predominantly at provoking people’s honest opinions/attitudes during research.

Quite similar to the universalists vs. situationalists debate, is the one between deontological and consequentialist ethics. Deontological ethics considers certain acts as wrong (or good) in and of themselves (e.g., covert observation independent from the research aim). Consequentialist ethics looks at the consequences of an act for guidance as to whether it is right or wrong (e.g. see covert research conducted in extremist communities/parties) (Bryman, 2012:134).

Ethical aspects

Anonymity/Confidentiality

Anonymity is when it is impossible for anyone, including the researcher, to connect specific data to any particular member of the sample (this is the case in quantitative surveys) (Babbie, 2021:67). To collect data anonymously, data must be collected without names, personal identification numbers, or information that could identify subjects. If it is not possible or practical to collect data anonymously (as is the case usually in qualitative research methods), research ethics dictate keeping them confidential. Confidentiality, or privacy, is not revealing publicly the identity of any given person that has participated in the research (Babbie, 2021:67). Thus, keeping the information of study participants, including their identities, disclosed from any other parties, (e.g., parents, teachers, school administrators, and any other individual or institution) (Adler & Clark, 2011:57). Identifying respondents by code numbers or pseudonyms are useful ways to keep identities and data separate. When results are made public, confidentiality can be achieved by not using real names or by grouping all the data together and reporting summary statistics for the whole sample. In addition, researchers can change identifiers like specific occupations or industries and names of cities, counties, states, and organizations when presenting study results (Adler & Clark, 2011:58). The issue of confidentiality raises particular difficulties for many forms of qualitative research. The use of pseudonyms is a common recourse but may not eliminate entirely the possibility of identification (Bryman, 2012:136), especially at the stage of the elaboration of the gathered qualitative data.

The need for confidentiality can present dilemmas for researchers. Sometimes during the course of research, observing others or interviewing study participants, the researcher finds out about illegal behavior (Adler & Clark, 2011:62). Westmarland (2001) has discussed the dilemmas she faced when observing violence by the police towards people being held in custody. She argues that, while a certain level of violence might be deemed acceptable, in part to protect the officers themselves and the public, there is an issue of at what point it is no longer acceptable and the researcher needs to inform on those involved. The problem for the ethnographer stems from the fact that revealing excessive violence may result in a loss of the researcher’s credibility among officers, premature termination of the investigation, or inability to gain access in the future (Bryman, 2012:136).

Another aspect of confidentiality and the management of it are the obligations on people and organizations who hold personal data on others. E.g., in the UK, the Data Protection Act (1998) points out that anyone who processes personal information must comply with specific principles, which make sure that personal information is fairly and lawfully processed, processed for limited purposes, adequate, relevant, and not excessive, accurate and up to date, not kept for longer than is necessary, processed in line with the participants’ rights, secure and not transferred to other countries without adequate protection. The second area covered by the Act provides individuals with important rights, including the right to find out what personal information is held on computer and most paper records (Bryman, 2012:137). These aspects on confidentiality remind us of the scandal of Cambridge Analytica, that collected -in early 2010s- personal data belonging to millions of Facebook users without their consent, predominantly to be used for political marketing/advertising.

Informed consent

The issue of informed consent is in many respects the area within social research ethics that is most hotly debated. The discussion tends to focus on what is called disguised or covert observation. Such observation can involve covert participant observation, or simple or contrived observation (observation conducted in controlled settings, e.g., laboratory), in which the researcher’s true identity is unknown. The principle means that (prospective) research participants should be given as much information as might be needed to make an informed decision about whether or not they wish to participate in a study. In other words, informed consent is a principle according to which subjects base their voluntary participation in research projects on a full understanding of the possible risks involved (Babbie, 2021:65). Covert observation transgresses that principle because participants are not given the opportunity to refuse to cooperate (Bryman, 2012:138).

It is extremely difficult to present (prospective) participants with absolutely all the information that might be required for them to make an informed decision about their involvement in a research project. In fact, relatively minor transgressions probably pervade social research on several occasions, such as deliberately underestimating the amount of time that an interview is likely to take so that people are not put off being interviewed, and not giving absolutely all the details about one’s research for fear of contaminating people’s answers to questions. Second, in ethnographic research, the researcher is likely to come into contact with a wide spectrum of people and ensuring that absolutely everyone has the opportunity for informed consent is not practicable, because it would be extremely disruptive in everyday contexts. Ideally, where informed consent has not been obtained prior to the research it should be obtained post-hoc (Bryman, 2012:139), this can be achieved through the debriefing of the participants, i.e. interviewing subjects to learn about their experience of participation in the project and to inform them of any unrevealed purpose. This is especially important if there’s a possibility that they have been damaged by that participation (Babbie, 2021:70).

Increasingly, researchers prefer to obtain the consent of research participants by getting them to sign informed consent forms. The advantage of such forms is that they give respondents the opportunity to be informed of the nature of the research and the implications of their participation. Further, the researcher has a signed record of consent if any concerns are raised by participants or ethics committees. The chief possible problem is that the requirement to sign the form may prompt rather than alleviate concerns on the part of prospective participants, so that they end up declining to be involved. Also, the direction of qualitative studies can be somewhat less predictable than with quantitative ones, so it is difficult to be specific within forms about some issues (Bryman, 2012:139) (e.g., in a semi-structured interview one cannot inform the interviewee in advance about the exact thematics that will be discussed, since the evolution of such an interview is rather dynamic and depends heavily on the answers provided by the interviewee).

The right to privacy

Social research, moreover, often requires that people reveal personal information about themselves—information that may be unknown to their friends and associates (Babbie, 2021:63). The right to privacy is very much linked to the notion of informed consent, because, to the degree that informed consent is given on the basis of a detailed understanding of what the research participant’s involvement is likely to entail, (s)he in a sense acknowledges that the right to privacy has been surrendered for that limited domain (Bryman, 2012:142). For example, when people agree to be interviewed, they may refuse to answer certain questions on whatever grounds they feel are justified. Examples might be questions about income, religious beliefs, or sexual activities. Covert methods are usually deemed to be violations of the privacy principle on the grounds that participants are not being given the opportunity to refuse invasions of their privacy (Bryman, 2012:142-143).

Deception

Deception occurs when researchers represent their work as something other than what it is. The experiment by Milgram involved deception, since participants are led to believe they are administering real electric shocks. Deception in various degrees is probably quite widespread in such research, because researchers often want to limit participants’ understanding of what the research is about so that they respond more naturally to the experimental treatment. One of the chief problems with the discussion of this aspect of ethics is that deception is, as some writers observe, widespread in social research, since it is rarely feasible or desirable to provide participants with a totally complete account of what your research is about (Bryman, 2012:143). Especially, qualitative research is frequently very open-ended, and, as a result, research questions are either loose or not specified, so that it is doubtful whether ethnographers in particular are able to inform others accurately about the nature of their research (Bryman, 2012:148).

In observing groups or organizations who meet in private settings, it is ethical to ask for permission from those you’ll be observing. If, however, those being observed are likely to change their behavior if they know about the observation, then you will need to make a choice. In a study of a Canadian neo-National Socialist organization that had a history of racism and using violence, Lauder (2003) first tried collecting information by interviewing people using informed consent. But many group members refused to sign informed consent forms and his formal interviews produced little useful information about the anti-Semitic and religious character of the organization’s ideology. Rather than abandoning the project, Lauder decided to change tactics. Using deception, he moved from an overt to a covert role by pretending to have converted to the group’s worldview and then by talking to members informally. He collected a great deal of information covertly and protected confidentiality when using the data. Lauder feels that he balanced the needs of society against the needs of individuals and concluded that he was morally obligated to conduct the study (Adler & Clark, 2011:65).

Ethical issues in the digital space

The Internet has also thrown up new dimensions of ethical decision-making for social researchers. For example, it is very tempting to use newsgroups, chatrooms, listservs, email discussion groups, and so on for examining interaction or a focus of interest. When participants have not given their consent to having their postings used in this way, it could be argued that the principle of informed consent has been violated. However, it could also be claimed that, in some cases, such postings are in the public domain (e.g. on twitter), so that seeking consent is unnecessary.

Whether electronic communications are public or private is a matter of debate (Adler & Clark, 2011). Pace and Livingston (2005: 39) argue that electronic communications should be used for research only if the information is publicly archived and readily available, no password is required to access the information, the material is not sensitive in nature, and no stated site policy prohibits the use of the material. If these conditions do not pertain, informed consent needs to be obtained and should be obtained without disrupting ongoing online activity. These guidelines, though, are not without problems. What is or is not sensitive is likely to be highly debatable, so treating it as a principle is not that straightforward (Bryman, 2012:149).

Conscious partiality

For some writers on social research, a “conscious partiality” is celebrated. Particularly among feminist researchers, to do research on women in a value-neutral way would be undesirable (apart from being difficult to achieve), because it would be incompatible with the values of feminism. Instead, many feminist researchers advocate a stance that extols the virtues of a commitment to women and exposing the conditions of their disadvantage in a male-dominated society. Much of such research has been concerned to change the situation of women, as well as to heighten our understanding of the disadvantages from which they suffer. Social researchers are sometimes put in the position where they take sides. This is precisely what many feminist researchers do when they focus on women’s disadvantages in the family, the workplace, and elsewhere, and on the possibilities for improving their position (Bryman, 2012:150).

Ethical (and political) issues related to funding (bodies)

Much social research is funded by organizations such as firms and government departments. Such organizations frequently have a vested interest in the outcomes of the research. The very fact that some research is funded, while other is not, suggests that political issues may be involved, in that we might anticipate that such organizations will seek to invest in studies that will be useful to them and that will be supportive of their operations and world views. When social researchers participate in such projects, they are participating in a political arena because they are having to tailor their research concerns and even research questions to a body that defines or at least influences those research concerns and research questions. Bodies like government departments are going to be influenced by notions of relevance to their work and by their understanding of ministers’ concerns. In addition, many agencies restrict what researchers are able to write about their findings by seeing drafts of all proposed publications (Bryman, 2012:150).

Gaining access is also a political process. Access is usually mediated by gatekeepers, who are concerned about the researcher’s motives: what the organization can gain from the investigation, what it will lose by participating in the research in terms of staff time and other costs, and potential risks to its image. Often, gatekeepers will seek to influence how the investigation takes place, what kinds of questions can be asked, who can and who cannot be the focus of study, the amount of time to be spent with each research participant, the interpretation of findings, and the form of any report to the organization itself. The police, for example, are usually concerned about how they are going to be represented in publications in case they are portrayed unfavorably. Firms are also invariably concerned about issues of how they are going to be represented. Consequently, gaining access is almost always a matter of negotiation, and as such inevitably turns into a political process. The results of this negotiation are often referred to as ‘the research bargain’ (Bryman, 2012:151).

Once in the organization, researchers often find that getting on in organizations is a process of constant negotiation of what is and is not permissible. In other words, there may be several layers of gatekeepers in any research project, so that issues of access become an ongoing feature of research. For example, for their research on cargo vessels, Sampson and Thomas (2003) sought initial access through ship-owning/managing companies only to find out that the key gatekeepers were the captains, who varied in the degree of willingness to accommodate the researchers’ investigative and other needs (Bryman, 2012:151).

Honest reporting, avoidance of plagiarism

In addition to their ethical obligations to subjects, researchers have ethical obligations to their colleagues in the scientific community. These obligations concern the analysis of data and the way the results are reported (Babbie, 2021:71). A central ethical principle is honest reporting, which is the responsibility to produce accurate data, report honestly, acknowledge the limitations of the research, and disseminate it in both professional and community forums (Adler & Clark, 2011:60; Babbie, 2021). Negative findings, for example, should be reported if they are related to the analysis. There is an unfortunate myth in scientific reporting that only discoveries of strong, causal relationships among variables are worth reporting. In science, however, it’s often as important to know that two variables are not related as to know that they are (Babbie, 2021:71). In addition, researchers should bear in mind that the avoidance of plagiarism -presenting other people’s work as our own- goes part and parcel with honest reporting, and the originality of any research project should be justified beyond any doubt.

The role of research committees

Most higher education organizations have established ethics committees that issue guidelines about ethical research practice. These guidelines are often based on or influenced by the codes developed by professional associations. Sometimes a researcher needs to submit her/his proposed research to an ethics committee of the university or college (s)he belongs to. The basic aims of these committees are, on one hand, to protect the researchers and the institutions, in terms of preserving their positive reputation and publicity, and, on the other hand, to protect the participants in the research project against any physical or psychological harm that can be caused due to their participation in the research.

One of the main approaches used by ethics committees is to ask researchers to indicate whether their research entails certain procedures or activities, such as disguised observation, so that effectively they self-declare whether they are likely to engage in ethically dubious practices. This process usually entails completing a form to show that you have considered potential ethical issues that might arise from your study. This form is likely to ask questions such as ‘Will there be any potential harm, discomfort, physical or psychological risks for research participants?’ and the researcher needs to answer ‘Yes’ or ‘No’. If there is a possibility that you may engage in such a practice, the proposed research is then ‘flagged’ for full scrutiny by the ethics committee. In such an instance, the researcher is required to provide a full account of the research and the rationale for using the ethically dubious practice(s). This can slow down research a great deal and can of course result in the committee refusing to allow it to proceed.

In recent years, research ethics committees have become quite controversial, since they are sometimes seen as having gone too far in terms of their role of protecting institutions (Van Den Hoonard 2001). In addition, there is a concern that, once the researcher has jumped over the bureaucratic hurdle of the ethics committee, (s)he may feel that the ethical issues have been covered. This is clearly not the case, as ethical issues can arise at all stages of the research process (Bryman, 2012:135), even at the production and presentation of the results, where the obligation of the researcher is to produce accurate results stemming from the scientific implementation of the research project.

Conclusion

Although social scientific research does not usually place subjects in situations that jeopardize their health and well-being, most social research involves some risk. Discomfort, anxiety, reduced self-esteem, and revelation of intimate secrets are all possible costs to subjects who become involved in a research project. The ethics of human subject research require that investigators calculate the risk–benefit equation and seek to balance the risks of a subject’s involvement in the research against the possible benefits of a study for the individual and for the larger society (Adler & Clark, 2011:67). Each researcher must think about the consequences of doing a given study as opposed to not doing the study and must consider all options and methods to find a research strategy that balances being ethical and being practical with the likelihood of obtaining good quality data. In doing research, each of us must recognize that we are balancing the rights of study participants against our desire for research conclusions in which we have confidence and that we can share with the public (Adler & Clark, 2011:68).

Summing Up Ethical Principles

Protect research participants from harm.
Get informed consent.
Be sure that study participants have not been pressured into volunteering.
Collect data anonymously or keep data confidential.
Submit the research proposal to a review board.
Provide accurate research findings.
Consider responsibilities to research participants, colleagues, and the general
public at all stages of research including after project’s completion and publication.
Maximize benefits and minimize risks (Adler & Clark, 2011:68).

Exercise

You have to create an informed consent document about a research project that includes semi-structured interviews with the Department of Communication and Digital Media graduates regarding their overall experience of their studies at the Department. The thematics to be discussed will include potentially unpleasant experiences as well. How would you inform them properly through the document?

Questions

Which is the main difference between “universalists” and “situationalists” in terms of the implementation of ethical principals in social research?
How can funding bodies interfere with/undermine the integrity and quality of social research?
Which online documents/posts can a researcher include in her/his research according to research ethics, without the need to have the informed consent of their creators/uploaders?
In which occasions could a research project benefit from conscious partiality?
How are anonymity and confidentiality of participants in a research project protected?

References

Adler, S. E., and Clark R. (2011). An Invitation to Social Research: How It’s Done (4^th Ed.). Wadsworth Cengage Learning.

Armenakis, A. (2021). Research Methods. Learning Material for Postgraduate students of the Department of Communication and Media Studies of the National and Kapodistrian University of Athens.

Babbie, E. R. (2021). The Practice of Social Research (15^th Ed.). Cengage.

Bryman, A. (2012). Social Research Methods (4^th Ed.). Oxford University Press.

Fielding, N. (1982). Observational Research on the National Front. In M. Bulmer (ed.) Social Research Ethics. Macmillan.

Lauder, M. A. (2003). Covert participation observation of a deviant community: Justifying the use of deception. Journal of Contemporary Religion, 18(2), pp. 185-196.

Pace, L. A., and Livingston, M. M. (2005). Protecting Human Subjects in Internet Research. International Journal of Business Ethics and Organization Studies, 10, pp. 35- 41.

Sampson, H., and Thomas, M. (2003). Lone Researchers at Sea: Gender, Risk and Responsibility. Qualitative Research, 3, pp. 165-189.

Tsigganou, I., Iliou, K., Fellas, K. N, Balourdos, D. and Giakoumatos, S. (2018). Research Methods. Learning Material for the National Center for Public Administration and Local Government.

Van den Hoonaard, W. (2001). Is Research Ethics Review a Moral Panic? Canadian Review of Sociology and Anthropology, 38, pp. 19-36.

Westmarland, L. (2001). Blowing the Whistle on Police Violence: Ethics, Research and Ethnography. British Journal of Criminology, 41, pp. 523-535.

Further Study

Bailey, K.D. (1978). Methods of social research. Free Press.

Crotty, Μ. (1998) The foundations of social research. Sage Publications.

Flick, U. (2009). An Introduction to Qualitative Research. 4th Ed. Sage.

Gray, D. E. (2004). Doing Research in the Real World. Sage Publications.

Kumar, R. (1999). Research methodology. A step-by-step guide for beginners. Sage Publications.

Mertens, D. (1998). Research methods in education and psychology: integrating diversity with quantitative & qualitative approaches. Sage Publications.

Pickering, M. (2008). Research Methods for Cultural Studies. Edinburgh University Press.

Quantitative Research and Analysis

Learning outcomes

Upon completion of the study of this chapter students will be able to understand key dimensions of quantitative social research and analysis (the cases in which we implement quantitative research, the different stages/phases of its implementation, and the ways in which the researcher can elaborate her/his data in order to produce results).

Introduction to Quantitative Research

Quantitative studies aim mainly at measuring (or manipulating through experimental designs) a set of predefined variables in order to answer the research questions and to test the research hypotheses that have been derived from the relevant literature (Creswell and Creswell, 2017). According to Creswell (1994) quantitative research tries to explain social phenomena by collecting data in numerical form that are analyzed via statistical methods. As Bryman and Cramer (2012, p. 35) note quantitative research is, “a research strategy that emphasises quantification in the collection and analysis of data…”. Thus, the unique feature of quantitative research is the collection of numerical data and the measurement of a phenomenon using variables as well as the use of statistics. As Babbie (2010) notes, it is important for quantitative research to develop instruments that include questions which can be answered numerically.

Regarding the use of numerical data, there are certain types of data or questions that can be answered in a numerical form. For example, how many hours do you use social media during the day? However, not all questions/a are represented in a quantifiable way. Thus, researchers need to convert phenomena in quantitative data so they can be analyzed statistically. One example is research on attitudes. Attitudes are not numerical data but can be treated as such by developing a research instrument (e.g., questionnaire) that will ask respondents to rate their attitudes on a scale ranging from 1: strongly disagree to 5: strongly disagree. Thus, attitudes have been measured and represent quantitative data that can be analyzed statistically.

According to Rahman (2017) the main advantages of quantitative research are the ability of researcher to generalize results to a whole population or sub-population is the sample is quite large and is selected in random way. Moreover, quantitative research utilizes a deductive process that focuses on objectivity. It adopts structured and formal procedures for the development of measurement instruments and data collection. In addition, the researcher is independent from the phenomenon that is examined, and quite often his/her point of view is external. In addition, most quantitative methods follow well defined and structured theoretical frameworks, models, and try to test sets of pre-defined hypotheses (e.g., Queiros, Faria, and Almeida, 2017).

However, the main weaknesses of quantitative research lie in the inability to get a deeper understanding of the phenomena examined as well as the temporal nature of the data collected. More specifically, quantitative research such as an attitude survey is regarded as a snapshot of the current situation as it measures people’s attitudes at a specific time.

Descriptive and Causal Studies

Quantitative research could take two main forms: (a) descriptive or (b) causal research. According to Zikmund (2003) descriptive research tries to portray the characteristics of a population, their attitudes, opinions, behaviors (e.g., demographic characteristics of voters, product preferences of consumers) based on a population’s sample. Descriptive research answers mainly the who, what, when, where, and how questions. For example, a researcher might be interested in learning about the characteristics of a group of consumers in terms of their demographic characteristics, attitudes towards the product, buying frequency, personality traits, etc. Another objective of a descriptive study is to shed light on the association that might exist between two variables and helps the researcher decide which variables might be included in a causal study.

In causal studies, the researcher tests whether two variables are related. In other words, it tries to identify cause and effect relationships among variables and explores whether a change in one variable can cause a change in or predict another variable. For example, a researcher might be interested in testing whether an increase in advertising expenditures can cause or predict an increase in product sales. In causal studies, the researcher tests a set of hypotheses through the use of statistical tests.

Research Design and Methods

In general, researchers should take into account the following steps for a successful research design:

Identification of the problem, formulation of research questions/objectives
Selecting of the type of research (e.g., descriptive or causal)
Development of the conceptual framework/model (optional)
Detecting of the appropriate information that is needed for addressing the research questions.
Identifying of the appropriate measurement methods regarding the information needed.
Deciding on the data collection method.
Choosing the population, the sample size, and the sampling technique to be used.
Determining the methods for analyzing the data.

Moreover, successful research designs have clearly defined research problems and objectives while they take into consideration the time as well as the financial resources required.

There are three main types of quantitative research design: (a) survey design, (b) observations, and (c) experiments.

A survey is related to a description of trends, attitudes, and opinions of respondents. It is defined as “a systematic method for gathering information from a sample of entities for the purpose of constructing quantitative descriptors of the attributes of a larger population of which entities are members” (Groves et al., 2011, p 2). The use of the term systematic differentiates survey from other methods of collecting information. Through surveys the researcher gathers information from a sample of respondents via a structured questionnaire that includes a set of pre-defined questions. Most of the time, surveys involve the collection of information be asking questions. This is done mainly either by interviewers who ask questions and collect answers or through self-administered questionnaires that are answered by respondents themselves. Important issues that need to be addressed when designing a survey are the sampling techniques, the design of the questionnaire, the data collection method, and the data analysis techniques.

Observational research examines respondents’ behaviors and actions without interaction on behalf of the researcher. Observational research is generally systematic and controlled in terms of the process followed. In other words, the research decides on what, when, where, and how to observe as well as the environment that the observation will take place. Moreover, the way that observational data are going to be measured and analyzed is another controlled element by the researcher. An example of quantitative observational research is related to consumer behavior where researchers could examine the shopping habits and patterns of shoppers in stores.

An experiment is another way to collect quantitative data by manipulating variables and controlling others. More specifically, in an experiment the researcher changes one or more independent variable and tries to decipher the effects on the dependent variable(s). Experiments are often used in consumer behavior, advertising, crisis communication, public relations, etc. In the communication field, researchers develop certain experimental conditions through scenarios that participants receive while controlling for all other factors that might affect the outcome. Then, participants answer a set of questions that represent the dependent variables to be examined.

Survey Errors

Researchers conducting surveys often face certain types of errors that in most occasions are associated either with chance variation (random error) or with systematic error. Chance variation is the difference between the values of the sample and the values of the population. However, if the sample size increases then the chance variation tends to decrease. Systematic error, on the other hand, usually is related to issues associated with research design and results from (a) sample design errors, and (b) measurement errors.

Sample design errors occur due to problems in the design of the sample such as errors in the sampling frame (list of population) that is used to extract the sample, the poor definition of the population, and issues about the selection of the sample from the population. Measurement errors result from differences that arise between the information that needed to be extracted and the information that was obtained by the respondents. Measurement errors are related to lack of a clear definition of the research problem, mistakes of the interviewer, problems associated with the research instruments (e.g., questionnaire), errors due to the analyzing methods of the data, and nonresponse as well as response bias. Nonresponse bias occurs when elements of the sample are not included in the final survey. Thus, nonresponse bias is the difference that might exist between individuals that responded to the survey and individuals that did not respond to the call to participate. In addition, response bias deals with errors on how people answer questions. This type of bias stems from “deliberate falsification” when people intentionally don’t give their true answers out of embarrassment or need to conceal information. Response bias can also arise from unintentional misrepresentation on behalf of the respondent who gives an inaccurate response due to misinterpretation of the question.

Questionnaire Design

A questionnaire is a set of questions that is administered to respondents to collect their answers in a standardized way so as the data to be consistent and well organized for statistical analysis. A questionnaire is an inexpensive way to collect primary data in a short amount of time. Questionnaires also enable participants to respond in an honest way as it can ensure anonymity and protect the privacy of respondents. As Roopa and Rani (2012, p. 273) “a questionnaire is the backbone of any survey and the success of it lies in the designing of a questionnaire”.

To design a questionnaire the researchers should make decisions about the following:

Information required
Content of questions
Format of questions
Wording of questions
Questionnaire pre-testing and revision

Information Required and Content of the Question

The first step for a well-designed questionnaire is to understand the research problem as well as the research objectives. Moreover, the researcher needs to examine secondary data, previous similar research studies, the results from prior qualitative research so as to formulate the questions for addressing the research objective. At this stage, the researcher needs to have a deep knowledge of the target respondents, their characteristics, ability as well as willingness to answer the questions. In addition, the mode of data collection is also an important factor that should be taken into consideration when designing the questionnaire. There is a difference between the type of questions that are required for personal interviews, telephonic, self-administered, and online surveys. The researcher should first create a list of the concepts and variables that will be examined. After defining the variables, then the specific questions and items that will measure the variables should be developed. This could be done by looking at how other researchers have measured the variables. For example, if the questionnaire is about consumer behavior and measures offline word-of-mouth communication of consumers, a literature review will help the researcher understand how others have treated the concept of word-of-mouth communication and identify relevant items and questions to include in the questionnaire. The researcher could also use a scale (set of items/questions that measure a variable) that has been validated in a previous study such as the Carroll and Ahuvia’s (2006) scale in word-of-mouth communication that includes four items such as: I have recommended this product to lots of people; I talk up this product to my friends; I try to spread good-word about this product; I give this product tons of positive word-of-mouth advertising.

Format of the questionnaire

The next step in the design of the questionnaire is choosing the format of the questions. There are two broad types of question format: (a) open-ended questions, and (b) close-ended questions. Open-ended questions enable participants to answer in their own words without fixed responses and give the respondent the freedom to express his opinion. For example, an open-ended question for a researcher examining leadership in public relations could be: What leadership style is perceived as the best or most appropriate for public relations? (Aldoory and Toth, 2004). Open-ended questions help the researcher to trigger the memory of the respondents when they need to recall previous experiences. Moreover, open-ended questions are suitable when the researcher cannot be sure about all the answers required. However, this type of questions is difficult in coding and could lead to low response rate as they require greater effort on behalf of the respondent.

Close-Ended Questions

On the contrary close-ended questions are those that have a pre-defined set of responses from which the respondent can choose. They are mainly used when there is a specific list of answer options, and the possible alternative answers are known by the researcher. Close-ended questions may lead to higher response rates and give a certain level of control over the data which are consistent and can be more easily coded and analyzed. Nonetheless, this type of questions may not include all the possible alternatives, thus, the respondent’s opinion might not be included in the answers. When designing close-ended questions two important issues need to be addressed that are related to the number of options that will be given to respondents as well as the order-position of the options. Close-ended questions could take the following forms:

Dichotomous questions
Multiple choice questions
Checklist
Rank-order questions
Rating scale questions
Constant sum questions
Paired comparisons

Dichotomous Questions

In this type of question, the respondent can choose only from two possible answers. Such questions are suitable for filtering the respondents and testing their appropriateness to be included in the sample. For example, a survey about smoking could use a dichotomous question at the beginning of the questionnaire such as: Do you smoke: Yes/No. This question could help identify respondents that can proceed to complete the questionnaire or need to be excluded from the survey. It should be noted that dichotomous questions should be used with care when measuring attitudes and opinions in the form of “Agree/Disagree” as they do not cover the full range of responses as attitudes might vary in strength. The position of the answers does not indicate any form of order.

Multiple Choice Questions

These questions are appropriate when the research knows most of the possible answers and respondents are asked to indicate or select the only one option that best describes their opinion. The available options should be exhaustive so that the respondent gives one answer. Moreover, the options/answers represent mainly categorical values. Below are several examples of this type of questions:

Which of the following channel is considered the most effective in reaching out stakeholders in your organization? (Please choose only one option) (European Communication Monitor, 2023).

Social media and social networks
Online communication via websites, emails, intranets
Face to face communication
Press and media relations with online newspapers/magazines
Mobile communication
Press and media relations with TV/Radio stations
Press and media relations with print newspapers/magazines

In your organization public relations are practiced by:

An internal department of public relations/corporate communications
A group of employees under the supervision of the marketing department
An external public relations agency

Checklist

This type of question enables respondents to choose more than one answer. The alternative options should be placed in a random order while the researcher could include the “other” option at the end so as the respondent to report an answer that has not been included:

Which of the following brands did you buy during the last month? (You can choose more than one option)

Brand A
Brand B
Brand C
Brand D
Other………. (please specify)

Rank-order questions

In rank-order questions respondents are asked to compare the different options and rank each of them based on their preferences:

Please rank from 1: not important to 5: very important the following factors based on the importance you assign when purchasing in a store:

Product variety ___

Prices ___

Store layout ___

Atmospherics ___

Location ___

Rating scale questions

A rating scale asks participants to select one response from options that are positioned in a hierarchical order. Each response could be labeled or only the two endpoints are labeled (Friedman and Amoo, 1999). Rating scales are often used to measure attitudes and they can take many forms.

Likert scale

The most common rating scale used is the “Likert” scale. This question often takes the form of a statement and respondents rate the extent to which they agree or disagree with the statement.

Please rate the extent to which you agree or disagree with the statement:

The store had a wide variety of products.

1.Strongly Disagree

2.Disagree

3.Neither agree/Neither disagree

4.Agree

5.Strongly Agree

An important debate exists between researchers regarding the optimum number of points that should be included in the rating scales and whether an even or odd number of responses is appropriate.

According to Krosnick (1991) using midpoint scales helps respondents choose the neutral option. However, the midpoint could also lead to less cognitive effort on behalf of the respondents who will choose the neutral point especially in controversial issues. This way the quality of the data will be minimized. On the other hand, not including the midpoint will lead respondents take side in one direction either agreeing or disagreeing with the statement (Wang and Krosnick, 1991).

Regarding the optimal number of points, most studies indicate that 5-point and 7-point Likert scales are the most reliable, and the reliability does not increase for scales with more than 7-points. Bouranta, Chitiris, and Paravantis (2009) suggested that 5-point scales can increase the response rate and are more easily understood by participants. Moreover, Weng (2004) showed that the reliability of scales does not necessarily increase by increasing the number of points and that 7-point scales could be used for respondents with cognitive ability close to that of university students.

Semantic Differential scale

In this scale only the two polars of the scale are labeled by a pair of opposite adjectives or phrases that describe the concept (friendly/unfriendly). Usually, semantic differential scales are measured using 5 or 7-point scales. Usually, the middle-point of the scale is interpreted as a neutral response. Below the product involvement scale of Zaichkowsky (1994) is presented.

Table 1. Product Involvement Scale by Zaichkowsky (1994)

Stapel scale

Stapel scale is a modified version of the semantic differential scale. Αn adjective or a phrase is placed at the center of the scale. This scale could have 6 or 10 points in numerical values ranging from +5 or +3 to -5 or -3. The Stapel scale can measure both the direction as well as the intensity of the attitude simultaneously.

Constant sum questions

The constant sum scales ask the respondent to distribute a certain number of points (usually 100 points) to two or more attributes based on the importance they assign to them. Quite often respondents find it difficult to assign the fixed sum of points to the attributes so as the total adds to 100.

Please allocate 100 points to the following criteria based on the importance you assign to each of them when making a purchasing decision about a product.

Quality ____

Quantity ____

Price ____

Utility ____

Packaging ____

_________________________

Sum 100

Paired Comparisons

This type of question requires participants to choose one option from a set of two criteria. Thus, a series of pairs is presented to respondents from which they choose their most preferred option. Paired comparisons questions are easier for respondents to answer, and they also eliminate the order bias of responses. However, this type of question might produce a high number of pairs that need to be compared by participants. Below is an example of paired comparison scales:

Which of the following platforms would you prefer (Please circle one platform from each pair):

Facebook or Instagram

Instagram or TikTok

TikTok or Facebook

YouTube or Instagram

YouTube or TikTok

YouTube or Facebook

Wording

Question wording is an important element of a well-designed questionnaire. Researchers need to avoid jargons, abbreviations, and acronyms and the use of technical terms with which the respondents are unfamiliar with. Moreover, the use of ambiguous terms should also be avoided. For example, the word “frequently” might be interpreted in a number of ways by respondents. “Double-barreled” questions should also not be utilized as they can confuse the respondent. These questions include in a statement two attributes that the participant must answer. For example, do you find the personnel of the store polite and helpful? These questions should be avoided as respondents might find the personnel polite but not helpful, thus their response will not reflect their true evaluations. The questions should not misguide and bias the respondent. For example, questions like “Do you buy lower-quality products such as Brand A?” obviously mislead the participant and should be avoided. In addition, researchers need to consider the ability of respondents to recall past events. Participants could not easily answer questions such as “How many times have you eaten popcorn during the last six months” that require him to remember specific details for a long period of time.

Sequence of Questions

The next step in the design process of the questionnaire is deciding the order of the questions. The first section of the questionnaire should include screening questions that help identify the respondents that can take part in the survey. Next, easy-to-answer questions should be inserted that will intrigue the respondent to complete the questionnaire followed by more difficult and complicated questions that require more effort from the respondent. At the end of the questionnaire classifying and demographic questions can be used. Sensitive questions should be placed near the end of the questionnaire and before the demographic questions.

Pretesting

Before collecting the data, a critical step is the pretesting of the questionnaire. Pretesting is about replicating the survey to a small number of respondents (usually 30 respondents) that share similar characteristics with the final sample. These respondents are asked to complete the questionnaire and then discuss with the researcher the problems that they faced such as questions that could not understand, unfamiliar terminology, offensive and sensitive questions etc. This feedback will help the researcher make the necessary changes to the questionnaire.

Data Collection Methods

There are several modes of data collection when administering a survey. Research can choose to conduct an in-person interview, a telephone survey, self-administered questionnaires, and an online survey. Each method has its own advantages and disadvantages.

In-person interviews require personal interaction between the interviewer and the respondent. Usually, the interviewer reads the questions to the respondent and records his/her answers. In-person interviews can be conducted in several settings (public areas, malls, etc.) and for a plethora of subjects. However, they are associated with disadvantages such as the high cost per respondent, the geographical limitations, the time pressure, and the interviewer bias who is present at the interview and may influence the way the respondent answers the questions (Oishi, 2003). Telephone surveys are a less expensive way to collect data compared to in-person interviews and can produce a high-quality sample. In the past, researchers utilized the phone book, but nowadays the random-digit dialing is the most popular way of conducting telephone surveys. The disadvantages of telephone surveys are the length of the questionnaire that might prevent the respondent from completing the survey as well as the inability to show the participant visual content. Using a self-administered questionnaire respondents complete the survey themselves and do not have any interaction with the interviewer. Thus, they cannot clarify any question or get explanations by the researcher. However, the absence of the interviewer can be viewed as an advantage as it eliminate the bias associated with his/her presence. Paper-and-pencil surveys with self-administered questionnaires are usually conducted in malls, hotels, airplanes, stores etc. Online surveys on the other hand, had become quite popular as they provide high response rates in a small amount of time and reduced costs. They also enable the anonymity of the respondent and are not geographically restricted. Online surveys can be conducted through emails, social media platforms by posting the link of the survey, websites, etc. However, the main disadvantage of online surveys is associated with the representativeness of the sample as respondents that do not have access to the Internet cannot participate in the survey. According to Evans and Mathur (2005) online surveys can raise several privacy concerns as well as confuse respondents due to the absence of the interviewer.

Sampling Process

Sampling is the process of collecting information from a group (sample) of a pre-defined population. The first step in the sampling process is defining the population. Population is the “entire group of people about whom the researcher needs to obtain information” (McDaniels and Gates, 2004). This step involves the specification of the characteristics of the population in terms of geographic, demographic, and other attributes. The next step is to identify the sampling frame (if possible). The sampling frame is the list of the members or elements of the population from which the researcher will draw the sample. But it is difficult to find a complete and accurate sampling frame due to several reasons. For example, if the population of interest is Greek doctors, then the members that have subscribed to the Greek Medical Association could be a sampling frame. This frame, however, could be incomplete or inaccurate as several members could have passed away or others might have changed their address. It should be noted also that the existence of the sampling frame is a factor that affects the sampling method that will be chosen. If a sampling frame exists, then probabilistic sampling methods are the most suitable and could produce representative samples. Then, the researcher has to select the sampling method. The method to be chosen depends on the objectives of the study, the resources, the population, the time limitations as well as the sampling frame. There are two broad sampling methods:

Probability sampling
Non-probability sampling

In probability sampling every element of the population has the same known non-zero probability of being included in the sample and the likelihood of extracting a representative sample is increased. Probability samples also allow the researcher to estimate the sampling error which is the difference between a sample value and a population value. The smaller the sampling error the more representative the sample is to the population.

The most common probability method is simple random sampling in which a sample is selected in a random way via the sampling frame. To extract the sample randomly a random number generator can be used. Another method is systematic sampling. In systematic sampling, the sample is selected based on a specific interval that is calculated by dividing the number of elements/units of the population by the sample size. Then the researcher selects a random number from the sampling frame (list) and using the interval selects the next element/unit. For example, from a population of 4000 a researcher wants to extract a sample of 100 elements. Using the systematic sampling, the interval will be (4000/100=40) 40. Then, from the list a random number is chosen (e.g., 5). After selecting the 5th element, the researcher will then choose the 45th element. Stratified sampling is another probability sampling method in which the entire population is divided into subgroups (called strata) with common characteristics. The population can be stratified based on characteristics such as age, gender, province of residence, etc). The strata need to be mutually exclusive. Then, for each stratum random sampling or another sampling method could be applied to derive the sample. For example, in a national survey about elections, each municipality could represent a stratum.

Efficiency in data collection as well as representation of all groups of interest are main advantages of the stratified sampling method. Cluster sampling is used when it is difficult to examine all the subgroups (strata) of the population. In this case, the population is divided into subgroups or clusters and then random sampling is conducted to select a number of clusters and then all members of the selected clusters are included in the sample.

On the contrary, non-probability sampling methods do not require a sampling frame and the elements that will comprise the sample are selected in a non-random way based that is based on the convenience of the researcher. Non-probability sampling methods are less expensive compared to probability. However, the sampling error cannot be calculated so the representativeness of the sample cannot be assessed.

Convenience sampling is the most popular non-probability method in which the elements of the sample are selected accidentally and based on their availability. In this method, members of the sample are recruited in convenient locations such as shopping malls, streets, events, etc. Purposive sampling is another method that allows the researcher to judge whether the population units are qualified for the sample. Thus, the researcher looks for specific cases when recruiting the members of the sample. For example, a researcher that wants to examine Christmas gifts shoppers will recruit people with shopping bags at a street. Snowballing method is a network-based sampling process in which the researcher recruits the “seeds” (the first units of the sample) that will then forward the survey to other units. Moreover, quota sampling helps the research divide the population into a number of subgroups of units that share common characteristics. These subgroups are mutually exclusive. Then, the researcher proceeds to a non-random selection of the units from each subgroup. Other non-probability sampling methods are volunteer sampling, expert selection, tele-voting, and self-selection in web-surveys (Vehovar, Toepoel, and Steinmetz, 2016).

After choosing the sampling method, the next step is deciding about the sample size. In non-probability sampling the sample size is determined based on the available resources while in probability sampling the size is decided on factors such as the levels of acceptable sampling error, the levels of confidence as well as the population size.

Data Analysis

After collecting the data, the next step is to codify the responses of the participants based on predefined levels of measurement and replace them with numerical values to be entered into the statistical program for data analysis (e.g., SPSS) and turn them into variables. There are four basic levels of measurement: nominal, ordinal, interval, and ratio.

Nominal variables have one or more categories which do not represent an intrinsic order or rank. Examples of nominal variables are gender, marital status, political preferences.
Ordinal variables have values that are ordered. For example, age groups such as 0-18, 19-25, 26-35, 36-45, 46+
Interval variables are similar to ordinal variables but the intervals between the neighboring points of the scale are equal.
Ratio variables have a true zero value and represent the actual amount of the variable (e.g., weight, height)

Depending on the type of variable researchers can choose the type of statistical analysis that is more appropriate. Data analysis can be performed through descriptive statistics as well as inferential statistics.

Descriptive statistics summarize the characteristics of the sample as well as present them in graphs. For nominal and ordinal variables, descriptive statistics can take the form of frequencies and percentages and the most suitable graphs are pie charts as well as bar charts.

For interval as well as ratio variables, central tendency and dispersion measures can be calculated such as mean scores, median, mode, variance, and standard deviation while histograms are the most suitable graphs for presenting these types of variables.

Inferential statistics aim at making inferences for the larger population based on the values that are derived from the sample. Inferential statistics can take the form of hypothesis testing as well as regression analysis.

Hypothesis testing involves five steps. First the hypothesis should be stated. A hypothesis is an assumption (in the form of statements) about a relationship between two or more variables that is going to be confirmed or rejected through statistical tests. Hypotheses take two forms: the null hypothesis (H0) that test the non-existence of a relationship and the alternative hypothesis (H1) that test the existence of the relationship under examination. For example, if a researcher wants to test the relationship between age and purchasing intention then the two hypotheses will be stated as follows:

H0: There is no relationship between age and purchasing intention.

H1: There is a relationship between age and purchasing intention.

The next step is the selection of the appropriate statistical test. This decision is usually based on the types of variables.

Chi-square test: This test is appropriate for examining the relationship between two nominal variables.
Kendall’s tau-b correlation coefficient can be used to test the correlation between two ordinal variables.
Pearson’s R correlation coefficient is suitable for testing relationships between two intervals as well as two ratio variables.
One sample t-test examines whether the mean score of an interval or a ratio variable differs from a test value.
Independent samples t-test is used when there is a dichotomous variable and an interval or ratio variable, and the researcher wants to examine whether two groups that are derived from the dichotomous variable differ in their mean scores of the ratio variable.
Analysis of variance is used for a nominal or ordinal variable with more than three categories and an interval or ratio variable. In this case the researcher examines the differences in the mean scores of interval or ratio variables across three or more groups.
Linear regression is used to predict an interval or ratio variable based on one or more independent variables.

After executing the appropriate test, then the statistical value of the test (sig 2-tailed) is compared to a predefined significance level (a) which is equivalent to 0.05 or 0.01. If the test value is greater than the significance level, then H1 is rejected. On the contrary, when the test value is lower than the significance level then the H0 is rejected, and the researcher can confirm that there is a significant relationship between the two variables.

Reliability and Validity

Often researchers measure a concept through more than two questions – items. For example, customer satisfaction can be measured using the Oliver’s (1980) scale that is comprised of the following three items:

I am satisfied with my decision to experience this activity.
I think that I did the right thing by deciding to experience this activity.
My choice to experience this activity was a wise one.

Then summative scales can be created (adding the scores of the three items) and used in subsequent data analysis. However, these summative scales or constructs need to be tested for their reliability and validity.

Reliability is the degree of the consistency of a scale and the ability to produce similar results for different samples. A common measure of reliability is the internal consistency reliability which can be measured through the Cronbach’ alpha value. A scale is considered reliable with Cronbach’s alpha value greater than 0.70.

Validity is the degree to which the scale measures what it is supposed to measure. Most of the times researchers assess the construct validity of a scale. Construct validity is related to the extent that the measurement scale is connected to the underlying theory. Two measures are utilized for measuring construct validity: (a) convergent validity and (b) discriminant validity. Convergent validity is the degree of correlation among the construct and the items that measure the construct and is calculated with measures such as the average variance extracted (greater than 0.50) as well as the composite reliability (greater than 0.70). On the other hand, discriminant validity tests the degree to which the construct is different from other unrelated constructs that are included in the study. Discriminant validity compares the average variance extracted of the construct with the square of correlations between the constructs. If the average variance extracted of the construct is greater than the square of correlation between the construct of interest and the unrelated construct, then the construct shows adequate discriminant validity.

Exercise

Suggest a research project and plan the basic steps for implementing it (research questions, questionnaire design, sampling, and expected results.

Questions

Which is the main scope and the basic limitations of quantitative research?
Which are the two wide categories of questions that can be included in a questionnaire?
Which are the two wide methods of sampling?
Which are the basic categories of variables?

References

Aldoory, L., & Toth, E. (2004). Leadership and gender in public relations: Perceived effectiveness of transformational and transactional leadership styles. Journal of Public Relations Research, 16(2), 157-183.

Babbie, E.R. (2010). The Practice of Social Research (12th ed). Belmont, CA: Wadsworth Cengage.

Bouranta, N., Chitiris, L., & Paravantis, J. (2009). The relationship between internal and external service quality. International Journal of Contemporary Hospitality Management, 21(3), 275-293.

Bryman, A., & Cramer, D. (2012). Quantitative data analysis with IBM SPSS 17, 18 & 19: A guide for social scientists. Routledge.

Carroll, B. A., & Ahuvia, A. C. (2006). Some antecedents and outcomes of brand love. Marketing Letters, 17, 79-89.

Creswell, J. W. (1994). Research design: Qualitative and quantitative approaches. Thousand Oaks, CA: Sage

Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications.

European Communication Monitor (2003), https://www.communicationmonitor.eu/2023/09/07/ecm-european-communication-monitor-2023/

Evans, J. R., & Mathur, A. (2005). The value of online surveys. Internet Research, 15(2), 195-219.

Groves, R. M., Fowler Jr, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2011). Survey methodology. John Wiley & Sons.

Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied cognitive psychology, 5(3), 213-236.

McDaniel, C. and Gates, R. (2004), Marketing Research Essentials (4th Edition), Wiley.

Oishi, S. M. (2003). A few words about qualitative interviewing. The Survey kit: How to conduct in-person interviews for surveys. https://doi.org/10.4135/9781412984416

Oliver, R. L. (1980). A cognitive model of the antecedents and consequences of satisfaction decisions. Journal of marketing research, 17(4), 460-469.

Queirós, A., Faria, D., & Almeida, F. (2017). Strengths and limitations of qualitative and quantitative research methods. European Journal of Education Studies, 3(9): 369-387.

Rahman, M. S. (2017). The advantages and disadvantages of using qualitative and quantitative approaches and methods in language “testing and assessment” research: A literature review. Journal of Education and Learning, (6): 1, pp. 102-112.

Roopa, S., & Rani, M. S. (2012). Questionnaire designing for a survey. Journal of Indian Orthodontic Society, 46(4), 273-277.

Vehovar, V., Toepoel, V., & Steinmetz, S. (2016). Non-probability sampling. In C. Wolf, D. Joye, T. W. Smith, & Y. Fu, (Eds.), The Sage handbook of survey methods (pp. 329–345). Thousand Oaks, CA: Sage Publications.

Wang, R., & Krosnick, J. A. (2020). Middle alternatives and measurement validity: A recommendation for survey researchers. International Journal of Social Research Methodology, 23(2), 169-184.

Zaichkowsky, J. L. (1994). The personal involvement inventory: Reduction, revision, and application to advertising. Journal of advertising, 23(4), 59-70.

Zikmund, W. G. (2003). Business research methods (7th ed.). Ohio-USA: Thomson, South-Western.

Types of Quantitative Research: Cross-Sectional Surveys, Longitudinal and Cohort Studies, Experimental and Quasi-Experimental Designs

Learning outcomes

Upon completion of the study of this chapter students will be able to understand key parameters of the cross-sectional survey, the longitudinal and cohort studies and experimental and quasi-experimental designs.

Introduction

Quantitative research is a methodological approach used to systematically collect and analyze numerical data to understand and explain various phenomena (Bloomfield & Fisher, 2019) (Rutberg & Bouikidis, 2018). It focuses on measuring variables, examining their relationships, and making objective conclusions based on statistical analysis. This type of research is commonly employed in the fields of social sciences, psychology, economics, education, and natural sciences. Within the domain of quantitative research, various methodologies exist, each tailored to specific research objectives and data collection requirements. The key aspects of quantitative research contain the following:

Objective and Measurable: Quantitative research focuses on obtaining objective and measurable data. It aims to quantify phenomena, variables, and relationships using numerical values and statistical techniques. This allows for precise analysis and comparisons (Caputi, 2001).
Hypothesis Testing: Quantitative research often starts with formulating hypotheses or research questions that can be tested through data collection and statistical analysis. Hypotheses provide a framework to investigate relationships between variables and make predictions about the outcomes (Lehmann, 1992).
Structured Data Collection: Quantitative research involves the use of structured data collection methods, such as surveys, questionnaires, experiments, or observational studies. These methods ensure consistency across participants or observations, allowing for reliable and comparable data (Körner et al., 1998).
Statistical Analysis: Quantitative research employs statistical analysis techniques to analyze data and draw conclusions. Descriptive statistics summarize and describe the data, while inferential statistics help determine the significance of relationships or differences between variables (Dixon & Massey Jr., 1951).
Large Sample Size: Quantitative research often utilizes large sample sizes to increase the statistical power and generalizability of findings. By collecting data from a representative sample, researchers can make inferences about the larger population (Kaplan et al., 2014).
Objectivity and Replicability: Quantitative research aims for objectivity by minimizing bias and subjectivity in data collection and analysis. It emphasizes replicability, ensuring that the research process can be replicated by other researchers to validate the findings (Boell & Cecez-Kecmanovic, 2015).
Generalizability: Quantitative research strives to generalize findings beyond the specific sample studied. Through proper sampling techniques and statistical analysis, researchers can make claims about the broader population from which the sample was drawn (Brennan, 1992).
Reliability and Validity: Quantitative research focuses on ensuring the reliability and validity of measurements and instruments used to collect data. Reliability refers to the consistency of measurements, while validity refers to the accuracy and relevance of measurements in capturing the intended constructs (Roberts & Priest, 2006).
Quantitative Data Visualization: Quantitative research often involves visualizing data through graphs, charts, and tables to aid in data interpretation and communication of findings. Visual representations can provide insights into patterns, trends, and comparisons within the data (Friendly, 2008).

This chapter aims to explore four prominent types of quantitative research designs: (a) cross-sectional surveys, (b) longitudinal and cohort studies, (c) experimental designs, and (d) quasi-experimental designs.

Cross-Sectional Surveys

Cross-sectional surveys are widely employed to gather data at a single point in time to examine the relationship between variables or capture the prevailing characteristics of a population (Kesmodel, 2018). This research design utilizes questionnaires or interviews to collect data from a representative sample of participants. The primary purpose of cross-sectional surveys is to gather information about the characteristics, attitudes, behaviors, or opinions of a population or specific subgroups within it. The data collected is analyzed using statistical techniques to identify patterns, associations, and trends.

Cross-sectional surveys provide a valuable method to obtain a snapshot of a population’s characteristics or behaviors at a specific point in time. By gathering data from a representative sample, researchers can gain insights into various aspects of a population and inform decision-making, policy development, or further research.

Advantages:

Efficient and cost-effective data collection.
Provides a snapshot of a population at a specific time.
Allows for the examination of relationships between variables.
Suitable for generating hypotheses and exploring potential correlations.
Limitations:
Susceptible to response bias and recall errors.
May not capture temporal changes or long-term trends.

Longitudinal and Cohort Studies

Longitudinal and cohort studies involve the collection of data from the same individuals or groups over an extended period (Caruana et al., 2015). These designs allow researchers to examine changes, developments, and relationships within a population over time. Longitudinal studies follow participants, while cohort studies focus on specific groups defined by a shared characteristic (e.g., birth year, exposure to a particular event). Data collection can occur through surveys, interviews, observations, or even medical tests.

More specifically, Longitudinal and cohort studies allow researchers to observe trends, identify causal relationships, and explore the effects of time on various variables.

Longitudinal Studies

Longitudinal studies involve the collection of data from the same individuals or groups repeatedly over an extended period. These studies focus on tracking changes within individuals over time and examining the impact of time on variables of interest. Longitudinal studies can be prospective (forward in time) or retrospective (backward in time).

Prospective Longitudinal Study: Researchers select a sample and collect data from participants at multiple time points in the future. Data is collected and analyzed over an extended period to observe changes, trends, or patterns.

Retrospective Longitudinal Study: Researchers collect data from the past by examining existing records or asking participants to recall information about their past experiences. This type of study investigates how past events or conditions affect outcomes in the present.

Cohort Studies

Cohort studies involve studying a group of individuals who share a common characteristic or experience and following them over time (Grimes & Schulz, 2002). The cohort can be defined by age, occupation, exposure to a specific factor, or any other relevant characteristic. Cohort studies focus on examining the relationship between the exposure or characteristic and subsequent outcomes.

Prospective Cohort Study: Researchers select a group of individuals without a particular outcome of interest and follow them over time. They assess exposure to certain factors or interventions and track the development of outcomes.

Retrospective Cohort Study: Researchers select a group of individuals with a particular outcome and look back at their past to determine the exposure or characteristic that may have contributed to the outcome.

Longitudinal and cohort studies are commonly used in various fields, including epidemiology, psychology, sociology, education, and healthcare. They are particularly useful for studying long-term effects, developmental processes, risk factors, disease progression, social trends, and the impact of interventions or policies over time. These study designs offer valuable insights into how individuals or groups change and develop over time and provide evidence for causal relationships and temporal dynamics. However, they require careful planning, long-term commitment, and thorough data management to ensure the validity and reliability of the findings.

Advantages:

Enables the study of changes and developments over time.
Can establish temporal sequences and causal relationships.
Allows for the exploration of individual variations.
Facilitates the assessment of long-term outcomes.
Limitations:
Time-consuming.
Participants may alter their behaviors due to study awareness.
Potential for bias due to selective attrition or cohort effects.

Experimental Designs

Experimental design is a research approach that allows researchers to systematically investigate cause-and-effect relationships between variables (Bell, 2009). It involves the manipulation of independent variables to observe their impact on dependent variables while controlling for other factors. The primary goal of experimental design is to establish causal relationships by ensuring that any observed differences or effects can be attributed to the manipulated variables. Experimental designs offer a high level of control, precision, and rigor in research, making them well-suited for hypothesis testing and making causal inferences. By carefully designing and implementing experiments, researchers can gain insights into the effects of interventions, treatments, or variables on the outcomes of interest.

Advantages:

Allows for the establishment of cause-and-effect relationships.
Provides control over the research environment.
Enables replication and generalization of findings.
Limitations:
May lack external validity due to controlled settings.
Ethical considerations may limit the range of manipulations.
Practical constraints, such as cost and time, can be substantial.

Quasi-Experimental Designs

Quasi-experimental designs share similarities with experimental designs but lack random assignment to groups (Campbell & Stanley, 2015). Instead, researchers use naturally occurring groups or pre-existing conditions to form comparison groups. Quasi-experiments are employed when ethical or practical constraints prevent random assignment. Data collection methods and analysis techniques mirror those used in experimental designs. Although they do not offer the same level of control as true experimental designs, quasi-experimental designs can still provide valuable insights and evidence of causal relationships. Some key features and types of quasi-experimental designs are related to the following:

Non-Equivalent Control Group Design: In this design, participants are assigned to different groups based on pre-existing characteristics or conditions. One group receives the treatment or intervention, while the other group serves as a comparison or control group. Although the groups are not formed through random assignment, researchers attempt to select groups that are similar in relevant characteristics to reduce bias.

Time Series Design: In a time series design, multiple measurements of the dependent variable are taken before and after the implementation of an intervention or treatment. This design allows researchers to examine trends or changes in the dependent variable over time and assess the impact of the intervention. It is particularly useful when random assignment is not feasible or ethical.

Nonequivalent Dependent Variables Design: This design involves comparing groups that have not been formed through random assignment but have similar characteristics. The focus is on examining the effects of the independent variable on different dependent variables. For example, in an educational setting, the same intervention could be applied to different subjects (e.g., math and science) to evaluate its impact on each subject separately.

Interrupted Time Series Design: Interrupted time series design involves collecting data on the dependent variable over a continuous period, both before and after an intervention or treatment. The design allows researchers to examine the effects of the intervention by comparing the pre-intervention and post-intervention trends in the dependent variable.

Quasi-experimental designs are commonly used in situations where random assignment is not possible due to ethical or practical constraints. While they may have limitations in establishing strong causal relationships, they can still provide valuable insights and evidence regarding the impact of interventions or treatments. Researchers should carefully consider the design’s strengths, limitations, and potential sources of bias when conducting quasi-experimental studies.

Advantages:

Allows for the study of variables that cannot be randomly assigned.
Facilitates research in real-world settings.
Offers a balance between experimental control and external validity.
Limitations:
Limited control over confounding variables.
Generalization of findings may be limited.
Subject to bias and potential alternative explanations.

Quantitative research offers a systematic approach to exploring phenomena, relationships, and causality. Cross-sectional surveys provide snapshots of populations, while longitudinal and cohort studies capture changes over time. Experimental designs establish cause-and-effect relationships, while quasi-experimental designs address practical and ethical constraints. Each research design possesses its strengths and limitations, and researchers must carefully select the appropriate approach based on their research objectives, available resources, and ethical considerations. By utilizing these diverse quantitative research designs effectively, researchers can deepen our understanding of various phenomena, inform evidence-based decision-making, and contribute to advancements in knowledge across disciplines.

A Comprehensive Guide to Conducting Experimental Research

Experimental research is a systematic approach employed by scientists to uncover the intricacies of the natural and social world (Okoli & Schabram, 2010). It involves the deliberate manipulation of variables, careful measurement of outcomes, and rigorous analysis of data. This essay aims to provide a comprehensive guide on how to perform experimental research, highlighting key steps, considerations, and best practices to ensure the validity and reliability of findings. By understanding and implementing these principles, researchers can enhance the quality of their investigations and contribute to the advancement of knowledge in their respective fields.

Conducting experimental research involves a systematic approach to investigating a scientific hypothesis or research question. Whether you are a student or a professional researcher, this comprehensive guide will provide you with the fundamental steps to conduct experimental research successfully.

Define Your Research Question

The first step in conducting experimental research is to formulate a clear and concise research question (Alvesson & Sandberg, 2013) (White, 2017). This question should be focused, specific, and address a gap in existing knowledge. It is crucial to ensure that the question is answerable through experimental methods, as not all research questions are suitable for experimentation. Consider the potential impact and significance of the research question and its relevance to the field of study.

Clearly articulate your research question or hypothesis. A research question should be specific, measurable, achievable, relevant, and time-bound (SMART) (Hughes et al., 2021). Ensure that your question is focused and addresses a gap in existing knowledge.

Defining a research question is a crucial step in the experimental research process. A research question specifies the focus of your study and guides your investigation. Here are some key points to consider when defining your research question:

Specificity: The research question should be clear, specific, and well-defined. It should address a specific aspect or problem within your field of study. Avoid vague or overly broad questions that are difficult to answer.

Example: What is the effect of caffeine consumption on cognitive performance in college students?

Relevance: Ensure that your research question is relevant and meaningful within the context of your field. Consider how your study can contribute to existing knowledge or fill a gap in literature.

Example: How does the use of virtual reality technology impact the rehabilitation outcomes of stroke patients?

Feasibility: Assess the feasibility of your research question in terms of resources, time constraints, and available data. Make sure that the research question can be realistically answered with the available resources.

Example: What is the prevalence of burnout among healthcare professionals in a specific hospital setting?

Measurability: Your research question should be measurable and allow for the collection of empirical data. Clearly define the variables and outcomes you intend to measure.

Example: What is the relationship between hours of sleep and academic performance among high school students?

Significance: Consider the significance of your research question. Reflect on its potential impact, both academically and practically. Think about how your findings could contribute to theory, policy, or practice.

Example: How does exposure to violent video games affect aggressive behavior in children, and what are the implications for parental guidance?

Remember, a well-defined research question serves as a foundation for your study. It sets the direction, scope, and purpose of your research, helping you stay focused and on track throughout the experimental process.

Review Existing Literature

Reviewing existing literature is a crucial step in conducting experimental research. It helps you understand the current state of knowledge in your field, identify gaps or controversies, and inform your research design and hypotheses. Identify relevant theories, methodologies, and previous studies related to your research question. This step will help you refine your research question, establish the significance of your study, and identify potential variables and measurement tools.

Develop a Research Design

Choose an appropriate research design that aligns with your research question. Common designs include pre-experimental, true experimental, quasi-experimental, and factorial designs. Consider factors such as feasibility, ethical considerations, and the type of causal relationship you want to establish between variables. It involves planning the overall structure and methodology of your study to address your research question and test your hypotheses (Wahyuni, 2012).

Pre-Experimental Design: Pre-experimental designs are the simplest and least rigorous types of experimental designs. They lack some of the key features necessary for establishing strong causal relationships. These designs are often used when conducting exploratory research or in situations where it is not feasible or ethical to implement more rigorous designs.

One-Shot Case Study: In this design, a single group is exposed to a treatment or intervention, and the outcome is measured. For example, assessing the impact of a one-time workshop on participants’ knowledge levels.

One-Group Pretest-Posttest Design: In this design, a single group is measured before and after an intervention or treatment. It allows for comparison of the pre- and post-intervention measurements within the same group. For example, measuring participants’ anxiety levels before and after a relaxation training session.

True Experimental Design: True experimental designs provide a higher level of control and rigor, allowing for stronger causal inferences. These designs involve the random assignment of participants to different groups and the manipulation of independent variables.

Randomized Control Trial (RCT): In an RCT, participants are randomly assigned to either a control group or one or more treatment groups. The treatment groups receive different levels or types of interventions, while the control group does not receive any intervention. This design allows for a comparison of the treatment effects between groups.

Quasi-Experimental Design: Quasi-experimental designs share some similarities with true experimental designs but lack random assignment. They are used when random assignment is not feasible or ethical. While they provide less control, they still allow for the examination of cause-effect relationships.

Non-equivalent Control Group Design: In this design, two or more groups are compared, but they are not formed through random assignment. One group receives the intervention or treatment, while the other group does not. The researcher then compares the outcomes between the groups.

Time Series Design: This design involves measuring the same dependent variable multiple times before and after an intervention or treatment. It allows for the examination of trends or changes over time and provides some indication of whether the intervention had an impact.

Factorial Design: Factorial designs involve the manipulation of multiple independent variables simultaneously to examine their individual and combined effects on the dependent variable. They provide insights into the main effects of each independent variable and their interactions.

2 x 2 Factorial Design: This design has two independent variables, each with two levels (e.g., presence/absence, high/low). It allows for the examination of the main effects of each independent variable and their interaction.

n x m Factorial Design: This design can involve more than two independent variables, each with multiple levels. It enables the investigation of more complex relationships and interactions among variables.

Each experimental design has its own strengths and limitations. Researchers should carefully consider the research question, available resources, ethical considerations, and feasibility when selecting the most appropriate design for their study.

Remember, a well-developed research design provides a framework for the successful implementation of your experimental study. It ensures that you have a systematic plan in place to gather reliable and valid data that can effectively address your research question and test your hypotheses.

Identify and Handle Variables

Identifying and controlling variables is vital to ensure the validity of experimental results. The independent variable is the variable manipulated by the researcher, while the dependent variable represents the outcome that is measured. Operationalize these variables by clearly defining how they will be measured or manipulated. It is essential to control extraneous variables – factors that may influence the dependent variable but are not the focus of the study – by using randomization, counterbalancing, and statistical techniques. Additionally, researchers should ensure that confounding variables are minimized or eliminated to accurately attribute the effects observed to the independent variable (Sharma et al., 1981).

Formulate Hypotheses

Based on your research question and knowledge from the literature review, formulate testable hypotheses. Hypotheses state the expected relationships between variables. A null hypothesis assumes no relationship, while an alternative hypothesis predicts a relationship (Dawes, 2011).

Below is an example of formulating hypotheses related to the use of virtual reality:

Research Question: How does the use of virtual reality technology impact individuals’ spatial navigation abilities?

Directional Hypothesis:

H1: Participants who undergo virtual reality training will exhibit significantly improved spatial navigation abilities compared to those who do not receive virtual reality training.

Explanation: This hypothesis predicts that virtual reality training will have a positive impact on participants’ spatial navigation abilities, leading to improved performance compared to those who do not undergo virtual reality training.

Non-Directional Hypothesis:

H1: There is a significant relationship between the use of virtual reality technology and spatial navigation abilities.

Explanation: This hypothesis suggests that there is a relationship between the use of virtual reality technology and spatial navigation abilities without specifying the direction of the relationship. It allows for the possibility of both positive and negative effects.

Null Hypothesis:

H0: There is no significant difference in spatial navigation abilities between individuals who undergo virtual reality training and those who do not.

Explanation: The null hypothesis assumes that there is no relationship or effect of virtual reality training on spatial navigation abilities. It implies that any observed differences are due to chance or factors unrelated to virtual reality training.

These hypotheses provide a foundation for investigating the impact of virtual reality technology on spatial navigation abilities. Researchers can collect data and conduct statistical analyses to determine whether virtual reality training has a significant effect on participants’ spatial navigation skills.

Select Participants and Sampling

Careful selection and recruitment of participants is crucial for the generalizability and validity of experimental research. Researchers must define the target population and employ appropriate sampling techniques to ensure a representative sample. Determine the target population and select a suitable sampling method to recruit participants. Common sampling techniques include random sampling, stratified sampling, and convenience sampling. Ensure that your sample size is appropriate for your research design and statistical analyses (Taherdoost, 2016).

Ethical considerations, such as informed consent, confidentiality, and protection of participants’ well-being, must be addressed throughout the research process. Institutional review boards or ethical committees often oversee the ethical aspects of the research and provide guidance on participant recruitment and informed consent procedures.

Design Experimental Materials

Once the research question is established, designing an appropriate experiment becomes imperative. The experimental design outlines the structure and procedures of the study, including the selection and manipulation of variables, the selection of participants or samples, and the allocation of experimental conditions. Researchers must carefully consider the type of experimental design that best suits their research question, such as between-subjects, within-subjects, or factorial designs. Develop or select the materials needed for your experiment, such as surveys, questionnaires, stimuli, or apparatus. Ensure that they are valid and reliable, and pilot test them if necessary (McLean, 2019).

Implement the Experimental Procedure

Executing the experiment requires meticulous attention to detail and consistent implementation of procedures. Researchers should follow a standardized protocol, ensuring that the independent variable is manipulated as intended and that data collection procedures are consistent across all participants and conditions. Maintaining a controlled environment, minimizing distractions, and documenting any unforeseen events or deviations from the original plan are essential to maintaining the integrity of the study (Leaf et al., 2020).

When implementing experimental procedures, it’s essential to establish a well-defined protocol and follow specific data collection procedures while maintaining a controlled environment. Minimizing distractions and documenting any unforeseen events are also crucial.

Protocol Development:

Develop a detailed protocol outlining step-by-step instructions for the experimental procedures. Include specific guidelines for each stage of the experiment, such as participant instructions, equipment setup, and data collection techniques.

Clearly define the manipulations of independent variables and the measurements of dependent variables.

Ensure that the protocol is clear, concise, and replicable, allowing for consistent implementation of the experiment.

Data Collection Procedures:

Specify the procedures for collecting data from participants. This may involve administering surveys, conducting interviews, performing observations, or using specialized equipment.

Standardize the data collection procedures to minimize variability and ensure consistency across participants.

Clearly define the timing and frequency of data collection points (e.g., pre-test, post-test).

Controlled Environment:

Create a controlled environment to minimize external influences on the experiment. Control factors such as lighting, temperature, noise, or distractions that could potentially affect participant responses or performance.

Designate a dedicated space for the experiment, ensuring it is free from unrelated stimuli or disturbances.

If necessary, use dividers or partitions to separate participants and minimize interactions between them.

Minimizing Distractions:

Instruct participants on how to minimize distractions during the experiment. For example, ask them to turn off mobile devices or avoid engaging in unrelated conversations.

Implement measures to reduce external distractions, such as ensuring a quiet environment or providing noise-canceling headphones.

Communicate with participants to address any concerns or distractions that may arise during the experiment promptly.

Documenting Unforeseen Events:

Maintain thorough documentation of any unforeseen events or deviations from the planned protocol. This includes unexpected participant behavior, technical issues, or external interruptions.

Record any changes or adaptations made to the experimental procedures during the study.

Note the impact of these events on the data collection process and consider their potential influence on the results during data analysis.

By developing a clear protocol, following well-defined data collection procedures, maintaining a controlled environment, minimizing distractions, and documenting unforeseen events, you can enhance the reliability and validity of your experimental procedures. These practices help ensure consistency, accuracy, and the ability to analyze and interpret the collected data effectively.

Collect and Analyze Data

Data collection involves systematically recording of observations and measurements, administering surveys, conducting observations, or using other appropriate methods. Researchers should employ reliable and valid measurement tools, taking care to minimize measurement error. After data collection, statistical analysis techniques appropriate for the research design and data type should be employed. Descriptive statistics, such as means, standard deviations, and frequencies, help summarize the data, while inferential statistics, such as t-tests or analysis of variance (ANOVA), correlation, regression, and multivariate analyses, allow for the testing of hypotheses and drawing conclusions from the data. Consider using statistical software for accurate and efficient analysis (Friedman, 2011).

Interpret Results

Interpreting the results requires a thorough understanding of statistical analysis and the context of the research question. Researchers should critically evaluate the findings, considering any limitations or potential biases that may have influenced the results. When reporting the results, clarity and transparency are paramount. Detailed descriptions of the methodology, results, and statistical analyses should be included, along with graphical representations and tables to aid comprehension. It is also important to discuss the implications of your findings in the context of existing literature, theoretical frameworks, and practical applications. Researchers must also acknowledge any potential sources of error or limitations, which contribute to the overall credibility of the study.

Draw Conclusions

Experimental research is a powerful tool for exploring causal relationships and uncovering new insights in various fields. By adhering to the key steps outlined in this essay, researchers can maximize the validity, reliability, and generalizability of their findings. From formulating a research question to interpreting and reporting results, each stage of the experimental research process demands meticulous attention and thoughtful decision-making. By upholding rigorous standards and embracing the principles of scientific inquiry, researchers contribute to the advancement of knowledge and foster innovation in their respective fields. Based on your analysis and interpretation, draw valid conclusions about your research question. Discuss the limitations of your study, potential confounding variables, and areas for further research.

Communicate Findings

Prepare a research report or manuscript to communicate your findings. Follow the guidelines of your target publication or institution. Include sections such as an abstract, introduction, methods, results, discussion, and references.

Prepare a research report or manuscript that follows the guidelines of your target publication or institution.

Structure your report with sections such as an abstract, introduction, methods, results, discussion, and references.

Clearly present your research question, hypotheses, methodology, data analysis, and main findings in a logical and organized manner.

Use appropriate headings, subheadings, tables, figures, and citations to enhance readability and understanding.

Reflect and Refine

Reflect on your research process, identify strengths, weaknesses, and areas for improvement. Incorporate feedback from peers, mentors, or reviewers to refine your research approach and enhance the quality of your future studies.

Remember, experimental research is a dynamic process that requires careful planning, rigorous execution, and critical thinking. Stay organized, adhere to ethical guidelines, and maintain open-mindedness throughout your research journey.

Exercise

Based on the presentation of the different stages of an experimental design, plan and go through the basic steps for the implementation of an experimental research.

Questions

Which are the similarities and differences between experimental and quasi-experimental designs? Under which circumstances does a researcher choose a quasi-experimental instead of an experimental design?
Which are the advantages of a longitudinal study?
Which are the advantages of a cohort study?
Which are the basic steps to conducting an experimental research?

References

Alvesson, M., & Sandberg, J. (2013). Constructing Research Questions: Doing Interesting Research. SAGE.

Bell, S. (2009). Experimental Design. In R. Kitchin & N. Thrift (Eds.), International Encyclopedia of Human Geography (pp. 672–675). Elsevier. https://doi.org/10.1016/B978-008044910-4.00431-4

Bloomfield, J., & Fisher, M. J. (2019). Quantitative research design. Journal of the Australasian Rehabilitation Nurses Association, 22(2), 27–30. https://doi.org/10.3316/informit.738299924514584

Boell, S. K., & Cecez-Kecmanovic, D. (2015). On being ‘systematic’ in literature reviews. In L. P. Willcocks, C. Sauer, & M. C. Lacity (Eds.), Formulating Research Methods for Information Systems: Volume 2 (pp. 48–78). Palgrave Macmillan UK. https://doi.org/10.1057/9781137509888_3

Brennan, R. L. (1992). Generalizability Theory. Educational Measurement: Issues and Practice, 11(4), 27–34. https://doi.org/10.1111/j.1745-3992.1992.tb00260.x

Campbell, D. T., & Stanley, J. C. (2015). Experimental and Quasi-Experimental Designs for Research. Ravenio Books.

Caputi, P. (2001). Introduction to Quantitative Research Methods: An Investigative Approach. 1–272.

Caruana, E. J., Roman, M., Hernández-Sánchez, J., & Solli, P. (2015). Longitudinal studies. Journal of Thoracic Disease, 7(11), E537–E540. https://doi.org/10.3978/j.issn.2072-1439.2015.10.63

Dawes, R. (2011). How Do You Formulate a Testable Exciting Hypothesis? In W. Pequegnat, E. Stover, & C. A. Boyce (Eds.), How to Write a Successful Research Grant Application: A Guide for Social and Behavioral Scientists (pp. 147–151). Springer US. https://doi.org/10.1007/978-1-4419-1454-5_13

Dixon, W. J., & Massey Jr., F. J. (1951). Introduction to statistical analysis (pp. x, 370). McGraw-Hill.

Friedman, D. A. (2011). How to Collect and Analyze Qualitative Data. In Research Methods in Second Language Acquisition (pp. 180–200). John Wiley & Sons, Ltd. https://doi.org/10.1002/9781444347340.ch10

Friendly, M. (2008). A Brief History of Data Visualization. In C. Chen, W. Härdle, & A. Unwin (Eds.), Handbook of Data Visualization (pp. 15–56). Springer. https://doi.org/10.1007/978-3-540-33037-0_2

Grimes, D. A., & Schulz, K. F. (2002). Cohort studies: Marching towards outcomes. The Lancet, 359(9303), 341–345. https://doi.org/10.1016/S0140-6736(02)07500-1

Hughes, A. C., Qiao, H., & Orr, M. C. (2021). Extinction Targets Are Not SMART (Specific, Measurable, Ambitious, Realistic, and Time Bound). BioScience, 71(2), 115–118. https://doi.org/10.1093/biosci/biaa148

Kaplan, R. M., Chambers, D. A., & Glasgow, R. E. (2014). Big Data and Large Sample Size: A Cautionary Note on the Potential for Bias. Clinical and Translational Science, 7(4), 342–346. https://doi.org/10.1111/cts.12178

Kesmodel, U. S. (2018). Cross-sectional studies – what are they good for? Acta Obstetricia et Gynecologica Scandinavica, 97(4), 388–393. https://doi.org/10.1111/aogs.13331

Körner, H., Söndenaa, K., Söreide, J. A., Andersen, E., Nysted, A., & Lende, T. H. (1998). Structured data collection improves the diagnosis of acute appendicitis. BJS (British Journal of Surgery), 85(3), 341–344. https://doi.org/10.1046/j.1365-2168.1998.00627.x

Leaf, J. B., Milne, C., Aljohani, W. A., Ferguson, J. L., Cihon, J. H., Oppenheim-Leaf, M. L., McEachin, J., & Leaf, R. (2020). Training Change Agents how to Implement Formal Preference Assessments: A Review of the Literature. Journal of Developmental and Physical Disabilities, 32(1), 41–56. https://doi.org/10.1007/s10882-019-09668-2

Lehmann, E. L. (1992). Introduction to Neyman and Pearson (1933) On the Problem of the Most Efficient Tests of Statistical Hypotheses. In S. Kotz & N. L. Johnson (Eds.), Breakthroughs in Statistics: Foundations and Basic Theory (pp. 67–72). Springer. https://doi.org/10.1007/978-1-4612-0919-5_5

McLean, V. L. A., Robert A. (2019). Design of Experiments: A Realistic Approach. CRC Press. https://doi.org/10.1201/9781315141039

Okoli, C., & Schabram, K. (2010). A Guide to Conducting a Systematic Literature Review of Information Systems Research (SSRN Scholarly Paper 1954824). https://doi.org/10.2139/ssrn.1954824

Roberts, P., & Priest, H. (2006). Reliability and validity in research. Nursing Standard, 20(44), 41–46.

Rutberg, S., & Bouikidis, C. D. (2018). Focusing on the Fundamentals: A Simplistic Differentiation Between Qualitative and Quantitative Research. Nephrology Nursing Journal: Journal of the American Nephrology Nurses’ Association, 45(2), 209–212.

Sharma, S., Durand, R. M., & Gur-Arie, O. (1981). Identification and Analysis of Moderator Variables. Journal of Marketing Research, 18(3), 291–300. https://doi.org/10.2307/3150970

Taherdoost, H. (2016). Sampling Methods in Research Methodology; How to Choose a Sampling Technique for Research (SSRN Scholarly Paper 3205035). https://doi.org/10.2139/ssrn.3205035

Wahyuni, D. (2012). The Research Design Maze: Understanding Paradigms, Cases, Methods and Methodologies (SSRN Scholarly Paper 2103082). https://papers.ssrn.com/abstract=2103082

White, P. (2017). Developing Research Questions. Bloomsbury Publishing.

The nature of qualitative research: main steps, theory and research, diverse paradigms

Learning outcomes

By the end of this chapter students are expected to:

appreciate the pluralistic nature of qualitative research and understand the differences between philosophical paradigms, such as critical theory, feminism, the indigenous paradigm, interpretivism, and social constructionism;
be able to identify the core (recurrent) features that most qualitative studies share, despite their diversity;
acknowledge the importance of reflexivity in qualitative research, which involves examining researcher assumptions and values, and how these can influence the research process and outcome;
have an overview of the main steps that need to be followed in designing a qualitative study;
be aware of, and able to use, appropriate evaluative criteria for assessing the quality of a qualitative research project.

Introduction

In chapter 1, qualitative research was defined as research that uses non-numerical data. However, this is a rather limited definition of qualitative research as it neglects important other features that profoundly differentiate qualitative research from traditional quantitative approaches both at ontological and epistemological level. The present chapter provides an overview of the nature of qualitative research. It first discusses the enormous diversity that characterises qualitative research in terms of its philosophical underpinnings, the types of data used, and the methods of analysis applied. It then identifies important common features that cut across this diversity, such as the naturalistic character of qualitative studies, the researcher’s personal engagement with the field, the “thickness” of the data generated, the focus on meaning, and its participatory character. A separate section is devoted on discussing reflexivity, that is, openly acknowledging the researcher’s position in relation to the phenomenon under investigation and recognising its influence on the process and outcomes of a study. Moreover, the chapter outlines the main steps that need to be followed in designing a qualitative research project and discusses appropriate evaluative criteria for judging its quality that suit its flexible and interpretative character. Finally, the chapter ends with a discussion of the main weaknesses for which qualitative research has been criticised.

Pluralism in qualitative research

Unlike quantitative research, which has a rather unidimensional (mainly positivistic) philosophy and avails of a relatively uncontested (unified) set of “tried-and-tested” methods, a dominant feature of qualitative research is its enormous variety and loose framing. This means that there are no universally agreed ways, or blueprints, of how qualitative studies should be conducted. Answers to questions such as when or how to engage with existing knowledge, what interviewing style to adopt, which analysis approach to use, how to identify a “theme” from the data, how to tell if an interpretation is “good” (and so on) are not straightforward and can create controversy in academic circles. Qualitative researchers are, therefore, faced with significant uncertainties and decision dilemmas (Holliday, 2016; Lyons, 2016; Punch, 2005).

This diversity is threefold; it concerns the paradigm positions of different qualitative studies, the types of data they rely on, and the ways these data are analysed (Punch, 2005). Even though most qualitative research is influenced by post-modernist thinking (see Chapter 1) and shares the assumption that what we call “reality” cannot be studied independently of human sense-making, there are finer paradigmatic distinctions within it. Common alternative paradigms (which may or may not be subsumed under the umbrella of postmodernism) include, but are not limited to, critical theory, feminism, the indigenous paradigm, interpretivism, and social constructionism (Walter, 2019; Coyle, 2016b; Lyons, 2016).

Critical theory challenges established forms of knowledge that are falsely viewed as “objective” or scientifically “true”, because, in reality, they incorporate unarticulated biases that serve the interests of powerful elites, creating varied forms of oppression and exploitation in society. For example, what is taught in schools is closely associated with the culture of students from dominant social groups which is bestowed greater value compared with the inherited knowledges and experiences of students from less privileged backgrounds (e.g. racial/ethnic minorities, lower socioeconomic classes, etc.). In this way, schools perpetually disempower the most vulnerable. Critical theorists proclaim that social research has traditionally functioned in this way; its knowledge claims have been largely based on the experiences of sovereign social groups, whilst the experiences of socially afflicted communities have been omitted or distorted. Research which adopts a critical theory perspective is emancipatory in character. It invites traditionally silenced groups (who may have unconsciously accepted their oppression as “natural” or “normal”) to provide authentic accounts of their experiences, helping them understand their social situation and take transformative action towards creating a more just society (Iosifidis, 2019; Ryoo and McLaren, 2010; Bourdieu, 1990; Freire, 1970).

Closely associated with critical theory are the feminist and indigenous paradigms. Feminism critiques the male-centric approach to the study of the social world, which contributes to the subordination and oppression of women (Walter, 2019; Punch, 2005). The indigenous paradigm challenges Western ways of thinking about, and studying, the social world that negatively affect indigenous communities (e.g. American Indian, Native Hawaiian or Aboriginal communities). The distorting effect of the western worldview on indigenous people is glaringly evidenced in the following extract from a Cherokee scholar (Awiakta, 1997, as cited in Walker, 2014, pp. 160-161):

I was centered and happy in my heritage until I went to college and began Western education in earnest. Everywhere I turned I found a “squared world,” a society so compartmentalized that life, including my own, had no room to move around, to breathe… I struggled against the Square World, but I unwittingly internalized it… One quiet line marked the beginning of my healing: “No more will I follow any rule that splits my soul.”…[No more] would I depart from the traditional teaching of my elders: “All of creation is one family, sacred.”

Interpretivism suggests that there is no single shared reality, but instead, multiple realities which correspond to the subjective ways people understand and experience the world (Sarafidou, 2011). For example, using swear words may be perceived as “misbehaviour” by middle-class teachers in a sub-urban high school of Greece, whereas for teenage boys, who are involved in this behaviour, it could just signify their transition from childhood to masculine adulthood. Unlike critical theory or the feminist/indigenous paradigms, interpretivism does not focus on “correcting” the power imbalances that make certain (hegemonic) versions of “reality” predominate and dictate social order. Hence, interpretivist research is not necessarily emancipatory in character and has a rather weak political/ethical lens (Gemma, 2018).

Social constructionism goes further than just exploring how people perceive the social world to identify how people actually construct this world through the use of language. It does not view language as merely a means of gaining access to (or describing) people’s subjective realities, but as a tool of actively constructing these realities by selecting from a range of linguistic resources (e.g. words, phrases, cliches etc.) that are available to them (Iosifidis, 2019; Coyle, 2016b).

As regards the types of data used in qualitative research, these are greatly varied, including texts, photos, drawings, videos, audio, and even smells or tastes. These may come from multiple sources, such as interview transcripts, conversation recordings, observational notes, personal diaries, minutes of meetings, TV programmes, ceremonies, performances, and other artistic (or non-artistic) creations. In fact, some qualitative researchers see literally everything as “data”. As opposed to quantitative data (which have predefined, highly standardised, categories/codes) qualitative data tend to be highly unstructured, allowing participants describe their emotions, thoughts, and experiences in their own ways (using their own terms, meanings, and understandings) instead of imposing upon them the researcher’s prefigured ideas. This does not mean that qualitative research is completely free from bias, but that researchers postpone their interpretations, and structuring, of the data until the analysis stage of a study. There is usually very little structure before data are collected which allows the emergence of structure when data are being analysed (Punch, 2005).

Paradigm and data diversity has given rise to many different methods of data analysis that can be flexibly applied either in combination or as stand-alone approaches, depending on the research questions one seeks to answer. These different methods are not meant to be slavishly applied as “recipes” for producing the right outcome, but researchers should use them resourcefully. Qualitative researchers view data analysis as a much more creative process compared with quantitative researchers who are preoccupied with correctly applying given (approved) techniques (Lyons, 2016). In chapter 6.4., we will discuss some of these different data analysis methods, including thematic analysis, grounded theory analysis, discourse analysis, narrative analysis, and visual analysis.

Common features of qualitative studies

Despite their great variety, qualitative studies share important (recurrent) features, some of which must have become evident from the above analysis. These include: (Aspers & Corte, 2019; Iosifidis, 2019; Isari & Pourkos, 2015; Punch, 2005)

Unfolding structure. In qualitative studies, research questions, design, data, and methods of analysis are not neatly prespecified in advance; they develop (unfold) as the research work progresses. Hence, a researcher might start a study which, in the initial stages, appears like action research, but which gradually takes the shape of an ethnographic study as the empirical work proceeds. This evolvement is based on a process of iteration, that is, moving back and forth between different phases of a study resulting in continual adjustments by the researcher.
Naturalistic character. Qualitative researchers prefer to study people and events in their natural (real world) settings, without artificially constructing situations or interfering with certain variables for research purposes.
Researcher’s personal engagement with the field. Qualitative research involves direct close contact with participants, as well as prolonged exposure to, and connection with, a certain “field” or life situation. As Aspers and Corte (2019, p. 148) note (referring to Emerson, 1988), qualitative research involves the “resocialization of the researcher through intense immersion in others’ social worlds”.Even if a study is limited to using only secondary data sources (such as texts, visual material, etc.), researchers will still immerse themselves in the texts or other material being analysed.
Due to the researcher’s close proximity to given aspects of participants’ social lives, a qualitative study often generates rich (detailed) data – known as “thick descriptions” – that enable him/her to develop a holistic understanding of the phenomenon under study: its context, deeper roots, evolution, internal structure and processes, outcomes, implicit features, etc.
Focus on meaning. Qualitative researchers aim to develop empathetic understandings of how participants subjectively perceive and experience certain things, events, actions, or situations. In other words, they seek to capture the “lived experiences” and “insider meanings” of local actors in a context. To achieve this, they focus on the use of language which is considered to be a central meaning-making medium in human communities. Social meanings may also be collective, that is, shared by members of a given group or community, whilst being different from those of other groups or communities. Such collective understandings compose what is known as “culture”, which constitutes a significant focal point of qualitative research.
Participatory character. Participants in qualitative research are not treated as “subjects” from whom researchers merely “extract” useful information, but as active co-creators of the entire study. In some studies, such as those adopting a participatory research design (discussed in chapter 6.3), participants are invited to play a central role in, and assume responsibility for, formulating the research questions, collecting the data, analysing these, and taking action in collaboration with the researcher.
Use of data in their original form. Whilst quantitative research converts all data into numbers and summarises them using statistical aggregates to provide an overall picture of trends and properties that apply to an entire population, qualitative researchers often use selected segments of raw data in their original (authentic) form throughout the study, even when reporting or disseminating research findings.
Theory generation. Qualitative research is often seen as primarily inductive in character, contributing to the development of theories (explanations of phenomena) based on rich data generated in the field. This type of research is particularly useful for exploring a completely unknown domain (in which there are no preexisting theories) or when one wants to re-examine something familiar in a completely new light.
Powerful convincing. As qualitative methods are not fixated but can be creatively adapted to meet the requirements of different studies, what becomes important for the researcher is to convince their audience about the internal consistency and theoretical validity of their study. This is far less straightforward than in quantitative research in which validity and reliability are inferred by the degree to which a researcher has carefully and unvaryingly applied specific standardised methodological procedures.
Attention to the political nature of research. Qualitative researchers have traditionally recognised that social research is politically uninnocent; it is a “human construction, framed and presented within a particular set of discourses (and sometimes ideologies) and conducted in a social context with certain sorts of social arrangements, involving especially funding, cognitive authority and power.” Social studies describe the world using selected concepts and methods that serve particular purposes (and interests!) and should never be seen as “neutral academic tools” (Punch, 2005, p. 135).
Qualitative researchers place great importance on openly acknowledging their own position in relation to the phenomenon under study, recognising that their ideologies, personal understandings, previous experiences, linguistic styles, ways of interacting with participants (and so on) greatly affect (and even determine) the research process and outcomes. Such recognition takes the form of a “reflexive” stance, a key feature of qualitative studies that is further discussed below.

7.4. Reflexivity

Reflexivity refers to the critical examination of one’s own assumptions, values, and practices during a study and considering how these may have influenced the research process and outcome. It requires openness and acceptance of the fact that the researcher is not a detached observer of social reality, but an active co-creator of it (Hammond & Wellington, 2021). As Johnson & Duberley (2003, p. 1279) put it, having a reflexive stance means that researchers engage in “thinking about [their] own thinking”. Yet, being reflexive does not imply that researchers should treat their own influence on a study as a “contaminating factor” which must be eliminated. What they should aim for is to increase transparency about which aspects of their personal philosophies (or interpretative frameworks) are most likely to have affected their research, so that readers can better understand and evaluate it. Such reflexive accounts may be incorporated into the methodology section of a research report (Coyle, 2016a).

It is common for qualitative researchers to keep a reflexive diary (or journal) for the duration of a project. In this diary, they may include a mini biography (e.g. academic background, family life, upbringing, etc.), record their thoughts and emotions about the topic of research (and how these may have impacted on the study), comment on their relationships with research participants, make procedural notes about what they did during the project and why, identify key decisions made and their rationale, explain how they created meaning from the data, and so on. Nadin and Cassell (2006) discuss the use of a reflexive diary in a doctoral study which explored the psychological contract in small businesses in the UK. The study was qualitative in nature and involved in-depth interviews with employers and employees from varied firms. Following an interview with an employer, the first author wrote down in her reflexive diary the following comments, expressing animosity for how the employer had treated her and considering his behaviour as being patronising, racist and sexist:

Aaaaggghhh!!….. Arrogant pig. Who’d work for him???!!! Sexist bigot. Felt really uncomfortable . . . like a little girl who was being told how it was in the world of the small business MAN!!! Liked the sound of his own voice. All his staff had been there a long time, all recruited through word of mouth. (Nadin & Cassell, 2006, p. 214)

Later in the article, the author comments:

…this experience highlighted the need to be aware of how my anger and dislike of him might impact upon the interpretation of his transcript… …The incident led me to reflect on the power dynamics of the interview situation and my role within that. This initially led me to question why I did not challenge his views and the implicit collusion signalled by remaining both silent and passive. This itself prompted consideration of whether indeed I should have challenged his views, (did I have a right to do that?), and what I would do if it happened again… Whilst comments regarding my dislike for the interviewee in question are not disputed, what is questioned is my naive assumption that transparency and objectivity are possible…Revisiting the example now, different reflexive interpretations are prompted. This is based on a consideration of the question: what function did the interview serve for the interviewee? For example, it is possible to regard the whole interview situation as an impression management exercise for the interviewee… the views he expressed, which I experienced as racist and sexist, could be regarded as extensions of a broader identity (e.g. that of a fair but firm businessman who has seen enough to know what people are like, including women). Alternatively, his comments could be regarded as reflecting the power dynamics of the interview situation in which he arguably occupied a superior position. (Nadin & Cassell, 2006, p. 214)

In the above example, we see how the researcher’s gender, age, and status (i.e. young female student) stimulated a certain reaction from the interviewee (male employer) which could have been very different if an older, more established, male researcher had carried out the interview. We also realise that the researcher is making a serious effort to understand her own part in co-shaping the situation under study (through the influence she inadvertently exerts on the interviewee) and her determination to bring to light alternative interpretations of the phenomenon under study without limiting herself to one single “version” or “view” of reality.

Main steps in designing qualitative research

Embarking upon qualitative research involves considering a number of steps that need to be taken (or choices that need to be made), including finding a topic, formulating research questions, defining the research setting, selecting participants, and deciding on data generation methods, all of which produce a given research design. Even though similar steps are followed in quantitative studies, what differentiates qualitative research is its flexible and emergent character. This means that the topic, the research questions, the setting, and so on, are not strictly specified ahead of the actual empirical work but they unfold as the study progresses whilst remaining open to change (Punch, 2005).

Finding a research topic

Holliday (2016, p. 23) notes that conceiving a good research idea for a qualitative study largely depends on “opportunism” or one’s ability to capitalise on “problematic or otherwise puzzling social realities that people find around them, whether personal, professional or institutional”. Hence, doing qualitative research means finding ways to investigate everyday social phenomena “in whatever form they present themselves”; grasping whatever opportunities exist at a given point in time to transform real life research into the more accountable activity characterising formal (academic) research. An oncologist, for example, may want to examine the reasons why some patients fully recover from advanced malignancies despite their poor prognosis. In this way, she can produce useful knowledge to help cure other patients. She may decide to examine the illness trajectories of one or two women who survived following diagnosis of a late-stage cancer, particularly focusing on social factors or distinctive personal features that played a critical role in their healing. Such real cases encountered in a professional context constitute readily accessible research settings. Teachers are another group of professionals who have access to such ready-made research settings. They can undertake qualitative research in collaboration with their students as part of normal school activities, using varied methods of data generation, including questionnaires, interviews, focus groups, observations, analysis of documents or works of art, to name but a few.

One should be conscious though that over-familiarity with a context can lead to important issues being missed or ignored, because much of the dynamics within that context are taken for granted. Qualitative researchers ought to make a serious effort to put themselves in the shoes of a stranger in the research settings where they naturally find themselves (Holliday, 2016). A critical friend from outside the setting can prove invaluable here; s/he can provide new focus and help those undertaking a study change the lens through which they view a familiar phenomenon. Costa and Kallick (1993, p. 50) define a critical friend as “a trusted person who asks provocative questions, provides data to be examined through another lens, and offers critiques of a person’s work as a friend”. It is someone who takes the time to understand the context being studied and the intended outcomes. S/he is “an advocate of the success of that work”.

Formulating research questions

Once the broader area or topic of research has been chosen, researchers need to determine what exactly they want to find out within that area. In other words, they need to formulate specific research questions. Formulating research questions can prove far more difficult than it might initially appear. Researchers’ thinking is usually very open-ended at the outset of a study and difficult to reduce to a few precise statements. The good news is that contrary to what is expected in quantitative studies, questions in qualitative research need not be too precise or narrow; they can remain sufficiently open to allow the emergence of factors that a researcher had not thought about when embarking on a study. In some cases, the whole focus of a research can change following unforeseen discoveries in its early or middle stages (Holliday, 2016).

Isari and Pourkos (2015) provide an indicative list of phrases that can help those new to qualitative research formulate appropriate research questions:

How do participants understand…?
What meanings do participants attach to…?
How are participants’ attitudes towards, or emotions about, [a given phenomenon] developed? How do they evolve over time?
What are participants’ representations of…?
How do participants experience…?
What strategies are used to…and why?
How does [a given phenomenon] take place? How does it evolve?

To build on our previous example, let’s say that an oncologist wants to examine whether being surrounded by close relatives and a loving partner during the difficult stages of cancer plays a central role in helping patients fully restore their health. Let’s also assume that the same researcher is willing to explore how the meaning patients attach to the concept of “illness” may profoundly affect the trajectory of the disease. Possible research questions could therefore be:

How do participants understand “illness”? What “being ill” means to them? Do such understandings influence cancer prognosis and in what ways?
Who is caring for the participants? Do they receive support from close relatives and/or a loving partner? How do such relationships shape patients’ experiences of cancer?

Determining the research setting

Another important task is to establish the research setting, that is, where, when, and with whom the research will take place. The setting provides a bounded environment in which research questions can be addressed. As Holliday (2016, p. 34) notes, qualitative research “[goes] deep into a definable setting in which phenomena can be placed meaningfully within a specific social environment. Such an environment can be groups of people, institutions, cases, geographical areas, communities, texts, and so on”. A number of criteria must be applied when selecting a research setting (Holliday, 2016):

(a) having a sense of boundedness in terms of time, place, culture/experience, etc.

(boundaries may not be real but operational ones, i.e. constructed for the purposes of the research),

(b) being small and manageable,

(d) providing sufficient variety of data,

(e) being accessible to the researcher.

In our research example, the setting could be defined as “the period of life of two female patients from the moment they were diagnosed with late-stage cancer up until their full recovery”. This setting is bounded in time, place, and experience and is accessible to the researcher (the two women were her patients). It also allows the generation of rich and varied data through interviews with patients themselves, focus groups with relatives, participant observations of daily routines, and so on. Moreover, the researcher has access to participants’ medical records, something that would otherwise be very difficult to get hold of. Finally, the setting consists of two cases (women), hence it is considered sufficiently small and manageable.

Selecting participants

In qualitative research, the selection of participants is usually purposive. The aim is not to generalise to the broader population but to identify cases that are information-rich (i.e. they allow an in-depth exploration of the phenomenon under study) (Patton, 2002). Sampling techniques that fall within the broader category of “purposive sampling” include (Isari & Pourkos, 2015):

extreme or deviant cases sampling (unusual, distinctive cases);
maximum variation sampling (small samples of heterogeneous cases);
homogeneous samples (small samples of homogeneous cases);
typical case sampling (selecting cases considered “typical” or “average” as far as the phenomenon under study is concerned);
critical case sampling (based on the logic “if it happens there, it will happen anywhere” or “if it does not happen there, it will not happen anywhere”);
snowball sampling (initial selection of some participants who then help identify further participants through their social networks);
criterion sampling (selecting participants based on a very specific criterion, e.g. students who live in single-parent families in an educational study);
confirming and disconfirming cases (once data have been analysed, a researcher may want to identify further cases that could confirm or disconfirm initial findings);
theoretical sampling (aimed at producing new ideas by inviting further new cases at subsequent stages of a study; when further new cases do not offer any new conceptual insights, the sample has reached its required size);
convenience sampling (selecting whoever is readily available).

The number of participants in a study depends on how many people are needed to answer the research questions, as well as on the number of resources available to the researcher (Isari & Pourkos, 2015). Participant selection in qualitative research is discussed in more detail in Chapter 8.

Choosing methods of data generation

There are numerous data generation methods that researchers can use in a qualitative study, but not all of them are appropriate in all cases. Data needs to be generated (or collected) from the right sources using suitable tools to ensure valid research findings. Mwita (2022) conducted a systematic review of literature between 2018 and 2022 on the factors researchers consider when choosing their da-ta generation methods. Among other things, the following key factors were identified: research goal, scope of the study, research approach, sample size, time fac-tor, user-friendliness to the participants, safety of the researcher, need for triangulation, and theoretical framework. The pool of data generation methods from which researchers can choose those most appropriate for their studies is immense and ever-expanding. It includes interviews, focus groups, (participant and non-participant) observations, document analysis, audio and video recordings, photographs, blog content analysis, social media posts, and so on. The most commonly used are thoroughly discussed in Chapters 8 and 9.

Emergent research design

By the time you have formulated research questions, identified a setting, considered your sample, and drafted an initial plan of data generation methods, you will have a pretty good idea of the research design that is emerging. The design of a qualitative study may be substantially fluid and changing as the study progresses. Holliday (2016) presents an example of a study that sought to explore the subculture (values and behavioural norms) of employees working behind the counter in a local McDonalds store. Although the research started as a case study of a given local store, it gradually evolved into an ethnographic study with prolonged immersion in the field and participant observations being a dominant method of data generation. What contributed to this turn of events was the fact that the researcher was actually an employee in that store. In our earlier example of cancer research, the design could be classified as a multimethod case study using extreme cases sampling and a combination of data generation methods.

Evaluative criteria of qualitative research

Given the pluralistic, flexible, and interpretative-reflexive character of qualitative research, any criteria rooted in the positivistic tradition and used to ensure that findings objectively reflect “reality” (such as the “disciplined application of prescribed methods”, the “repeatability of process and outcomes”, or the “elimination of researcher bias”) are inappropriate for evaluating qualitative research. If such criteria were applied, any qualitative study would be found deficient (Coyle, 2016a).

An alternative way of judging quality and rigour in qualitative research is proposed by Holliday (2016, p. 8) who states that good qualitative research is one that “shows its workings every single time”. This means that “the researcher needs to justify every move” offering a detailed rationale on a wide range of issues, including the choice of social setting, participant selection, data generation methods, time spent in the field, changes in research direction, analytical procedures, how certain themes emerged, why they were significant, and so on. The same author compares this process to maths assessment in school: an answer to a math problem is not considered valid unless the steps taken to figure it out are clearly explained by the pupil. The importance of “showing one’s workings” is the reason why writing up becomes so critical in qualitative research; the final text is what sticks all research parts together, creating a coherent whole that justifies the validity of a study. The notion of “justifying” the validity of a study is also known as the “trustworthiness” of research (Denzin & Lincoln, 1998, as cited in Sarafidou, 2011, p. 85).

Referring to Yardley’s (2000) evaluative scheme, Coyle (2016a, p. 23-24) provides a more traditional list of criteria against which qualitative researchers may evaluate their work. These include “sensitivity to context”, “commitment and rigour”, “transparency and coherence”, and “impact and importance”. Sensitivity to context means that the research takes account of: (a) previous knowledge on the topic, (b) the culture and history of participants (including the researcher) and how these affect data, and (c) the specific nature of the relationship between the researcher and the researched. Commitment implies prolonged engagement with the topic and the field, whilst rigour refers to the “completeness” of data generation and analysis. Transparency involves detailed description of every aspect of the research process, resembling Holliday’s (2016, p. 8) notion of “showing one’s workings every single time”. Coherence relates to the quality of the overall narrative (i.e. the “fit” between research questions, paradigms adopted, methods of data generation and analysis, etc.). Impact and importance reflect the usefulness of a study in terms of its contribution to theory, professional practice, participants’ lives, social policies, or culture in general.

To take an example, we may consider a (hypothetical) study seeking to explore adolescent smoking and how young minors perceive their experience of smoking, i.e. what smoking means to them. Let say that the researcher generated data by conducting individual interviews with adolescents who reported that they smoked cigarettes on a regular basis. Sensitivity to context, in this case, would mean that the researcher related his/her study to previous research and theories on adolescent smoking (or adolescent consumption of substances such as alcohol or drugs). One would also expect participants to be placed in context through detailed description of their demographic and other background characteristics and consideration of how these may have affected their accounts. The researcher would also be expected to describe his/her own socioeconomic profile, reveal whether s/he had smoked whilst in adolescence, and how this experience may have affected the way s/he conducted research or interpreted data. What would also need to be reflected upon is the particularities of the researcher-participant relationship, with emphasis on the likely power differentials existing between an adult researcher and underage participants and how these may have affected research findings.

It is also worth presenting here Maxwell’s (1992) evaluative criteria which include: (a) “descriptive validity” that refers to the precision with which a researcher describes the data or, in other words, the accuracy of reporting events, behaviours, settings, times, and so on, (b) “interpretative validity” or “confirmability” which signifies that the researcher’s interpretations of the data can be confirmed by participants themselves, and (c) “theoretical validity” which suggests that theoretical concepts developed as a result of a study correspond to, or “fit”, the empirical data generated.

Finally, others talk about the “internal consistency” or “dependability” of a qualitative study, which is the extent to which the same results can be produced by different researchers if they re-alyse a given set of data (Lincoln & Guba, 1985, as cited in Sarafidou, 2011), as well as the “transferability” of findings which depends on a study providing detailed enough descriptions of a context so that results can be transferred to other similar settings (Holloway, 1997, as cited in Sarafidou, 2011). This latter criterion corresponds to what is known as “generalisability of findings” in large-scale quantitative research.

Critique of qualitative research

Qualitative research has often been criticised for having a number of weaknesses (Iosifidis, 2019; Jensen & Laurie, 2016):

It is often based on extremely small samples of participants who hardly represent any meaningful segment of a population. Hence, it is hard to know whether their characteristics, experiences, perceptions, and so on, apply more broadly to a given society.
It is time-consuming and costly. For example, 10 minutes of recorded interviewing can take an hour to transcribe and produce more than 10 pages of text. This is often the reason why qualitative researchers limit their fieldwork to a small number of participants, taking the risk of missing out key perspectives on the phenomenon under study.
“Trustworthiness” or the “ability to justify every single move” largely depends on a researcher’s communication/writing skills rather than the true quality (or rigour) of a study.
As qualitative researchers become intensely involved in their research setting, this is likely to influence and bring about significant changes to the setting itself or the phenomenon under investigation.

Many of the shortcomings outlined above can be overcome by combining qualitative and quantitative data within the same research. This is why social researchers increasingly adopt mixed methods designs, so that the strengths of quantitative approaches mitigate the weaknesses of qualitative approaches and vice versa (Sarafidou, 2011; Creswell, 2003).

Concluding remarks

This chapter pointed out the main features of qualitative research. First, it highlighted its pluralistic nature placing particular emphasis on the different philosophical paradigms that underlie it, including critical theory, feminism, the indigenous paradigm, interpretivism, and social constructionism. It then identified a number of common (recurrent) features that most qualitative studies share, despite their diversity, namely their unfolding structure, naturalistic character, depth, focus on meaning, theory generation, powerful convincing, participatory character, the researcher’s personal engagement with the field, the use of data in their original form, and reflexivity. What has been emphasized is the importance of adopting a reflexive stance when conducting a qualitative study which implies that the researcher must be able to examine his/her own assumptions and values and how these influence the research process and outcomes. Next, the chapter outlined the main steps that need to be followed in designing a qualitative study, namely finding a topic, formulating research questions, determining the research setting, selecting participants, and choosing data generation methods, all of which produce a given research design. Finally, the chapter pointed to the use of appropriate evaluative criteria when assessing the quality of a qualitative research project which are fundamentally different from those applied in quantitative research.

Self-assessment questions/quizzes

For the research that you are doing (or thinking about doing) as part of your doctoral studies, how does your personal life or professional experience influence your research questions and your choice of research setting?
Considering the different philosophical paradigms presented in section 1.6. (of chapter 1) and section 6.1.2. (of the present chapter), where would you position the research that you are planning to conduct? Explain your answer.
Using Yardley’s (2000) criteria, evaluate the study reported in Cottingham, M. D. (2012). Interaction ritual theory and sports fans: Emotion, symbols, and solidarity. Sociology of Sport Journal, 29(2), 168–185. What can you conclude about the quality of the study? Does any of the criteria seem more appropriate for this study than others?
A qualitative research problem statement:
Specifies the research methods to be utilised
Specifies a research hypothesis
Expresses a relationship between variables
Conveys a sense of emerging design
Which of the following are not true of reflexivity?
It recognises that the researcher is not a neutral observer
It is mainly applied to the analysis of qualitative data
It is part of a post-positivist tradition
A danger of adopting a reflexive stance is that the researcher can become the focus of the study

References

Aspers, P., & Corte, U. (2019). What is qualitative in qualitative research. Qualitative Sociology, 42(2), 139-160.

Bourdieu, P., & Passeron, J. C. (1990). Reproduction in education, society and culture (2^nd ed.) London: Sage.

Coyle (2016a). Introduction to qualitative psychological research. In E. Lyons & A. Coyle (Eds.), Analysing qualitative data in psychology (2^nd ed.) (pp. 9-30). London: Sage.

Coyle (2016b). Discourse analysis. In E. Lyons & A. Coyle (Eds.), Analysing qualitative data in psychology (2^nd ed.) (pp. 160-181). London: Sage.

Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods approaches. London: Sage Publications.

Freire, P. (1970). Pedagogy of the oppressed. New York, NY: Seabury Press.

Gemma, R. (2018). Introduction to positivism, interpretivism and critical theory. Nurse Researcher, 25(4), 41–49.

Hammond, M., & Wellington, J. (2021). Research methods: The key concepts (2^nd ed.). Oxon: Routledge.

Holliday, A. (2016). Doing and writing qualitative research. London: Sage Publications.

Iosifidis, T. (2019). Ποιοτικές μέθοδοι έρευνας και επιστημολογία των κοινωνικών επιστημών [Qualitative research methods and the epistemology of social sciences]. Thessaloniki: Tziola.

Isari, F., & Pourkos, M. (2015). Ποιοτική μεθοδολογία έρευνας: Εφαρμογές στην ψυχολογία και στην εκπαίδευση [Qualitative research methodology: Applications in psychology and education]. Greek Academic Electronic Textbooks, www.kallipos.gr.

Jensen, E. A., & Laurie, C. (2016). Doing real research: A practical guide to social research. London: Sage Publications.

Johnson, P., & Duberley, J. (2003). Reflexivity in management research. Journal of Management Studies, 40(5), 1279-303.

Lyons, E. (2016). Doing qualitative research: Initial questions. In E. Lyons & A. Coyle (Eds.), Analysing qualitative data in psychology (2^nd ed.) (pp. 3-8). London: Sage.

Nadin, S., & Cassell, C. (2006). The use of a research diary as a tool for reflexive practice: Some reflections from management research. Qualitative Research in Accounting & Management, September(3), 208-217.

Maxwell, J. A. (1992). Understanding validity in qualitative research. Harvard Educational Review, 62(3), 279-299.

Mwita, K. M. (2022). Factors to consider when choosing data collection methods. International Journal of Research in Business and Social Science, 11(5), 532-538.

.Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks: Sage Publications.

Pels, D. (1997). StrangePunch, K. F. (2005). Introduction to social research: Quantitative and qualitative approaches (2^nd ed.). London: Sage Publications.

Ryoo, J. J., & McLaren, P. (2010). Critical theory. In P. Peterson, E. Baker, & B. McGraw (Eds.), International Encyclopedia of Education (3rd ed.) (pp. 348–353). Oxford: Elsevier.

Sarafidou, G. O. (2011). Συνάρθρωση ποσοτικών & ποιοτικών προσεγγίσεων: Η εμπειρική έρευνα [Reconciliation of quantitative & qualitative approaches: The empirical research]. Athens: Gutenberg.

Walker, P. (2015). Indigenous paradigm research. In D. Bretherton & S. F. Law (Eds.), Methodologies in peace psychology: Peace research by peaceful means. Springer International Publishing (pp. 159-175).

Walter, M. (2019). Social research methods (4^th ed.). Oxford: Oxford University Press.

Participant selection in qualitative research-Collecting qualitative data

Learning Outcomes

By the end of this chapter students should be able to understand and implement specific methods for collecting qualitative data, such as interviewing (informal, unstructured, semi-structured) and focus groups. In addition, students should be able to follow the different steps required in order to implement qualitative methods of collecting data.

Introduction

This chapter presents and describes in detail techniques and methods used to select participants as well as the methods for collecting qualitative data and more specifically, unstructured, semi-structured/in-depth interviews and focus groups.

The intent of qualitative research is to contribute to in-depth understanding of the issue under study. Hence, the research procedures for participant selection, analysis of data, and ensuring research severity differ from those for quantitative research. Data collection methods most commonly used in qualitative research are individual or group interviews (including focus groups), observation, and document review. They can be used alone or in combination. In this section we will focus on using interviews or focus groups to collect data, and explain the principles described for participant (sample) selection, data analysis, and quality assurance across qualitative approaches.

Participant selection refers to the process of choosing individuals or entities to take part in a specific activity, program, study, research, or event. This process involves identifying and recruiting suitable candidates who meet certain criteria or qualifications required for the particular purpose. Criteria for selection can vary widely based on the objectives of the activity or study, including factors like age, gender, location, specific skills or expertise, health status, socioeconomic background, or other relevant characteristics. The selection process often involves defining eligibility criteria, recruiting potential participants, screening them based on the established criteria, obtaining informed consent, and sometimes randomization (in the case of randomized controlled trials) to ensure fairness and reduce bias. Ethical considerations and adherence to guidelines governing participant selection are also essential to safeguard the rights and welfare of the individuals involved. Decisions regarding participant selection are based on research questions, theoretical perspectives, and evidence informing the study.

Important considerations in participant selection

a) The participants selected must be able to inform important facets and perspectives related to the phenomenon being studied.
b) The sample size in qualitative research, is not generally predetermined. The number of participants depends upon the number required to inform fully all important elements of the issue being studied and usually the researcher defines it in relation to the point at which no new information, new themes, concepts or feedback is being retrieved during the data collection process. This end point is named “data saturation” (Tsiolis, 2014).

Important note: finding a potential participant who has experience with the phenomenon/issue/topic under study and is willing to share their thoughts is at the heart of a proposed study. The best topic in the world cannot be explored without willing participants.

Steps and strategies in participant selection

Steps for participant selection

The steps in participant selection can vary depending on the specific context and purpose of the activity or study. However, here is a general outline of the typical steps involved in participant selection.

Define Selection criteria: Clearly outline the criteria or qualifications that potential participants need to meet. Identify and describe the type of participant you need. Criteria may include demographic factors (age, gender, and location), specific skills or expertise, health conditions, or other relevant characteristics essential for the study.
Recruitment Strategy: Develop a plan to attract potential participants. This may involve various methods such as advertising through social media, websites, community centers, mailing lists, word-of-mouth, or collaboration with organizations relevant to the participant pool.
Initial Screening: Collect information from interested individuals through applications, surveys, or initial interviews to assess whether they meet the predetermined criteria. This step helps to narrow down the pool of potential participants.
Informed Consent: Provide detailed information about the study or activity to those who pass the initial screening. Obtain informed consent from participants, explaining the purpose, procedures, risks, benefits, and their rights regarding participation.
Final Selection: Review the applicants who have given informed consent and assess if they meet all the necessary criteria. This might involve further interviews, assessments, or evaluations to ensure they are suitable for the study or activity.
Confirmation and Participation: Once selected, confirm the participation of chosen individuals and provide them with any additional information, guidelines, or instructions they might need before the study or activity begins.
Data collection: Carry out the study or activity according to the predefined protocols while collecting relevant data or observations from the participants.
Follow-up and Closure: After the study or activity is completed, follow up with participants as needed. Provide closure by sharing results, thanking them for their participation, and addressing any concerns they might have.

Throughout these steps, it’s crucial to adhere to ethical guidelines, respect participant autonomy, maintain confidentiality, and ensure the safety and well-being of the individuals involved in the selection (Tsiolis, 2014). According to Ezzy (2002/2011:63) every social research has political dimensions and consequences for the participants. This fact is connected with the concern regarding the moral standing during the research process.

Strategies of participant selection

Participant selection is crucial for obtaining diverse perspectives and rich data. Several strategies are used to select participants based on the research objectives, the specific population being studied, and the desired depth of information. Some common participant sampling strategies in qualitative research include:

Purposeful sampling: This involves selecting participants based on specific criteria relevant to the research question or objectives. Different types of purposeful sampling include: a) Maximum variation sampling when the selected participants represent a wide range of characteristics or experiences related to the research topic; b) Homogenous sampling when chosen participants share similar characteristics or experiences to focus on a specific subgroup; c) snowball sampling when initial participants refer other potential participants who meet the study’s criteria. This method is useful for studying hard-to-reach populations.
Convenience sampling: Participants are selected based on their accessibility and availability to the researcher. This method is often used for its ease and practicality but may limit the diversity of perspectives.
Theoretical sampling: Commonly used in grounded theory studies, this approach involves selecting participants based on emerging themes or concepts that develop during data analysis. New participants are chosen to further explore these emerging ideas, allowing for a more in-depth understanding of the phenomenon.
Purposive quota sampling: Researchers establish quotas based on certain characteristics (e.g., age, gender, socioeconomic status) and purposefully select participants to meet these quotas, ensuring diversity within predefined categories.
Criterion sampling: Participants are chosen based on specific predetermined criteria relevant to the research question or objectives.
Expert sampling: Involves selecting individuals with specialized knowledge or expertise related to the research topic.
Deviant case sampling: Focuses on selecting participants who deviate from the norm or expected patterns within a particular context, providing insights into unique or exceptional situations.
Extreme or critical case sampling: Involves selecting participants who represent extreme or critical cases to gain deeper insights into specific conditions or phenomena.
Random sampling: Less commonly used in qualitative research due to the emphasis on purposeful selection, but in some instances, researchers may employ random selection methods to ensure a more statistically representative sample.

It is essential to carefully consider the strengths and limitations of each sampling strategy and choose the most appropriate method based on the research aims, population characteristics, and available resources. Combining multiple sampling strategies or employing a mixed-methods approach can also enhance the comprehensiveness and validity of qualitative research findings (Tsiolis, 2014).Αρχή φόρμας

Collecting qualitative data

Among the necessary skills that the researcher should have developed is interview control. At one end is informal interviewing characterized by a total lack of structure or control, next comes unstructured interviewing, where there is a clear plan in mind but minimum of control over the informant’s responses, and finally, the semi structured interviewing in situations where you won’t get more than one chance to interview someone. In this case there is always an interview guide, a list of questions and topics that need to be covered in a particular order (Bernard, 1995:209).

Informal Interviewing

Informal interviewing is characterized by a total lack of structure or control. It is a qualitative research technique characterized by its relaxed and conversational nature. It involves open-ended discussions between an interviewer and a participant in an informal setting, allowing for a free flow of conversation and exploration of various topics. It is commonly used in qualitative research across various fields, including sociology, anthropology, psychology, education, and market research. Researchers utilize this method to gain rich insights into individuals’ lived experiences, beliefs, behaviors, and perceptions within specific contexts. It’s particularly useful when seeking to understand complex human experiences. Unlike structured or formal interviews, which follow a predetermined set of questions, formats, and protocols, informal interviews are more flexible and unstructured. “The researcher just tries to remember conversations heard during the course of a day “in the field”. […] Informal interviewing is the method of choice during the first phase of participant observation, when you’re just settling in and getting to know the lay of the land. It is also used throughout fieldwork to build greater rapport and to uncover new topics of interest that might have been overlooked (Bernard, 1995:209).

Although informal interviews offer flexibility and rich qualitative data, they also present challenges, such as potential biases introduced by the interviewer, lack of standardization, and difficulties in analyzing unstructured data. Researchers employing this method must balance the benefits of a more naturalistic approach with the need to maintain rigor and validity in their research. The key features of Informal Interviewing include flexibility, open-endedness, casual setting, exploratory nature, relationship building.

Unstructured Interviewing

Unstructured interviewing is a qualitative research method that involves open-ended, exploratory conversations between the interviewer and participant. It consists of the informal interviewing of people during the course of an ordinary day of participant observation. There is a vast literature on how to conduct effective interviews: how to gain rapport, how to get participants to open up, how to introduce an interview and how to end one (Bernard, 1995). This method is commonly used in various fields where gaining a deep understanding of subjective experiences, opinions, perceptions, and emotions is essential. It is the most widely used method of data collection in sociology, anthropology, psychology, and social work where unstructured interviews are used to delve into individuals’ perspectives, experiences, behaviors and social interactions.

Some other fields, besides Social Sciences, that frequently utilize unstructured interviewing include: a) Healthcare and Medical fields where qualitative researchers use unstructured interviews to understand patients’ experiences, perspectives on illness, treatment adherence, healthcare needs, and the impact of health interventions; b) Marker Research and Consumer Behavior, where UI is used to gather in-depth insights into consumer preferences, perceptions, and behaviors, aiding businesses in understanding market trends and customer motivations; c) Education where researchers use unstructured interviews to explore teachers’ and students’ experiences, learning methodologies, challenges, and perceptions of educational systems; d) Criminology and Criminal Justice where UI can be valuable in understanding offenders’ experience, motivations, decision-making processes, and attitudes toward crime and criminal justice system; e) Media Studies and Communication where Researchers might conduct unstructured interviews on individuals’ attitudes and behaviors to understand audience perceptions, media consumption patterns, or the impact of media content; f) Organizational Studies and Human resources where UI are used to explore workplace dynamics, employee experiences, organizational culture, leadership styles, and employee attitudes.

Unstructured interviews offer flexibility and allow participants to express themselves freely, providing rich qualitative data that can’t be captured through more structured or quantitative methods. Researchers often use these interviews to generate hypotheses, explore complex phenomena, and gain in-depth insights into the lived experiences and perspectives of individuals within diverse contexts.

Semi-structured Interviewing

Semi-structured interviewing is a qualitative research method that involves conducting interviews guided by a flexible yet predefined set of open-ended questions, allowing for both planned inquiries and the exploration of unanticipated topics that arise during the conversation. This approach provides a balance between structure and flexibility, enabling researchers to gather in-depth insights while maintaining a degree of consistency across interviews. Some key characteristics of semi-structured interviews include:

Flexibility: While researchers have a pre-designed set of questions or topics to cover during the interview, they have the freedom to adapt the sequence and wording of questions based on the interviewee’s responses. This flexibility allows for a more natural flow of conversation and the exploration of unexpected or unanticipated areas of interest.
Open-ended questions: Questions in semi-structured interviews are generally open-ended, allowing participants to elaborate on their thoughts, experiences, perceptions, and feelings. These questions typically begin with phrases such as “Can you tell me about…?” or “How do you feel about…?” This approach encourages participants to provide detailed and nuanced responses.
Structured framework: Despite the flexibility, semi-structured interviews follow a loose structure or interview guide to ensure that key topics or themes related to the research objectives are covered. Researchers often prepare an interview protocol outlining the main areas of inquiry to maintain focus and consistency across interviews.
Probing and follow-up questions: Interviewers use probing techniques to delve deeper into participants’ responses, seeking clarification, additional details, or different perspectives. Follow-up questions are based on the participant’s previous answers and aim to extract richer and more comprehensive information.
Participant-centered approach: Semi-structured interviews prioritize the participant’s viewpoint and experiences, allowing them to express their thoughts in their own words. This approach helps in understanding diverse perspectives and uncovering underlying motivations, beliefs, and attitudes

Semi-structured interviews are commonly used in various fields such as sociology, psychology, anthropology, and market research to explore complex issues, gather qualitative data, and gain a deeper understanding of individuals’ experiences, behaviors, and perceptions. The flexibility of this method allows researchers to adapt to different participants and contexts while ensuring a systematic approach to data collection.

Focus groups

Focus groups are a qualitative research method involving a facilitated discussion among a small group of participants who share common characteristics, experiences, or interests related to the research topic. In a focus group setting, participants engage in a structured conversation led by a moderator or facilitator to explore their perceptions, opinions, attitudes, beliefs, and experiences about a specific subject.

The key characteristics of focus groups include:

Participant interaction: Focus groups encourage active interaction among participants. Participants can respond to each other’s comments, share diverse perspectives, and generate discussions that can lead to deeper insights into the topic.
Moderator guidance: A skilled moderator or facilitator leads the discussion by using a predefined set of open-ended questions or topics while also allowing the conversation to flow naturally. The moderator ensures that the discussion remains focused on the research objectives and encourages all participants to contribute.
Group dynamics: The dynamics within the group can influence the conversation. Participants may build upon each other’s ideas, challenge opinions, or offer contrasting viewpoints, creating a dynamic environment for rich data collection.
Exploration of perceptions and attitudes: Focus groups are effective for exploring participants’ perceptions, attitudes, and beliefs regarding a particular subject. They allow researchers to delve into the reasons behind these opinions and understand the range of viewpoints within the group.
Sample composition: Participants in a focus group are selected based on specific criteria relevant to the research question. The group composition may reflect diversity in terms of demographics, experiences, or relevant characteristics, allowing for a range of perspectives.
Data collection: Focus groups generate qualitative data through audio or video recordings, notes taken by the moderator or observers, and transcripts of the discussions. Researchers analyze these data to identify patterns, themes, and insights related to the research topic.

Focus groups are commonly used in market research, social sciences, healthcare, and various other fields to gather qualitative data, obtain insights into consumer behavior, evaluate products or services, explore attitudes towards specific issues, or generate ideas for further research. They provide a valuable method for understanding the complexities of human perceptions and behaviors within a social context.

Exercise

Plan the initial stages/steps of a qualitative research. Form the research question, and then conduct all the necessary steps needed to choose participants and plan the method(s) you are going to use in order to obtain the information you need to answer you research question.

Questions

Which are the similarities and differences between the informal, the unstructured and the semi-structured interviews?
Which are the main reasons why a researcher should choose to conduct focus group(s) in order to collect qualitative data? In which occasions are focus groups usually deployed as a method to gather data?
Which are the strategies followed in terms of participant selection?
Which are the basic steps that need to be followed by the researcher in order to select the participants in her/his research?

References

Babbie, E. (2011). Introduction to Social Research [Εισαγωγή στην Κοινωνική Έρευνα]. Athens: Kritiki.

Bernard, H. R. (1995). Research Methods in Anthropology: Qualitative and Quantitative Approaches. London: Altamira Press.

Ezzy, D. (2002). Qualitative Analysis, Practice and Innovation. New South Wales: Allen & Unwin.

Tsiolis, G. (2014). Methods and analysis techniques in qualitative social research [Μεθοδοι και τεχνικές ανάλυσης στην ποιοτική κοινωνική έρευνα]. Athens: Kritiki.

Qualitative Research Designs

Learning Outcomes

Upon completion of this chapter students are expected to be able to understand the main tenets of four main qualitative research designs, namely ethnography, case study, grounded theory, and participatory action research. Students should also be able to choose which of these qualitative research designs fit their research question(s) in order to implement them, bearing in mind their advantages and disadvantages.

Introduction

This chapter presents four main qualitative research designs: ethnography, case study, grounded theory, and participatory action research. According to Punch (2005, p. 63), a research design is “the basic plan for a piece of research” that “connects research questions to data”. It includes decisions about what or who will be studied, the types of data that will be used (primary, secondary, qualitative, quantitative, etc.), how participants will be selected, what tools and procedures will be used for generating data, and the methods of data analysis. It is important to note that distinguishing between different research designs is not always an easy task. There are significant overlaps that make it difficult to determine whether a particular study is one thing or another. So, for example, a case study may adopt ethnographic approaches (such as immersing oneself in a given setting for an extended period of time) whilst an ethnographic study may be perceived as a case study of a particular culture.

Ethnography

The term ethnography refers to two distinct senses: that of ethnographic research (fieldwork) and that of an ethnographic monograph (ethnographic writing). Here we will focus on ethnology as a category of anthropological research. As such, ethnology involves first-hand study of a small community or ethnic group. Such studies combine to a varying degree descriptive and analytical element. The central characteristic of conventional ethnographies is that they focus on one specific culture or society and consider theoretical or comparative generalizations from the standpoint of the ethnographic example. The origin of the modern ethnographic research tradition is traced to B. Malinowski, who as part of his Functionalist theory of society stressed the primacy of field research and Participant Observation. Also, F. Boas, like Malinowski reacted against the ‘speculative history’ of Evolutionary theory and advocated the careful description of specific cultures.

Ethnography in both US cultural and British social anthropology from the postwar period until recently had acquired a generally anti-historical or at least ahistorical perspective, concentrating on the construction of a specific cultural or social system without regard to its historical development, and relegating historical considerations to a separate area labeled as the study of social or cultural change as an aberrant rather than a normal feature of human groups. Thus, as a related tendency in this type of ethnography is the artificial isolation of units of study (the tribe, the hunting and gathering band, the peasant community) considering it as a self-contained culture or society and failing to consider regional, national, and international politico-economic and social structures with which the local community interacts. New Ethnography introduced new theoretical perspectives (Critical Anthropology, Marxist Anthropology, Dependency, and World Systems) and challenged both Structural Functionalist and Cultural Relativism. These new theoretical perspectives are conscious both of historical process and of regional, national and international power structures as these impinge on the local communities.

In fact, the distinction between ethnography and anthropology is questioned within the tradition of Ethnography since it is argued that there can be no general ‘science of man’ apart from the comparative and historical study of peoples.

Ethnographic research –Fieldwork by Participant Observation

Ethnography has generally been treated as synonymous with Fieldwork, as a method rather than a product or research. “Fieldwork by participant observation is recognized within and outside anthropology as a hallmark of the discipline” (Roldan, 1995:143).

Ethnography is highly descriptive writing about a particular group of people. It might refer to both a process (the research) and a product (the writing) and can take many forms: (i.e. life history- critical ethnography or autoethnography-feminist ethnography). Ethnography requires: the language of the culture under study, first-hand participation and interpretation, and intensive work with a few informants from that setting. This sort of description can only emerge from spending a lengthy amount of time intimately studying and living in a particular social setting (Van Maanen, 1982, p. 103-104), must use the lens of culture to understand the phenomenon being examined, must also depict the researchers’ understanding of the cultural meaning of the phenomenon.

Types of ethnographic studies:

Realist ethnographies: Narrates study from in person reporting what is observed. The research reports objective data free from personal bias and the researcher produces the participants’ views through closely edited quotes and has final word on interpretation.
Case studies: Case may be a single individual or several individuals or a process. A case may be selected because it’s unusual. The researcher develops understanding of the case by collecting multiple forms of data and locates the “case” or “cases” within their larger context.

How to do ethnographic research-some practical suggestions:

Write up field notes on a regular basis – write it down!
Write up an interview ASAP
Regularly review and develop your ideas as the research progresses
Develop strategies to deal with a huge amount of data

Advantages and Disadvantages of the method:

Advantages: one of the most valuable aspects is the depth of understanding, can challenge ‘taken for granted’ assumptions
Disadvantages: takes a long time, does not have much breadth

An ethnographic design might include an exploration of cultural themes drawn from cultural anthropology, a study of a culture-sharing group, an examination of shared patterns of behavior, belief and language, a collection of data through fieldwork experiences, a description and analysis of themes about a culture-sharing group, a presentation of description, themes, and interpretation within the context or setting of a group, a reflexivity by the researcher about his/ her impact on the research site and cultural group.

Steps in conducting ethnographic research:

Identify intent and type of design and relate intent to your research problem
Discuss approval and access considerations
Use appropriate data collection procedures
Analyze and interpret data within a design
Write report consistent with your design

Criteria for evaluating an ethnographic design:

Clearly identify culture-sharing groups.
Identify the cultural patterns.
Describe the group or case in detail.
Identify the context of the group or case.
Reflect on the researcher’s role.
Develop a broad interpretation of the patterns.
Have the interpretation flow from the description and themes.
Provide a sense of how the culture works.

How to take notes

Try to write as many direct quotes as possible so you can capture the language and emotions being used
Make sure your notes follow the conversation as spoken
Make sure you use a cover sheet and date and time each interview
Devise a system to categorize interviews for quick reference
Record your own thoughts on the interview

Unobtrusive Observations: Made with minimum research participation.

Key Cultural Ideas to watch.

Proxemics – how people use physical space
Kinesics – how people convey meaning through body language

Structured Observation: (sometimes also called systematic observation) is a technique for data collection that has two defining characteristics. First, it is part of the broad family of observational techniques in which the investigator(s) gather information directly without the mediation of respondents, interviewees, and so on. Second, it is a structured or systematic technique in which data are collected according to carefully defined rules and prearranged procedures.

Using a fixed setting to see how people interact in it for comparability.
Using an inventory for comparison across places or groups.
Usually, unobtrusive research is structured.

Participant Observation: More typical of ethnography. Generally, only works if people in the study group have become familiar with the researcher. Generally, note-taking occurs in the evening or next day out of sight of the study group. Participant observation is unstructured observation.

Ethnographic Interviewing

Research Observation is not natural or intuitive. There are specific types of interviews for subjects dependent on the sort of information needed for the research.

Types of interviews:

Exploratory
In-depth
Key Informant
Open-ended
Semi-Structured
Life History
Oral History

Exploratory is the most typical of all ethnographic Methods. Usually In-Depth to give great detail and enhance the researchers understanding. Usually Open-Ended so the respondent explains things from their viewpoint and the context of things. There are no forced choice questions. Effectively builds rapport in a conversational setting. Identifies differing views of events. Sometimes collaborators can become emotional. Conversational style is preferred—it is okay to probe or ask challenging questions. Start the interview with a culturally appropriate conversation starter. Generally, an inquiry of family, friends, even pets. Be in a comfortable setting. Privacy may be a concern (balance with personal safety). Gender issues could be important. Watch for cues of discomfort, length, annoyance. GET THE RIGHT PERSON—KEY INFORMANT.

Case studies

A case study is a research design in which one case is studied in depth, in its natural setting, using multiple methods of data generation. The aim is to develop a holistic (thick) understanding of that case; that is, to examine the entirety of the case, not selected aspects of it. The case may be an individual, a group, an organisation, a community, or a nation. It can also be a behaviour, an attribute, an action or interaction, an event, a policy, or a process (Tight, 2017; Punch, 2005). At a more general level, a case is defined as “a spatially bounded phenomenon observed at a single point in time or over some delimited period” (Patnaik & Pandey, 2019, p. 164). Hence, a case must be bounded by space and time. If the phenomenon we wish to study is not intrinsically bounded, it cannot be treated as a case.

To provide an example of a case, we could think of a co-habiting elderly couple who both have dementia and who live in a small town in close proximity to their children and grandchildren. A researcher may wish to conduct a case study of this couple to gain in-depth understanding of how they experience living with dementia and to explore the relationships developed with, and the roles/responsibilities assumed by, family members who live nearby. So, in this example, the case (or unit of analysis) would be “the elderly couple with dementia”. The researcher would also need to delimit their study to a given period of time; so, they may choose to study the couple over a six-month period starting from July and up until the end of January, which is a period that includes different quarters of the year, extended business (working) periods, as well as festive seasons. Yet, a case study need not be confined to a single case; it may well involve the comparative study of two or more cases in order to identify common patterns or significant divergences between them (Yin, 2018). So, in the above example, the researcher could choose to also study the case of another elderly woman with dementia living in the same town, who is single, lives alone, and has two carers but no family members nearby. One could regard these two cases as “polar types” that allow easy identification of contrasting patterns in the data (Patnaik & Pandey, 2019).

Even though case studies are holistic, open-ended research strategies, some focus is still required as it is not possible to study everything, even about one case. It is research questions that provide this focus (Punch, 2005). In our previous example, research questions could focus on how people with dementia cope with the practicalities of everyday life and how the existence of a family (or significant other) determine how effective or sustainable these coping mechanisms are. As case studies use multiple methods of data generation, a researcher might decide to conduct interviews with those having dementia and their carers, ask them to keep reflective diaries of daily routines, conduct participant observations by spending time with them in their houses, and/or invite them to take photographs of significant events and share these with the researcher. In studies involving larger units of analysis (e.g. in a case study of a nursing home), one may also distribute closed questionnaires to large numbers of people (residents) to generate numerical data alongside other qualitative data. From this point of view, case studies are not strictly equalitative in nature but can heavily rely on statistical information as well (Coombs, 2022).

Stake (as cited in Tight, 2017) describes three main types of case study. The intrinsic case study, which focuses on one case only because the case itself is so unique, rare, or important that it is worth studying in its own right. The instrumental case study, which (again) focuses on one case, not because the case is interesting in its own right, but because its examination can shed light on a wider issue or concern, or help improve a theory. Here, the researcher is primarily interested in gaining in-depth understanding of an issue or phenomenon and selects one bounded case in order to illustrate this phenomenon. The collective case study is an extension of the instrumental case study; it examines several cases jointly to ensure a more nuanced understanding of the issue or phenomenon under investigation. Our previous example on dementia could be regarded as a collective case study because it covers two different cases: (a) a couple with dementia enjoying close family ties and receiving help from children and grandchildren and (b) a single woman with dementia who only receives help from professional (paid) carers. These contrasting cases will allow the researcher to examine the experience of dementia from different perspectives (i.e. more holistically).

The selection of cases is generally guided by the aim of a study. Case study researchers commonly apply purposive sampling, selecting cases that permit the generation of rich information on the phenomenon under investigation. Cases may be representative ones that help capture commonplace situations or may be extreme (atypical) cases markedly different from the general pattern. For example, Patnaik and Pandey (2019, p. 168) refer to Galunic and Eisenhardt who studied “organizational adaptation in an exemplar firm that was the highest performing technology-based company in the world for several years”.

A common criticism of case studies is that they cannot provide generalisable conclusions, as they focus on one case only or on just a few cases. According to Yin (2018), however, case studies have been unnecessarily devalued by comparisons with statistical methods. Case studies aim to expand and generalise theories (analytical generalisations) rather than extrapolate probabilities. Such theory-connected generalisations are, in fact, only possible through the case study approach, because it permits the generation of rich (multifaceted) data on a given phenomenon which help reveal its deeper causes and consequences (instead of describing it in a superficial way). Theoretical insights developed through case study research can be tested for their applicability to other cases. Even though it may sound paradoxical, one case is enough to permit analytical generalisation and case-to-case transfer, as long as it has been purposefully selected (Patnaik & Pandey, 2019; Yin, 2018; Punch, 2005). An example of analytical generalisation is provided by Giannakaki and Batziakas (2016) who conducted a case study of school culture in a Secondary Vocational School of Greece revealing that much of what is often perceived as “student-centred education” is, in fact, a type of “benevolent custodialism” or, in other words, an intensely controlling teacher behaviour concealed under the cloak of “humanism”.

Grounded theory

Grounded theory is both a research design and a specific method of data analysis. This chapter discusses grounded theory as a research design, whilst chapter 6.4. looks at it as a set of procedures for analysing data. The essential idea in grounded theory is that theory is developed inductively from data. The researcher collects empirical data first, and through a systematic process of analysis, creates theory that is “grounded” (rooted) in the data. This contrasts with most traditional research approaches (esp. quantitative studies) in which researchers start with a theory in mind, formulate hypotheses based on that theory, and collect data at subsequent stages to test whether initial hypotheses can be verified (deductive reasoning) (Charmaz, 2016; Punch, 2005). Data in grounded theory may be qualitative, quantitative, or both; hence, just like case studies, grounded theory is not a strictly qualitative research approach in the technical sense of the term.

Even though grounded theory starts without a theory in mind, and aims to create one, once researchers start theorising and forming initial concepts, they soon want to test their emerging ideas by collecting further new data. They, therefore, engage in an iterative process of (first) collecting, (then) analysing data, (and then) coming up with provisional theories which are subsequently tested and refined through further collection and analysis of additional new data, and so on. This process is depicted in Figure 6.3.1. The figure indicates that developing a theory cannot rely on inductive reasoning only but necessitates theory verification as well (deductive reasoning). As shown, the first set of data to be collected is guided by research questions and is usually small. At that point, data analysis begins, generating some initial theoretical ideas. After this first analysis, the researcher decides what further data are needed to test the theoretical concepts developed. Hence, a second set of data are collected based on theoretical developments from the first analysis (Punch, 2005). This iterative process is known as “theoretical sampling” and involves actively seeking new data at consecutive stages of a study, which can refine, enrich, or even challenge emerging theories. This cycle of alternation between data collection and analysis only stops when saturation is achieved, that is, when new data are not showing any new theoretical elements but are confirming what has already been found (Glaser and Strauss, 1967). Grounded theory requires that the researcher has strong theoretical insight or “theoretical sensitivity” which refers to the ability of gaining deep (nuanced) understanding of what the data say, or in other words, giving meaning to the data by separating out what is relevant and what is not (Noble & Mitchell, 2016).

Figure 6.3.1. Theoretical sampling in grounded theory (adapted from Punch, 2005, p. 158).

Grounded theory is useful when no satisfactory theory exists on a given topic or in relatively new areas of inquiry that lack theoretical concepts for describing and explaining what goes on. In such cases, researchers approach the data with an open mind (Charmaz, 2016). To take an example, let’s assume that a company has experienced high rates of employee attrition over the last two years, resulting in the reduction of talent within its workforce. Let’s also assume that managers have tried to address the issue – but to no effect – by introducing varied financial incentives, promising career opportunities, and work-life balance measures. As the problem remains, the company might decide to conduct an original study to thoroughly examine the reasons why employees leave their jobs. Because existing theories on human motivation have not helped resolve the issue so far, a grounded theory approach is deemed appropriate for shedding new light on the problem. It may include unstructured interviews and focus groups with current and/or former employees, as well as participant observations of employee behaviour during business hours. Through the process of theoretical sampling described earlier, researchers will gain new in-depth understandings of employee attrition that will help resolve the problem in the long-term.

In grounded theory research it is discouraged to conduct a literature review before data collection and analysis. Reviewing the literature at an early stage is considered a constraining exercise rather than a guiding one. If the literature is reviewed before data collection and analysis, existing theories may impose themselves on the analysis. It is, therefore, suggested that the literature is read in later stages of a study so that theoretical concepts emerging from the data are not “contaminated” with old ideas. Yet, much grounded theory has been criticised for not being truly grounded in the data, because it is almost impossible for researchers to approach a study with a completely open mind; is not only knowledge acquired through studying the literature that researchers bring with them to the data but also their personal and professional experiences more generally (Ramalho et al., 2015; Punch, 2005).

Self-assessment questions/quizzes

Does the development of grounded theory from the analysis of qualitative data signify that an inductive approach has been adopted? Explain your answer.
Considering the different research designs discussed in this chapter, where would you position the research that you want to conduct as part of your doctoral studies? Explain your answer.

References

Charmaz, K. (2006). Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. London: Sage Publications.

Coombs, H. (2022). Case study research: single or multiple [White paper]. Southern Utah University. https://doi.org/10.5281/zenodo.7604301.

Giannakaki, M. S. and Batziakas, G. (2016). ‘This is a beautiful school.’ ‘This school is useless!!’ Explaining disengagement in a Greek vocational school through the examination of teacher ideologies. Research in Post-Compulsory Education, 21(4), 409–433.

Glasser, B. G. and Strauss, A. L. (1967). The Discovery of Grounded Theory. Chicago: Aldine.

Noble, H. and Mitchell, G. (2016). What is grounded theory? Evidence Based Nursing, 19(2), 34-35.

Patnaik, S. and Pandey, S. C. (2019). Case Study Research. In Subudhi, R. N. and Mishra, S. (Ed.) Methodological issues in management research: Advances, challenges, and the way ahead. Bingley: Emerald Publishing (pp. 163-179).

Punch, K. F. (2005). Introduction to social research: Quantitative and qualitative approaches (2^nd ed.). London: Sage Publications.

Radin, P. (1987). The Method & Theory of Ethnology. Massachusetts: Bergin & Garvey Publishers, INC.

Ramalho, R., Adams, P., Huggard, P., and Hoare, K. (2015). Literature Review and Constructivist Grounded Theory Methodology. Forum: Qualitative Social Research, 16(3), Art. 19, http://nbn-resolving.de/urn:nbn:de:0114-fqs1503199.

Tight, M. (2017). Understanding case study research: Small-scale research with meaning. London: Sage.

Van Maanen, J.; Dubbs, J. M; Faulkner, R. R. (1982). Variaties of Qualitative Research. Sage Publications.

Vermulen, Han F.; Roldan, A. (1995). Fieldwork and Footnotes: Studies in the History of European Anthropology. London: Routledge.

Yin, R. K. (2018). Case Study Research and Applications: Designs and Methods (6^th Edition). London: Sage Publications.

Analyzing qualitative data: diversity in qualitative analy-sis, coding/memoing/ abstracting/comparing, thematic analysis, nar-rative analysis, textual analysis, discourse and critical discourse anal-ysis, conversation analysis, semiotics, visual analysis

Learning outcomes

By the end of this chapter students are expected to:

understand a range of qualitative data analysis methods that are commonly applied in varied social research disciplines with emphasis on thematic analysis, visual analysis, and grounded theory analysis;
critically evaluate the advantages and disadvantages of these methods and select those most appropriate for application in their own study;
possess adequate coding knowledge and skill to apply in the analysis of real world data;
be able to generate substantive meanings from multiple pieces of empirical data and progressively proceed to higher levels of abstraction and conceptualisation in explaining a social phenomenon;
assemble and present the results of qualitative data analysis in a written report.

Introduction

This chapter provides an introduction to the analysis of qualitative data. Qualitative data includes non-numerical information in the form of texts, images, audio recordings or videos. A core process in qualitative analysis is searching for patterns of meaning within a dataset (Jensen & Laurie, 2016). There exists a vast repertoire of methods for doing this, which interconnect and overlap with one another. Yet, if applied to the same body of data, each method will most likely produce unique results which illuminate a (more or less) different aspect of the phenomenon under study compared to other methods. In this chapter, we will limit our discussion to three such methods, namely thematic analysis, grounded theory analysis, and visual analysis. Thematic analysis is perhaps the most widely used approach in qualitative research, particularly popular among young (beginning) researchers due to its simplicity and clear set of guidelines. Grounded theory analysis is another widely discussed method in the relevant literature which comprises a well-defined set of procedures. It is appropriate to apply when knowledge about a social phenomenon is either non-existent or very limited, creating the need for developing new theories. In grounded theory analysis, data generation and data analysis coevolve in an iterative process. Hence, analysis of initial data produces preliminary findings which are then tested against further new data, and so on. The whole process continues until one reaches “theoretical saturation”, the point at which additional data adds no further insight into the findings (Charmaz, 2006). The third method presented in this chapter is visual analysis, which is a form of thematic analysis applied to images rather than texts (Adler & Clark, 2015). The use of visual analysis in qualitative research is ever expanding due to the ease with which researchers and participants can create images nowadays, given their almost instant access to a smartphone.

Situating the data: the importance of context in qualitative analysis

Irrespective of the method used, qualitative data cannot be meaningfully analysed without careful consideration of the context in which they are generated. Researchers must provide adequate contextual information (e.g. who they generated the data with and why, in what circumstances, over what period of time, etc.) so that any factors possibly affecting the data are foregrounded and discussed. This, in essence, constitutes the process of “situating the data” in its context (Jensen & Laurie, 2016). So, for example, if you conducted an interview with a woman who recently gave birth to her first child in a maternity hospital in order to understand perceptions of motherhood during the first hours (or days) of parenting, you would need to take account of the location in which the interview was conducted, that is, within the walls of the hospital where the participant was cared for by staff. If the new mother reported that she felt “happy” and “content”, one should ask whether this statement was influenced by the environment in which it was made, such as the possible presence of staff, relatives, or other people in physical proximity to her. First time mothers may have felt pressure to give certain answers, felt “guilty” about sharing negative experiences/emotions while in this setting, or may have felt none of these things and have just given frank responses.

Another key aspect of the context is the researcher’s subjectivity and its influence on results. The identity and outward behaviour of the researcher can affect the data generated in important ways. For example, participants may respond differently based on whether the interviewer is a man or a woman, dressed formally or casually, and whether they feel s/he empathizes with their concerns or is an indifferent outsider (Jensen & Laurie, 2016). In our previous example, first time mothers are likely to feel uncomfortable discussing certain things with a male interviewer. So, in such a case, this should be openly discussed and taken into account in the analysis and writing up phase of the study.

Last but not least, researchers should make explicit the theoretical and philosophical lens through which they examine a given social phenomenon. We should realise that qualitative data never speak for themselves. As Clark and Braun (2016, p. 85) graphically note, “the notion that themes just emerge from data like bubbles rising to the surface of a pot of boiling water is too good to be true”. Themes develop as researchers actively engage with the data, creating (rather than uncovering) a story about the phenomenon being examined. Unlike quantitative studies, the researcher’s distinctive perspective always affects findings in unique ways rendering the criterion of reproducibility irrelevant when evaluating qualitative research (Punch, 2005).

Thematic analysis

Thematic analysis is a flexible method that can be used to address most types of research question, from studying what participants think about a phenomenon to exploring what they do in relation to it and why they do it. It can also be applied to different types of data (such as interviews, documents, photographs, or other visual material) and to datasets of different sizes (from only one participant to as many as 50 or more). Thematic analysis is also theoretically flexible and can be applied within any ontological, epistemological, or theoretical framework, from positivism and post-positivism to critical theory and social constructionism. Unlike other methods, thematic analysis has not been developed by a specific person or group of researchers and is often used interchangeably with term “content analysis” (Clarke & Braun, 2016). Its flexible character is probably the reason why it has become so popular among social researchers.

Below, we outline a structured, sequential approach to thematic analysis that includes six main phases/steps: (a) data transcription, (b) data familiarisation, (c) coding, (d) developing themes, (e) reviewing themes, and (f) writing up (Tsiolis, 2018). In practice, this process may be more recursive than linear, involving some going back and forth between phases. To better illustrate specific aspects of this process, we use examples from a case study conducted by Giannakaki and Batziakas (2016) in a vocational school of Greece which aimed to explore teachers‘ deeper assumptions about education (its place in society, the role of teachers, the role of students, the nature of learning, what “knowledge” means to them) and how these affect morale, engagement, and interpersonal relationships in the school context. Even though the study used multiple methods of data generation – including a questionnaire survey, interviews with teachers, observations of school life, photographs, and school document analysis – the examples we use are all drawn from a small set of three transcribed teacher interviews.

According to Clarke and Braun (2016), thematic analysis can be used to either capture the surface meanings of data (what participants explicitly say) or to examine the latent meanings hidden underneath what is actually said or reported. In their study, Giannakaki and Batziakas (2016) adopted the second approach, seeking to decipher teachers’ deep-seated assumptions about education which may not have been obvious even to teachers themselves. Data analysis was conducted from a social constructionist perspective that treats reality not as independent of the observer, but as dependent on how people come to know, and actually, construct it (Coyle, 2016).

Data transcription

Transcription is the process of producing a text-based version of an original audio or video recording. It involves writing down every word of a recorded interview, focus group, or other social interaction, including nonverbal cues such as laughs, pauses, nuances of the voice, and (in the case of videos) body postures, gestures, facial expressions and so on. Transcription is an important step in preparing the data for analysis. Ideally, a researcher would carry out a complete, verbatim transcription that captures the entire length of a recording. Yet, such detailed transcripts require considerable time and effort to complete and are often unrealistic. At times, it may be wiser to transcribe those parts of a recording that are directly related to a specific study, leaving out nonessential or irrelevant segments, such as introductions, participants’ background information, project descriptions, or lengthy anecdotal information that respondents might provide and which are not relevant to the topic of the study (Jensen & Laurie, 2016). The following example illustrates the differences between a verbatim (exact) transcription and an edited (reduced) version of it.

Verbatim transcription:

Interview Title – 7 October 2010

I: First, I’d like to thank you for agreeing to take part in this interview 00:00:01-7

T: No problem at all Chris. I am talking to you, you know… [2 second pause] … my appreciation for you, you know it. 00:00:04-7

I: So, let’s start with something relatively simple. Where did you first work, where was your first teaching job? 00:00:06-8

T: In a vocational school and in particular in Makrihori [name of village]. My first teaching job was there. 00:00:15-7

I: Right. Was there anything special in that school, anything about your experience there, that perhaps influenced you as a professional more generally? 00:00:23-0 T: It was a very good school, a provincial school, hence there was very good chemistry between colleagues, yeah, um, there was a positive mindset, a desire to do things… [2 second pause] … and we had a management that I would say, um, was very good. From then on, all colleagues were willing to offer. And I think this was largely due to management. That is, the management listened to and was caring to all colleagues. At the time, I drove 160 Km to go to school and 160 Km to return home, right? And it felt so easy. I mean, if he [the principal] asked you for anything, you did it with great pleasure. This had an impact on children as well.

Edited transcription:

Interview Title – 7 October 2010

I: Where was your first teaching job? 00:00:06-8

T: In a vocational school in Makrihori [name of village]. 00:00:15-7

I: Was there anything special in that school that influenced you as a professional? 00:00:23-0 T: It was a very good school, a provincial school, hence there was very good chemistry between colleagues. There was a positive mindset, a desire to do things, and we had a management that was very good. From then on, all colleagues were willing to offer. And I think this was largely due to management. That is, the management listened, and was caring, to all colleagues. At the time, I drove 160 Km to go to school and 160 Km to return home. And it felt so easy. I mean, if he [the principal] asked you for anything, you did it with great pleasure. This had an impact on children as well.

You probably observed that, in the edited version of the transcription, all grammatical and syntax errors have also been eliminated, rendering the text easier to read and comprehend. Yet this practice could significantly impoverish the quality of a transcript because it removes elements that could shed light on important aspects of a phenomenon. For example, the difficulty with which someone articulates their thoughts could partly explain why they struggle in communications with colleagues or, in the case of teachers, in communications with students.

It is advisable that researchers transcribe all recordings themselves, as this helps familiarise well with the data. However, one may also use dictation software to save considerable time. Google docs voice typing is free and works well. Dragon Naturally Speaking is better, but comes with a cost. There is also software for slowing down an audio recording, such as Transcribe or other free alternatives. For more information on transcription and related digital tools, see Paulus et al. (2014).

Data familiarisation

Once data has been transcribed, the next step is to begin reading and re-reading the transcripts to gain a good grasp of what participants are saying (Jensen & Laurie, 2016). If someone else (other than the researcher) has done all the transcribing, it is important to also listen to any original audios (or view any original video data) so that the researcher develops a thorough understanding of his/her dataset by capturing even the subtlest nuances of the voice or slightest body movements that may not have been captured in the text.

During data familiarisation, the researcher primarily looks at the semantic (obvious) meanings of what participants say. However, s/he is also expected to start engaging with the data more interpretatively by noting down any latent meanings that may be discerned (i.e. what specific words or phrases could mean beyond their obvious content). It is important, in this initial phase of data familiarisation, to make casual notes of interesting/relevant points and to write down some personal thoughts and reflections on them (memos). For example, a researcher may underline an interesting phrase, circle a keyword commonly used by participants, tentatively explain what such keyword could imply, identify a preliminary (unrefined) category of data which could later be developed into a code or theme, and so on. What needs to be kept in mind is that this phase is much more unstructured and unsystematic in character compared with the next one which refers to the systematic coding of the data (Naeem et al., 2023; Clarke & Braun, 2016).

Coding

Once data familiarisation is complete, the researcher can start coding the data (audio or video transcriptions) in a systematic way. Coding involves (a) identifying words, phrases, or other segments of data in the transcripts which offer potential answers to the research questions and (b) assigning labels to them that summarise their meaning. These labels (tags) are known as “codes”. Codes do not just capture the surface meanings of the data; they encapsulate the researcher’s interpretations of it as well. A good code should delineate a full meaning even without the data. In other words, one should be able to follow the story “told” by the data by just looking at the codes (Tsiolis, 2018; Clarke & Braun, 2016).

As one proceeds with coding, they will need to decide whether to use an existing code (or create a new one) every time something of interest is discerned in the data. Eventually, most data in the transcripts should have a code attached to them, except for those clearly irrelevant to the topic or research questions. Once the first round of coding is complete, the researcher may go through the data once again for a second round to ensure that relevant data has not been missed and that coding has been done thoroughly and consistently. In this second round, some codes may prove redundant and abandoned altogether, others will look too similar and will be merged, whilst some may be renamed to better express the meanings they seek to represent (Tsiolis, 2018; Clarke & Braun, 2016).

Table 10.1 presents an example of coding from Giannakaki and Batziakas’ (2016) study. To illustrate the process, we provide four verbatim (teacher) responses to the interview question: “How do you think a school should deal with students with difficult behaviour?” When we look at the responses, we easily notice that three of the (four) interviewees referred to the need of having a specialist in school as the first most important step for solving the problem. Interviewees 1 and 3 suggested a psychologist while interviewee 2 referred to a specialist with a combined expertise in psychology, sociology, and pedagogy. Two codes were thus developed – i.e. “need for a psychology expert” and “need for an expert” – the latter seeking to capture the broader field of expertise (beyond just psychology) mentioned by participant 2. During the second round of coding, these two codes appeared too similar to be kept separate and were aggregated to create one comprehensive code named “need for an expert authority”. This new label also included the word “authority” to reflect the great power experts can accumulate in contemporary society; such (essentially political) power enables them to channel people’s actions in multiple domains, reducing the spaces in which laypeople can have a say, and thus, posing a threat to democracy (hegemony of experts).

Working further with our example data, we notice that the first interviewee referred to pupils with difficult behaviour as “cases that are peculiar enough”, while the second one referred to them as “children with special problems”. These two phrases were initially coded as “problem lies in pupils who are ‘special cases’ (deviant)” yet, during the second round, this code was renamed as “problem lies in the individual child (inherent factors)” in order to better juxtapose it with the contrasting codes “problem lies in social factors” and “problem lies in the school itself” which offered very different explanations of difficult behaviour.

TABLE 10.2: Coding example from the teachers’ educational beliefs study (2^nd round).

Overall, 14 different codes were developed following the second round of coding (see Table 10.2):

Need for an expert authority.
Problem lies in the individual child (inherent factors).
Need for expert guidance/support.
Reformative procedure prescribed by the expert (remedy).
Child seen as partner in exploring & solving the problem.
“All children can.”
Problem lies in social factors (not the child).
Healing comes from self-understanding.
Problem lies in the school itself (structural factor).
Schools are uninteresting.
Pupils become useless/unable to work.
The solution lies in having forward-thinking (pioneering) schools.
Schools lack know-how.
Know-how and expertness = essential qualities for autonomy and liberation.

Some of these codes could be further combined (e.g. codes 1 and 3, or codes 10 and 13) as they correspond to very similar concepts. As the analysis progresses, it is possible that one’s understanding of specific codes will shift. It is therefore important to develop a clear definition of each code. This will greatly facilitate the process of adapting the coding scheme if needed (Jensen & Laurie, 2016). An example of a code definition is given below.

Definition of the code “Need for an expert authority”

This code captures the importance attached nowadays to specialists in varied “scientific” fields. Specialists have become indispensable in a broad range of contexts, including the school. They are often treated as “experts” who are able to offer informed, effective, and “objective” solutions to almost any problem in their field of specialisation. Such idealised expertise has created a situation whereby laypeople and other professionals are greatly dependent upon them to lead a satisfactory personal and/or work life. Such expert power acquires political character as people who blindly trust those “experts” can easily become victims of manipulation by them (hegemony of experts).

Developing themes

Once coding is complete, a researcher needs to start developing themes. Themes represent a higher level of abstraction, and entail a greater degree of interpretation, than codes. While codes relate to a specific aspect of the data, a theme reflects a more general pattern of meaning that cuts across codes. In practical terms, this means that the researcher must group together different codes relating to the same issue, idea, or concept in order to produce a theme that is centrally relevant to the research questions (even though, sometimes, a code that is rich in meaning may become a theme in itself). Each theme must be assigned a name that conveys its core (essential) content, while it also needs to be clearly defined (just like a code) (Clarke & Braun, 2016; Isari & Pourkos, 2015).

An overarching theme emerging from our coding example above could be named “schooling is NOT questioned”. This theme essentially brings together all codes from 1 to 8[1] and represents a dominant teacher subculture who explains difficult pupil behaviour (and apparently other problems faced within schools, such as poor learning outcomes) on the basis of factors lying outside the school, such as the qualities of individual children or the social circumstances in which they live.

Reviewing themes

Once themes are fully developed, the researcher needs to ensure that they adequately represent the content of the dataset and tell a coherent “story” that answers the research questions. Each theme must be checked against all coded data relevant to that theme to ensure there is a good fit between the two. Compiling all the data extracts associated with all the codes that are relevant to a specific theme helps to do this (Clarke & Braun, 2016; Jensen & Laurie, 2016).

A researcher must also examine and highlight how individual themes are related and connected to one another. A useful way of doing this is through creating a thematic map (Clarke & Braun, 2016). Figures 10.1 and 10.2 are examples of two thematic maps from the Giannakaki and Batziakas’ (2016) study. The two figures depict two contrasting teacher subcultures (worldviews) that exist in the same school. The first subculture (Figure 10.1) does not question the school as an institution (theme 1). Any problems of adjustment or compliance to school norms are attributed to factors beyond the school itself, such as the qualities of individual children (subtheme 3a) or the social circumstances in which they live (subtheme 3b). Overcoming these problems necessitates carefully designed interventions that treat the school as given, i.e. a taken-for-granted (universally accepted) institution in which children learn. Such interventions aim at correcting either the child itself or (the damaging impact of) the social environment in which s/he lives, without any meaningful changes to the school itself (theme 2). Such “rationalistic solutions” are developed by powerful experts, and then, imposed on

[1]It must be noted that, in reality, this overarching theme encompasses a much wider range of codes (that cut across the entire dataset) than just those presented in our example.

children either with or without consultation with them (themes 2b and 2a, respectively). Overall, this subculture of teachers sees as solution to the problem the “remoulding” of pupils (giving them a new “shape”) to make them match official school norms and expectations (theme 4). This subculture represented the majority of teachers in that school, and in our example, three of the four interviewees.

Figure 10.1: Thematic map A

In stark contrast to the first (dominant) subculture, the second subculture of teachers (Figure 10.2) starts by questioning the nature and usefulness of the school itself (theme 5). Any problems of adjustment or compliance to school norms are attributed to the school as an institution – i.e. its internal structures, including the contents, methods, and desired outcomes of teaching and learning (theme 6). It is noteworthy that these teachers do not talk about any specific children who constitute “peculiar” or “special” cases. These teachers believe that problems of pupil maladjustment to school norms or expectations could be overcome if the school itself changed fundamentally (theme 7). Although not apparent in our (rather limited) example, many interview extracts from this teacher subculture referred not just to the need of updating the curriculum to become more in line with the latest scientific and technological developments, but also talked about a new way of teaching based on co-creating the curriculum in collaboration with pupils – i.e. seeing pupils as useful partners in a mutual learning process (instead of individuals who need treatment to be “amended”). For this subculture, the desired outcome of schooling is not to “help” pupils adjust to existing structures but to let them grow into spiritually wise and autonomous citizens, liberated from external oppressive forces (theme 8). This subculture represented only a minority of teachers in that school, and in our example, just one of the four interviewees.

Figure 10.2: Thematic map B

Writing up

In writing up the results of a thematic analysis, one should begin with presenting the core themes identified, and then, use relevant data extracts to illustrate the core concepts or ideas underlying these themes. It is also important to contextualise the analysis in relation to existing theory and research. To achieve this, one does not only need to include a literature review section in the final report, but, in the analysis and/or discussion sections, the researcher must also re-engage with the relevant literature to highlight points of agreement or contradictions and to offer some possible explanations (Clarke & Braun, 2016).

Visual analysis

Visual analysis involves the analysis of data which take the form of images, such as drawings or photographs (Adler & Clark, 2014). Just like textual data, images can be analysed using a variety of different methods. In this section, we present an example of thematic analysis as applied to the drawings of children aged 7 to 11 years. The example is based on a study conducted by Giannakaki et al. (2018) in the island of Ireland (between 2015 and 2017) which explored whether primary school pupils had a meaningful voice in their schools (i.e. if, and how, they participated in school decisions) and how such participation related to the dominant culture in their schools, namely teachers’ beliefs about education and learning.

Among other things, participating pupils were asked to draw pictures of how a “good teacher” and a “good pupil” looked like. Three of these drawings are presented below. Figures 1 and 2 depict a “good teacher”, while Figure 3 depicts a “good pupil”. Following the basic steps of thematic analysis, researchers first familiarised themselves with the drawings, and then, a number of codes were identified. These included:

A focus on the individual (rather than his/her interactions with others). Both teachers and pupils were depicted as disconnected from others. Instances of social interaction were scarce.
The centrality of schoolwork and homework in pupils’ conceptualisations of a “good” teacher or pupil.
Orders flowing from teachers to pupils who follow them happily. This was reflected: (a) in the text that made part of the drawings, (b) the big size of teachers (relative to pupils), and (c) the central location that teachers had in those drawings.
Being “happy” and agreeable, i.e. in agreement and in harmony with others.

Initial codes were further examined to discern broader patterns of meanings that could be categorised into a theme. A central theme identified was ‘the school as a benign autocracy’. Schools were depicted as institutions which unilaterally imposed certain rules and regulations on their attendees that were considered beneficial and useful. In other words, schools were wrapped with the cloak of ‘benevolence’ which concealed their oppressive character. They were also injected with ‘fun’ activities offering to pupils a window of joy in their (otherwise dull) school routines (Giannakaki et al. 2018).

Grounded theory analysis

Grounded theory analysis involves identifying what is central in the data and generating theory that explains it. It entails three main steps: open coding, axial coding, and selective coding. Open coding aims to identify meaningful units of text (words, phrases, sentences, or larger sections) and generate from these data abstract conceptual categories that can be used to build theory in later stages of the analysis. These conceptual categories are known as “substantive codes” and are more abstract than the pieces of data they describe. Axial coding involves finding interconnections between the substantive codes generated during the open coding stage. The word “axial” denotes the idea of putting an axis through the data that connects the categories identified in open coding. These axial codes are also referred to as “theoretical codes”. Finally, selective coding entails finding (or selecting) one higher-order, more abstract, code (the core category) that integrates and pulls together all substantive and axial codes generated so far and gives shape to a coherent (overarching) theory that describes and explains them. It is important that this core category is validated (i.e. seen as central) by the participants whose behaviour is being studied (Payne, 2016; Charmaz, 2006; Punch, 2005; Glasser & Strauss, 1967). The three coding stages described above are graphically depicted in Figure 10.3.

Figure 10.3. The three stages of open, axial, and selective coding (adapted from https://delvetool.com/blog/ccm).

What makes grounded theory distinct from other methods of analysis is its insistence, from the very beginning, on developing abstract conceptual categories to account for the empirical data being analysed. Coding in grounded theory is not concerned with description, interpretation, or developing themes, but on abstracting and building a coherent theory. The direction is from the data to a higher level of abstraction. Only concepts generated from the data are used to ensure that the emerging theory is grounded in these data. The analyst starts with no preconceived conceptual ideas and no a priori coding schemes are applied to the data (Punch, 2005).

A central activity in open coding is making constant comparisons. Different segments of data are constantly compared with each other to help generate abstract conceptual categories (substantive codes). As coding proceeds, new segments of data are identified which are compared with previous categories to find similarities and/or differences. This may result in changing and relabelling initial categories, as well as revisiting previously coded transcripts to account for the new insights. Apart from making constant comparisons, open coding is also guided by constantly asking certain questions that help raise the data to a conceptually higher (more abstract) level. Examples of such questions are “what does this piece of data stand for, or represent?” or “what is this piece of data an example of?” (Punch, 2005, p. 207)

Once a theory has emerged that explains the data, the researcher needs to apply this theory to further new data to test its validity. This involves theoretical sampling, namely selecting new participants who can help shed further light on the research problem and test whether the developing theory is useful and transferable to a broad range of cases. In this stage, it is important to find participants who are different in some ways from those already studied. For example, a researcher may approach participants of a different gender, age group, socioeconomic background, profession, personality, lived experience, and so on. What is being sought at this stage are both confirming and disconfirming new participants so that the researcher can test the limits of his/her theory. This theory testing must continue until saturation is achieved. Saturation means that the theory can fully explain all variations in the data and that no further data can be generated that necessitates new revisions or adaptations of the theory (Hawker & Kerr, 2016).

Concluding remarks

This chapter presented three commonly used methods of qualitative data analysis: thematic (text) analysis, visual analysis, and grounded theory analysis. These methods have important similarities, including a sequence of progressive coding stages that start with the identification of preliminary categories of meaning within a set of data (initial codes) and finish with the generation of more abstract concepts or theories that adequately explain the phenomenon being studied (higher-order codes). The data may be textual, visual, or even relating to other human senses. It is important that the researcher adopts a reflexive stance when undertaking qualitative data analysis, which means that s/he must examine his/her own assumptions and values and how these influence the analysis process and outcomes. One main difference between thematic analysis and grounded theory analysis is that the former does not necessarily aim at generating one overarching theory that explains all variations in the data; it usually attains lower levels of abstraction and interpretation. Moreover, it is not unusual in thematic analysis to apply preexisting coding schemes to the data which are drawn from an initial review of the literature. Contrary to that, grounded theory analysis starts with no preconceived ideas in mind and, from the very beginning, focuses on creating an overarching theory grounded in the data which can be tested through generating further data by newly sampled participants until theoretical saturation is achieved. In grounded theory analysis, the review of the literature is only conducted when data analysis is complete.

Self-assessment exercises

Choose an online version of a national newspaper to collate a dataset of between five and 10 articles on a topic of your choice. For example, use the search terms “public” or “state university” to focus on the coverage of issues related to public tertiary education in your country. Try coding these articles using the research question “How do the media represent state universities, and the issues they face, in your country?”. In doing so, apply the principles of thematic analysis discussed in this chapter.
In Appendix A, you will find part of an interview transcript from a study that looked into how teachers understand and practise children’s voice in primary schools in a European country. Apply open coding to the data, keeping in mind the research question “How do teachers understand and practise children’s voice in their schools?” The more data you can work with, the better. You may be able to engage in constant comparison as you generate categories. It is difficult to undertake meaningful axial coding when working with only a small extract from the data. Nonetheless, see if you can find any links between the categories that you develop. Invite other students from your course to code the same dataset and compare the codes you have produced. What are the differences and similarities? Try to explain any differences identified between your codes and those of fellow students.

References

Adler, E. S., & Clark, R. (2015). An invitation to social research: How it’s done (5^th ed.). Stamford, CT: Cengage Learning.

Charmaz, K. (2006). Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. London: Sage Publications.

Clarke, V. and Braun, V. (2016). Thematic analysis. In E. Lyons & A. Coyle (Eds.), Analysing qualitative data in psychology (2^nd ed.) (pp. 84-103). London: Sage.

Giannakaki, M. S., Flynn, P., Hayes, N., and Fitzsimons, S. (2018) Teachers’ beliefs about education and children’s voice practices in the Island of Ireland (Research Report). Armagh, Northern Ireland: Centre for Cross Border Studies. Link to full text: http://scotens.org/teachers-beliefs-about-education-and-childrens-voice-practices-in-the-island-of-ireland/.

Glasser, B. G. and Strauss, A. L. (1967). The Discovery of Grounded Theory. Chicago: Aldine.

Hawker, S., & Kerr, C. (2016). Doing grounded theory. In E. Lyons & A. Coyle (Eds.), Analysing qualitative data in psychology (2^nd ed.) (pp. 147-159). London: Sage.

Naeem, M., Ozuem, W., Howell, K., & Ranfagni, S. (2023). A step-by-step process of thematic analysis to develop a conceptual model in qualitative research. International Journal of Qualitative Methods, 22, 1–18.

Paulus, T., Lester, J. N., & Dempster, P. (2014). Digital tools for qualitative research. London: Sage.

Payne, S. (2016). Grounded theory. In E. Lyons & A. Coyle (Eds.), Analysing qualitative data in psychology (2^nd ed.) (pp. 119146). London: Sage.

Punch, K. F. (2005). Introduction to social research: Quantitative and qualitative approaches (2^nd ed.). London: Sage Publications.

Terry, G. (2016). Doing thematic analysis. In E. Lyons & A. Coyle (Eds.), Analysing qualitative data in psychology (2^nd ed.) (pp. 104-118). London: Sage.

Tsiolis, G. (2018). Θεματική ανάλυση ποιοτικών δεδομένων [Thematic analysis of qualitative data]. In G. Zaimakis (Ed.), Ερευνητικές διαδρομές στις κοινωνικές επιστήμες. Θεωρητικές-μεθοδολογικές συμβολές και μελέτες περίπτωσης [Research paths in the social sciences. Theoretical-methodological contributions and case studies ] (pp 97-125). University of Crete.

Secondary research data: types and sources (archived documentary data, statistical databases, etc.). Quantitative and qualitative content analysis

Learning outcomes

In the following chapter the use of secondary research data along with the types and sources, such as interviews, archived documentary data, statistical databases, audio and video material, academic books, census reports, etc. will be deployed. Secondary data is defined as datasets collected by other people than the investigator using them, in other words using data that already exist and not collecting new (Hillier, 2022; George, 2023). Furthermore, content analysis as an analysis form of quantitative and qualitative data will be shortly examined.

Upon completion of the study of this chapter the students will be able to define what is secondary research and secondary research data. Likewise, they will be able to identify and comprehend the types and sources of secondary data, where they can be found and how they may be retrieved. Furthermore, they will be able to appreciate the ways in which secondary data can be used to help answer the research question(s) and achieve the scope of the research, along with ways that will aid in their evaluation regarding the criteria for suitability, sufficiency, and quality.

The students will be able to understand the advantages and disadvantages (strengths and limitations) of the use of secondary data and finally, the types of analysis that need to be performed will be made clear to them.

Likewise, the students will understand the basic theory of content analysis as a research method.

Introduction to Secondary Research Data

Secondary research data, in contrast to primary research data which involves collecting new data directly from individuals or sources, is data that is collected by other people than the researcher(s) that use it in a current investigation (Bhat, 2023) and synthesizing that data and the acquired information from the various sources. However, there is a case where researchers may re-use their own self-collected data to investigate new questions than those examined in the primary research or even to verify their previous findings (Heaton, 2008). It can be either qualitative or quantitative and may derive from published peer-reviewed papers, meta-analyses, or databases and datasets (George, 2023). Secondary research has become popular since publicly available data provide resources that can be re-examined and new hypotheses which vary from the original objective, may be set and analysed, thus delivering new interpretations, conclusions and knowledge (Sherif, 2018).

Secondary research analysis – basic theory

As Sherif (2018) argues, the secondary research method was initially introduced before the second World War, however, it appears to have been introduced in the literature in the 1960s, since then numerous definitions of secondary analysis have appeared. Also, as Heaton (2008, p33) points out, “since the mid-1990s, there has been growing recognition of and interest in the potential for carrying out secondary analysis of qualitative data”.

As in every research, a clear idea of the scope is mandatory. The investigator should clarify the gaps in his/her knowledge of the issues under study and regarding the questions or hypotheses that need to be tested. Therefore, he/she should define the research topic, establish the statement of purpose and design the research (Hillier, 2022). Secondary analysis may provide additional details and/or indicators of an already examined phenomenon and grant new perspectives, and it can be used to review previous research into an area of interest. However, as Kedler (2005) argues, context had to be considered at two levels, the first one being the one of the researcher(s) who originally created the secondary data and the second one the new context into which the data record was being applied.

Heaton (2008, p.39) identifies five types of secondary analysis of qualitative data: a. supplementary analysis, where an issue that was not addressed in the primary study is subject to a more in-depth analysis, b. supra analysis, where the aims of the secondary study transcend those of the original research, c. re-analysis, where data is reexamined to confirm findings of a primary study, d. amplified analysis, where two or more existing datasets may be compared or combined for the scopes of a secondary analysis and e. assorted analysis, where the re-employment of existing data is carried out alongside the gathering and analysis of primary data for the same study.

Another issue that should be considered in secondary research is the presumption of accountability on an ethical basis and whether researcher discretion and judgement are required when contemplating of using or not material such public comments that might accompany a published blog (von Benzon, 2019).

Prior to using secondary data, assessment of their overall suitability in regard to the research question(s) has to be performed, as well as assessment of their suitability according to the analyses required to answer the research question(s). Some of the queries that should be replied are:

What will be the time limit for the source search?
Does the data set satisfactorily approximate the data you need?
Does the data set cover the population of interest?
Does the data set cover the geographic area you are interested in?
Does the data include all the variables you need to answer your research questions?
How reliable is the data set you intend to use?
How reliable is the data source? Do the credentials of the data source indicate credibility? Can it be verified?
Is there a related copyright statement?
What was the original purpose for which the data was collected?
How consistent is the data from that source relative to other sources?
Is there, and if so, what are the financial and time costs of obtaining this data?

Types of secondary data

There are many types of secondary data that can be used in secondary research, including qualitative and quantitative forms but not restricted. Secondary data can be:

government statistics, such as demographic data,
technical reports, such as reports provided by institutions, organizations, foundations, companies, etc.,
social media accounts data, where besides the text other kind of data such as likes, shares, mentions, etc., may be collected,
blogs, where besides the original text written by the blogger, commentaries can also be used, however the trustworthiness of the blog must be checked,
websites, however, there is a need for checking the trustworthiness of the website and whether the uploaded data is authentic,
website analytics, such as visitor’s location, visitor’s activities in the site, most popular content, etc.,
sales data,
academic data, such as university registration information,
historical archives,
biographies,
sound recordings, such as podcasts or archive radio productions,
videos,
newspaper articles and mass media products in general,
books.

Depending on the scope of the investigation, the researcher(s) may look for the proper kind of data that meets the demands of the study.

Besides personal traditional search in libraries and the Internet, large volumes of secondary data can be collected by dedicated software which are called bots (from robots) that perform automatic web scraping to find the needed data.

Advantages of secondary research analysis

Although secondary research may not allow the extent of control as primary research, it can be a very valuable instrument for gathering new knowledge and identifying trends since a very large amount of data coming even from different sources can be used (Bhat, 2023). An important advantage of secondary research analysis is that it can broaden and deepen knowledge by providing another aspect and further comprehensive understanding of the nature of an issue. Even the same authors may use their original data to examine fresh arguments, thus revealing additional context through comparative and specialized analysis of the results (Sherif, 2018).

Another advantage is that using already collected data may save resources, such as time and money. Data collection can be the most expensive part of a research procedure and surely time consuming (Sherif, 2018), by using secondary data that can be low-cost or totally free, the expenses of an investigation are greatly reduced (Valcheva, 2023). Secondary data analysis provides the investigators more time to deal with their theoretical goals and other essential matters, since the time used to gather it is absolutely minimized.

Furthermore, in cases where privacy or special treatment is required with certain sample categories (such as children), the employment of secondary analysis may protect identities and privacy while providing new aspects (Sherif, 2018).

Likewise, since secondary data can be traced back in time, allows the researcher(s) to compare data over time (Valcheva, 2023) and longitudinal studies are facilitated.

Disadvantages of secondary research analysis

One of the disadvantages of secondary research analysis is that the data used were collected following the demands and objectives of a distinct research and may not fully match the plan of another research. The analyst(s) are therefore trying to harmonize their research questions with data already collected. In this way, secondary data are mostly useful as extensions of the original research (Sherif, 2018).

Likewise, secondary data collection procedures must be evaluated prior to the employment of the datasets. Of course, there might be cases where errors may have occurred during the data collection process and the researcher of the secondary analysis cannot be aware of. The lack of control of data collection is another disadvantage of the method. Especially regarding qualitative data where the context is also very important, the investigator of the secondary examination will be missing this information when they try to interpret the data.

Furthermore, research data is restricted by the time of its collection, it may not be used after a long time since situations may have changed and the need to collect new data may not be overcome.

Also, since secondary data may be found freely, the investigator is not its owner and even more investigators can have access to it, therefore it is not exclusive minimizing the originality of a research (Valcheva, 2023).

Content analysis

Content analysis is a social scientific methodology in which the goal is to identify patterns, themes, and meanings in recorded communication and decipher messages contained in texts of various formats including news media, policy documents, and even video or novels (Luo, 2019; Baxter, 2020). It mainly refers to written texts such as books, newspapers, magazines, parliamentary speeches, press releases, slogans, etc., however, it is not limited to written texts only. Many analyses involve radio broadcasts, spoken word, interviews, photographs or images such as analyses of motion pictures, newsreels, cartoons, television programs, illustrated magazines, etc. It is characterized by systematicity and great rigor. The fields of knowledge to which the method of content analysis is applied are quite extensive. It can be applied in sociology, political science, where it is popular, also in psychology, history, anthropology, etc. Initially, content analysis was empirical in nature and the aim was to turn the content under analysis into measurable data, which could be summarized and ultimately compared with each other in order to draw generalizable conclusions.

In content analysis words, themes, and concepts within the texts are broken down at different levels to smaller units, such as sentences, phrases or even words (Baxter, 2020). As Krippendorff (2018, p. 83) states “unitizing is the systematic distinguishing of segments of text-images, voices, and other observables-that are of interest to an analysis”. All units are determined according to the content, objectives, hypotheses, tasks in a particular study and they must correspond to specific scientific concepts that carry a research project. For example, a word, as a unit, is examined in terms of functionality within the text (active or passive, if it refers to the past, present or future, and so on).

Afterwards, they are “coded”, meaning that ideas are identified, categorized and labeled, which is a step that bridges the unitized texts and what people see in them (Krippendorff, 2018) and then the results are analyzed aiming at making the results comprehensible to the readers. At this point, a code scheme that provides a clear and consistent structured coding framework is created and registered in a codebook. In this way the researchers are enabled to identify patterns and themes in the data and furthermore the codebook serves as a reference throughout the study. The researchers calculate the number and frequency of occurrence of all expressions and words, and they draw appropriate conclusions. Content analysis is a systematic technique of compressing and transforming the multitude of words in a text into fewer content categories, which are based on specific coding rules, allowing researchers to study large volumes of data through a systematic methodology.

Content analysis can be both quantitative, where the focus is set to counting, measuring and presenting frequencies and qualitative, where the focus is set to interpreting and understanding (Luo, 2019) and also mixed. Furthermore, it allows the comparative analysis between different categories of meanings, through the statistical processing of the coded form of the information under investigation. Content analysis procedures demands attention while making the segments that will be analyzed, the selection of the units, of the code scheme and also in order to be reliable different researchers have to perform the processes themselves (Coe, & Scacco, 2017).

Quantitative content analysis

Quantitative data is data that can be expressed as a number or can be quantified, and quantitative content analysis is a research method where textual, visual, or aural material is categorized through a systematic coding and quantification of content process in order to be analyzed and conclusions to be drawn (Coe, & Scacco, 2017). It aims to develop numerical data that will be studied statistically.

Quantitative content analysis, as with other quantitative approaches, starts by identifying the relevant concepts in response to the research question. Sequentially, the material is selected which is the sampling process. At this point the coding unit (words, phrases, images, etc.) is also decided. Afterwards, a coding scheme should be developed to assign coding units to particular categories or concepts which are then registered in a codebook.

Qualitative content analysis

Qualitative data is the kind of information that cannot be expressed as a number and therefore cannot be measured. It consists of words, pictures, observations, and symbols (Valcheva, 2023). Qualitative content analysis is employed when exploration of complex phenomena, such as attitudes, beliefs, and behaviors is demanded.

In qualitative content analysis, which is a method often used in social science research, there exists a dualism that suggests the adoption of “inductive” or “deductive” approaches of reasoning in the process of qualitative data analysis (Armat, Assarroudi, Rad, Sharifi, & Heydari, 2018). Inductive content analysis involves a process of collecting and analyzing data without preconceived categories or theories. In that way the researcher’s analysis while identifying emerging patterns, themes, and concepts emerges from the data through careful examination and constant comparison. However, qualitative content analysis does not exclude deductive reasoning which is an approach that begins with a theoretical framework and tests hypotheses (Delve, & Limpaecher, 2023). As Streefkerk (2023) states “the main difference between inductive and deductive reasoning is that inductive reasoning aims at developing a theory while deductive reasoning aims at testing an existing theory”.

Nowadays, there is commercial data analysis software that allows researchers to ask complex questions and create codes, categories and themes (such as NVivo, ATLAS.ti and MAXQDA). They also have visualization tools to present even in a more vivid way the results.

Real world examples

In this part, real world examples that researchers have faced will be presented to help in comprehending the way issues were dealt while providing insight thoughts.

Secondary analysis of qualitative data: a valuable method for exploring sensitive issues with an elusive population? (Long-Sutehall, Sque, & Addington-Hall, 2011):

In this study, a secondary analysis of 28 transcripts, sorted from two primary datasets was carried out to research on what the diagnosis of death means to the family members being approached and requested to consider donating the organs of a family member. Interview transcripts of two primary investigations that provided coherent details were subjected to secondary analysis which was selected as a form of research since it addressed a sensitive area of research and the population under study is considered elusive (Fielding, 2004).

A qualitative approach to intersectional microaggressions: Understanding influences of race, ethnicity, gender, sexuality, and religion. (Nadal, Davidoff, Davis, Wong, Marshall, & McKenzie, 2015):

In this study, the authors employed the secondary analysis of qualitative data, to comprehend concepts that would not have been researched through a singular qualitative analysis. The researchers analyzed data from 6 previous qualitative studies to explore whether qualitative analysis can be used effectively to examine intersectional microaggressions and to examine whether people with multiple identities could identify intersectional microaggressions in their everyday lives.

Concluding remarks

Much information exists in documented form and the era we are going through is a digital era where data, datasets and big data are easily found and accessible, thus learning the significance and the way of using secondary data in research can be beneficial for novice and expert investigators.

Depending on the research and the population under study secondary analysis may provide newer insights through another prism. It is chosen by the researcher(s) for various reasons, such as resources constraints, availability, ethics, historical reasons, and the needs of the research itself. Longitudinal and international analysis of information are possible due to secondary data analysis. Of course, there are shortcomings and limitations while using secondary data, such as the time of primary research, the lack of control on the primary data and the fact that the researcher(s) would need to familiarize themselves with the data.

The methods involved in secondary research analysis vary according to the primary data. Content analysis, quantitative and qualitative, is commonly employed in cases of secondary data analysis.

Exercises

If you want to study the influence of gender and age in courses selected by the students at a university, one way would be to ask all the students, or a credible sample of them. How could you achieve the same, or even better, result finding secondary research data? Where should you address to get this information? Will the anonymity of the students be protected?
By visiting the https://www.bfi.org.uk/industry-data-insights and gathering any kind of information needed, organize a research based on the secondary data that can be retrieved through the website of the British Film Institute.
Find a speech by a politician on the unemployment problem and locate the words that identify the problem within a context. Then, try to analyze the meanings of possible word relationships to understand the targets of the political party.

Self-assessment questions/quizzes

In this part, short self-assessment questions/quizzes will aid in the comprehension of the issues presented in the chapter, challenging the readers to check on their understanding capabilities.

1) The term “secondary analysis” refers to the technique of:

a) Data analysis in two different ways
b) Analysing existing data that has been collected by another person or organization
c) Working on a project as a secondary occupation

2) Which of the following is not an advantage of secondary analysis?

a) Only one researcher/research team has access to the data
b) It provides an opportunity for longitudinal analysis
c) It allows the researcher(s) to study social trends over time

3) Which of the following is not a disadvantage of using secondary analysis?

a) Data is restricted by the time of its collection
b) Secondary data may save resources
c) The researcher has no control over the quality of the data

4) Which of the following is an example of secondary data?

a) Information gathered by a clothing store about customer satisfaction
b) Census information used to determine where a new hospital should be built
c) Information gathered by a band about what its name should be

5) Secondary research relies upon ______ data.

a) tailor-made
b) primary
c) already existing

6) Secondary data can ______ research questions.

a) answer
b) generate
c) both answer and generate

7) Content analysis means

a) analysing the contents page of a textbook
b) performing a quantitative technique for methodically describing written, oral, or visual communication
c) being involved in the process of obtaining meaning from raw data

8) Which of the following could be subjected to content analysis?

a) Interview transcripts
b) Press releases
c) Photographs
d) All of the above

9) Which of the following are considered as units in content analysis?

a) Sentences,
b) Phrases
c) Words
d) All of the above

10) Which of the following statements is true of content analysis?

a) It generates rich written descriptions of archival records.
b) It may be applied to both verbal and nonverbal materials.
c) It seldom involves quantification.

11) Which research method is a bottom-up approach to research?

a) Deductive method
b) Inductive method
c) Exploratory method

12) Identify which of the following characteristics apply to qualitative research:

a) Aims to discover empirical evidence – quantitative
b) Uses fixed, structured designs – quantitative
c) Findings presented as detailed descriptions often with direct quotes – qualitative
d) Semi or unstructured data collection tools – qualitative

References

Armat, M. R., Assarroudi, A., Rad, M., Sharifi, H., & Heydari, A. (2018). Inductive and deductive: Ambiguous labels in qualitative content analysis. The Qualitative Report, 23(1), 219-221.

Atlas.ti (2023). Inductive reasoning and analysis. https://atlasti.com/guides/qualitative-research-guide-part-2/inductive-reasoning

Baxter, J. (2020). Content Analysis. In A. Kobayashi (Ed) International Encyclopedia of Human Geography (Second Edition), Elsevier, pp. 391-396, https://doi.org/10.1016/B978-0-08-102295-5.10805-4.

Bhandari, P. (January 20, 2022). What Is Deductive Reasoning? | Explanation & Examples. https://www.scribbr.com/methodology/deductive-reasoning/

Bhandari, P. (June 22, 2023). What Is Qualitative Research? | Methods & Examples https://www.scribbr.com/methodology/qualitative-research/

Bhat, A. (2023). Secondary Research: Definition, Methods and Examples. https://www.questionpro.com/blog/secondary-research/

Coe, K., & Scacco, J. M. (2017). Content analysis, quantitative. The international encyclopedia of communication research methods, 1-11.

Columbia, (2023). Content Analysis. https://www.publichealth.columbia.edu/research/population-health-methods/content-analysis

Delve, Ho, L., & Limpaecher, A. (2020a, August 31). How to Do Thematic Analysis. Essential Guide to Coding Qualitative Data. https://delvetool.com/blog/thematicanalysis

Delve, Ho, L., & Limpaecher, A. (2023c, March 10). Inductive Content Analysis & Deductive Content Analysis in Qualitative Research https://delvetool.com/blog/inductive-content-analysis-deductive-content-analysis

Fielding, N. (2004). Getting the most from archived qualitative data: Epistemological, practical and professional obstacles. International journal of social research methodology, 7(1), 97-104.

Heaton, J. (2008). Secondary analysis of qualitative data: An overview. Historical Social Research/Historische Sozialforschung, 33-45.

Hillier, W. (November 30, 2022). What Is Secondary Data? A Complete Guide. CareerFoundry. https://careerfoundry.com/en/blog/data-analytics/what-is-secondary-data/

George, T. (June 22, 2023). What is Secondary Research? | Definition, Types, & Examples. https://www.scribbr.com/methodology/secondary-research/

Kelder, J. A. (2005, January). Using someone else’s data: problems, pragmatics and provisions. In Forum Qualitative Sozialforschung/Forum: Qualitative Social Research (Vol. 6, No. 1).

Krippendorff, K. (2018). Content analysis: An introduction to its methodology. Thousand Oaks: Sage publications.

Long-Sutehall, T., Sque, M., & Addington-Hall, J. (2011). Secondary analysis of qualitative data: a valuable method for exploring sensitive issues with an elusive population?. Journal of Research in Nursing, 16(4), 335-344.

Luo, A. (July 18, 2019). Content Analysis | Guide, Methods & Examples. https://www.scribbr.com/methodology/content-analysis/

Nadal, K. L., Davidoff, K. C., Davis, L. S., Wong, Y., Marshall, D., & McKenzie, V. (2015). A qualitative approach to intersectional microaggressions: Understanding influences of race, ethnicity, gender, sexuality, and religion. Qualitative psychology, 2(2), 147.

Rush, K. L., Watts, W. E., & Stanbury, J. (2011). Mobility adaptations of older adults: A secondary analysis. Clinical Nursing Research, 20(1), 81-100.

Sherif, V. (2018, March). Evaluating preexisting qualitative research data for secondary analysis. In Forum qualitative sozialforschung/forum: Qualitative social research 19(2).

Sreekumar, D. (March 23, 2023). What is quantitative research? Definition, methods, types, and examples https://researcher.life/blog/article/what-is-quantitative-research-types-and-examples/

Streefkerk, R. (2023, June 22). Inductive vs. Deductive Research Approach | Steps & Examples. Scribbr. https://www.scribbr.com/methodology/inductive-deductive-reasoning/

Valcheva, S. (2023). Secondary Data: Advantages, Disadvantages, Sources, Types. https://www.intellspot.com/secondary-data/

von Benzon, N. (2019). Informed consent and secondary data: Reflections on the use of mothers’ blogs in social media research. Area, 51(1), 182-189.

Wagh, S. (April, 2023). Public Health Research Guide: Primary & Secondary Data Definitions. https://researchguides.ben.edu/c.php?g=282050&p=4036581

Mixed methods research designs

Learning outcomes

In the following chapter the mixed methods research designs will be deployed. As derived by the name, mixed methods combine elements both of quantitative and qualitative research within or across one or more stages of the research process to deliver a complete outcome when a thorough understanding of a phenomenon demands the examination of different kind of data (Alivernini, 2012; Almeida, 2018). Their power is their ability to deal with diversity and divergence (Schoonenboom, & Johnson, 2017).

Upon completion of the study of this chapter the students will be able to define mixed methods research as research tools that combine the qualitative and quantitative approaches and realize their employment (Tashakkori & Teddlie, 2008). By comprehending and identifying the basic features of the various methods that fall under this category, will be able to plan and develop the steps involved in a mixed methods research approach within the different phases of the research process leading to successful data analysis. They will be able to choose appropriately among various mixed methods design depending on the situations and furthermore they may describe the advantages and disadvantages of the different methods. combining quantitative and qualitative data in mixed methods evaluation.

Introduction

Initially, the main reasons that lead to the implementation of a mixed research method ought to be clarified to the investigator and will be discussed in this chapter. Subsequently, the characteristics of the various models along with the data analysis techniques involved in the mixed methods research designs will be examined.

Starting out, the need to answer different questions which require different types of data in a single study is the reason that demands to follow a mixed research approach. It is common, while executing a research, more than one perspective to be required to better understand a phenomenon. Sometimes, quantitative data need to be confirmed and/or explained with personal experiences of the participants that derive from qualitative analysis. In those and more other cases the mixed methods research design by providing more evidence may deliver safer conclusions, thus granting increased confidence in the findings (Creswell, & Plano Clark, 2011).

Prior to proceeding in the design of the methodology that will be followed, the researcher should be able to answer questions that will aid in the procedure and will be discussed in this chapter. Such questions may be “What is the order in the collection of quantitative and qualitative data?”, “How will the analysis of the data will be performed?”, “At what point will the data be merged?” (Teddlie, & Tashakkori, 2009).

Mixed research design – basic theory

As previously pointed out, mixed methods research, that began between 1985 and 1990, involves the employment of both quantitative and qualitative data to answer the research questions linking the two databases in a creative way (Creswell, & Hirose, 2019). The idea for mixing both kinds of data within one study is rooted in the fact that quantitative and qualitative methodologies collect data with different methods and they arise from different ontological and epistemological positions (Bryman, 2007). Since they provide different types of information neither is sufficient, by itself, to capture the details of a situation, thus mixed research designs may overcome this limitation. The adoption of mixed methods design is growing in the academic society (Almeida, 2018).

The methodology of mixed research design was driven by the developments in procedures and designs (such as quantitative, experimental, qualitative) and philosophies (such as pragmatism, critical realism, dialectic pluralism) which “advanced the key idea that researchers bring to their mixed methods study core assumptions or beliefs that shape the types of procedures used in their studies” (Creswell, & Hirose, 2019, p.2).

Therefore, the three key features are: the collection of quantitative and qualitative data, the analysis of these two datasets and finally the mixing of the two datasets in a meaningful way and the development of an overall interpretation while framing the study within larger philosophical assumptions, beliefs or orientations (Clark, Creswell, Green, & Shope, 2008; Creswell, & Hirose, 2019).

As Guest (2012, p. 146) argues the common descriptive dimensions of mixed research design are the following:

“Timing of the interface between data sets (e.g., simultaneous or successive)
Purpose of the interface between data sets (e.g., inform, explain, triangulate)
Theoretical orientation (e.g., inductive vs. deductive, interpretive, feminist)
Purpose of the research (e.g., advocacy, applied, theoretical)
Number of points of interface or degree of integration (e.g., fully or partially integrated/mixed, single, or multistrand)
Relative importance of qualitative data and quantitative data (weighting)”

Furthermore, the reasons that induce the use of these methods may be found in the necessity to meet several research parameters (George, 2021).

One of these necessities is the generalizability of the findings. As qualitative research usually hold smaller sample sizes is not feasible to generalize their findings. By applying mixed methods research, which involves the use of quantitative data, this comparative weakness can be mitigated, especially if the sample consists of a large number of items.

Another need that is compensated by the employment of mixed methods is the contextualization. When mixing methods, the researcher is able to set the findings in the correct context and therefore provide thorough insight to the conclusions. This is more obvious when qualitative data enlighten the quantitative data.

Credibility is achieved when using different methods to collect data on the same subject. By converging different types of data, the validity of the conclusions is strengthened, and this process is called triangulation.

Steps and criteria of a mixed research methods design

When a mixed research methods is to be implemented, the first stage should involve literature review on the existing methods that will reveal the steps followed in other studies, as well as their advantages and disadvantages (Almeida, 2018). Afterwards, the formulation of the research questions will follow and according to Creswell & Plano Clark (2011) there are two approaches on the issue. The first one suggests that a single question is the prime one and is expanded to quantitative and qualitative sub questions. On the other hand, the second suggests that separate quantitative and qualitative questions are set, followed by a question regarding the nature of their integration (e.g., are the results of both researches integrated?) (Subedi, 2016).

Initially, the researcher must clarify whether a mixed methodology is appropriate, the reasons for using it and then place his/her research in the context of a mixed methodology by justifying his/her choice, always paying attention to the validity issues (Schoonenboom, & Johnson, 2017). Sequentially, the formulation of a diagram will distinguish the methods, the chronological order and the evolution of the research. By being able to answer to the question: “In what way do I want to use the results from one data type with the other?” the choice of the right type of method followed will be determined.

The basic approach commences by deciding whether the research will be performed sequential and/or concurrent. Sequential means that there will be two stages when collecting the data, while concurrent means that all data will be collected simultaneously. The decision regarding the timing of collecting the qualitative and quantitative components is very important. As Guest (2013) indicates timing holds two factors, simultaneity which indicates if the studies can be performed at the same time or only after completion of the other, and dependence which determines if the process of one study depends on the results obtained in another study. However, Schoonenboom and Johnson (2017) argue that simultaneity and dependence are two separate dimensions.

Another very important factor in mixed research design is the integration of the quantitative and qualitative results which might be very challenging (Burt, 2015). The first step after collecting the data is the evaluation of each research scientific impact autonomously.

At the end, while writing the manuscript, the reader should be informed by the title but also within the text that mixed methodology is implemented.

Advantages of mixed methods research design

The main advantage of employing mixed methods research is that the researcher by using both qualitative and quantitative research methods for the issue under study is able to use the strength of one method (quantitative: numbers, trends, generalizability & qualitative: words, context, meaning) to compensate for the weakness of the other and therefore comprehend thorough the phenomenon. The examination of the same phenomenon under multiple perspectives, is more likely to provide more complete understanding (Clark, et al., 2008).

Since mixed research methods are not limited to research method, they allow the researcher to handle a broad range of research questions and by collecting different kind of data stronger evidence is provided.

Disadvantages of mixed methods research design

Mixed methods research can be time-consuming and need more resources since it requires collecting and analysing two types of data. If the researcher is one and not a team, he/she may not be familiar with analysing both quantitative and qualitative data and thus may find difficulties in interpreting the results and drawing conclusions.

Types of mixed methods research design

In the following paragraphs the main types of mixed methods research designs will be shortly deployed. Of course, the main point of the research is the first concern and by deciding that, the component that corresponds to it is specified as the “core” component, while the other is known as the “supplemental” component; however, research where both the qualitative and quantitative component are of equal value and weight is also possible (Schoonenboom, & Johnson, 2017). In all cases methodological issues have to be considered prior to their implementation and advantages and disadvantages may be encountered in each one of them.

Convergent parallel design

In a convergent parallel design, the quantitative and qualitative elements are collected in the same phase of the research process and the methods weigh equally. The analysis of the two components is performed independently and the results are interpreted together, drawing the conclusions (Creswell & Plano-Clark, 2011). A visual model of the research is presented in Figure 1. This design may be chosen if there is a need to collect both types of data in one visit to the field and/or if both types of data have equal value for understanding the research issue. It is also used when there is a necessity of validating and/or illustrating qualitative findings with quantitative results.

Figure 1: The convergent parallel design (source: http://pubs.sciepub.com/education/4/7/10/)

Explanatory sequential design

In explanatory sequential design the purpose is to explain quantitative results with qualitative data. Therefore, the priority is given to the quantitative data which is collected in the first phase of the research and afterwards the qualitative data is collected as a follow-up to the quantitative results. Then, the researcher decides how and when to connect the quantitative and qualitative phases and the way the integration of the results of both phases is performed to answer the research questions (Figure 2). It is a method that is highly popular among researchers (Ivankova, Creswell, & Stick, 2006).

Figure 2: The explanatory sequential design (source: http://pubs.sciepub.com/education/4/7/10/)

Exploratory sequential design

In exploratory sequential design the purpose is to investigate qualitatively the questions to be asked, the variables to be measured and the people to be interviewed. The priority is set to the qualitative approach, therefore qualitative data are collected initially, and quantitative data are used to explain the original qualitative findings. The diagram of the method is presented in Figure 3. The purpose of first gathering qualitative data is to explore a phenomenon under study and afterwards by collecting quantitative data to explain any relationships retrieved in the qualitative data (Creswell & Plano-Clark, 2011). The two phases are connected by using the qualitative results in shaping the research questions and variables of the quantitative research. This method may be implemented in the case that theories or hypotheses are not known and need to be explored and/or to assess whether qualitative themes can be generalized to a population.

Figure 3: The exploratory sequential design (source: http://pubs.sciepub.com/education/4/7/10/)

Embedded design

In embedded design the purpose is to add qualitative or quantitative data, before, during or after the procedure. The priority is set in the main data collection process which may be either the quantitative or the qualitative. Data is collected either simultaneously or sequential and the use of the secondary/ supplemental data format supports and improves the one of the primary one (Wolf, Stidham, & Ross, 2015). The diagram of the embedded design is presented in Figure 4.

Figure 4: The embedded design (source: http://pubs.sciepub.com/education/4/7/10/)

Transformative design

As Creswell and Plano Clark (2011) state the purpose of the transformative mixed methods design is to encase the design within a transformative framework by using one of the convergent, explanatory, exploratory or embedded designs. It employs qualitative and quantitative methods within a single phase, sequential phases, or concurrently and is designed for large-scale investigations that involve equally emphasized qualitative and quantitative methodological frameworks (Flynn, & Waterhouse, 2021). It is very similar to “explanatory design”, besides the fact that everything can be transformed according to the theoretical background. The diagram of the embedded design is presented in Figure 5.

Figure 5: The transformative mixed methods design (source: http://pubs.sciepub.com/education/4/7/10/)

Multiphase design

In Creswell and Plano Clark’s (2011) typology, the multiphase design is a complex process that builds on the previously mentioned designs and by using it becomes necessary to consider several dimensions of analysis but also how they are dependent on each other (Almeida, 2018). It is considered the one when more than two research phases (series of phases) that may be sequential or concurrent are combined within one study addressing an overall program objective. This design is often used in program evaluation, where quantitative and qualitative approaches are used over time to support, develop, adapt, and evaluate individual aspects of a program. The diagram of the embedded design is presented in Figure 6.

Figure 6: The multiphase mixed methods design (source: http://pubs.sciepub.com/education/4/7/10/)

Real world examples

In this part, real world examples that researchers have faced will be presented to help in comprehending the way issues were dealt while providing insight thoughts.

Digital Storytelling in Sports Narrations: Employing Audiovisual Tools in Sport Journalism Higher Education Course (Matsiola, Spiliopoulos, & Tsigilis, 2022):

An embedded mixed research design was implemented in an attempt to reach to safer conclusions regarding the use of digital storytelling in sports narrations. Specifically, the employment of audiovisual tools in sport journalism higher education course was investigated via a short questionnaire (quantitative) and two focus group discussions (qualitative) among students that attended the course of Sports Journalism in the Department of Journalism and Mass Communications at Aristotle University of Thessaloniki, Greece (Matsiola, Spiliopoulos, & Tsigilis, 2022). In this case, the researchers’ priority was focused on the data that derived from the qualitative method (QUAL) from the semi-structured interviews of the focus group since the interpretation of the students’ choices and actions along with their lived experience were the main issues. However, the quantitative data (QUAN) that was gathered by all students that were enrolled in the course was used supportively for the comprehension of the results of the qualitative data. The dual analysis that was performed provided valuable conclusions on the subject under investigation.

Mixed Methods- Theory and Practice. Sequential, Explanatory Approach (Bowen, Rose, & Pilkington, 2017):

In this study the sequential, explanatory, mixed methodology was implemented to research the emotional intelligence in higher education. A sample of 533 academics was used and the findings from interviews helped in explaining the findings from quantitative data regarding how academics feel. The quantitative phase was followed by the qualitative phase where the personal experience was stated and the qualitative findings were used to contextualise the quantitative data (Bowen, Rose, & Pilkington, 2017). The whole process was implemented in 3 phases. During the first phase quantitative data was collected via online questionnaire, which was then analysed and evaluated; during the second phase semi structured interviews, combining formal and informal features, were undertaken and the qualitative data was analysed and evaluated and final during the third phase triangulation of data was performed where qualitative data was used to contextualise the quantitative findings.

Mixed methods and survey research in family medicine and community health (Creswell, & Hirose, 2019):

In this study, quantitative research was effectively combined with qualitative data to form a mixed methods study for primary care. It provides an opportunity via an applied discussion to show how a survey (or questionnaires) fits into mixed methods commencing with an overview of the key principles both in survey and in mixed methods research. Sequentially, a specific example (Sonnenberg, Pritchard-Wiart, Hodgson, Yu, & King, 2017) in conducting a mixed methods study using survey research employing six steps is presented. The six steps are the following: 1. Expression of the reasons for mixed methods study, 2. Specify the quantitative and qualitative databases, 3. Identify the mixed methods design needed, 4. Analyse and report the results of the quantitative and qualitative databases, 5. Present the integration of the findings and 6. Explain the value of using mixed methods.

Concluding remarks

Social Science Research Methodology displays a multitude of approaches, seemingly justified by the existence of different philosophical considerations and assumptions regarding social reality. As described in this chapter, the mixed research design aims at collecting, analyzing and extracting conclusions from combined qualitative and quantitative data in one study. The qualitative data are used to investigate and understand social phenomena, such as attitudes, representations, or perceptions in depth, while the quantitative data deriving from statistical analyses may be generalised.

There are various types of mixed research methods depending on the issues under investigation the purposes of the study and the priorities given either by the research and/or the researchers to the data collection methods and the timing of the investigations. It is not easy to determine how many types of research there are because different researchers and educators use different criteria (professional, personal, political) to conduct research. In general, typologies aid in designing the research, especially novice researchers and they also establish a common language and structure for the field (Guest, 2012).

Of course, as in all research, ethical considerations must be considered prior to dealing with the technical issues. Finally, the conclusions should derive from the integration of the findings of the methods involved and not from autonomous processes.

Exercises

Design of research that would connect the students’ perception of the university environment in regard to their grades. What kind of data should be collected and in what order to deliver safe results?
Design of research that would connect the job satisfaction of employees with the sales over the years. What kind of data should be collected and in what order to deliver safe results?
Design of research that will evaluate any possible social media addiction by the members of Generation Z. What kind of data should be collected and in what order to deliver safe results?
Design of research on the aspects regarding the smoking habits of pupils in relationship with extracurricular activities. What kind of data should be collected and in what order to deliver safe results? Develop the instruments needed (e.g., questionnaire, interview questions, observation situations, etc.)

Self-assessment questions/quizzes

In this part, short self-assessment questions/quizzes will aid in the comprehension of the issues presented in the chapter, challenging the readers to check on their understanding capabilities.

In exploratory sequential design, the priority is set to:
Quantitative data collection
Qualitative data collection
Simultaneously to quantitative and qualitative data collection
The method that sets priority to quantitative data collection is called:
Exploratory sequential design
Explanatory sequential design
Convergent parallel design
The process of using multiple datasets, methods, theories, and/or investigators is called:
Contextualization
Credibility
Generalizability
Triangulation
One of the advantages of mixed research methods:
It can be time-consuming
It needs more resources
It may use the strength of one method to compensate for the weakness of the other
In a mixed methods research study when the weight is given to quantitative data, the qualitative data is known as the:
Complementary
Core
Primary
Generalizability of the findings is considered the process of:
Setting the findings in the correct context
Being able to say that the findings may be induced for large population
Using different methods to collect data on the same subject
By combining two or more research methods in a single study, researchers can ensure that:
they will collect some of the applicable data
they will cover every aspect and perspective
they will make a mistake
Mixed methods research can be defined as:
Using either qualitative or quantitative methods in a single study.
Collecting quantitative data and reporting it qualitatively.
Collecting data from different research groups at the same time.
Using both qualitative and quantitative methods in a single study.
In mixed methods research:
‘how’ and ‘what’ questions are best answered by qualitative data
‘why’ questions can be answered by quantitative data
the sequence in which you ask such questions is crucial
In mixed methods research design:
Data should be collected concurrently
Data should be collected sequentially
It has to involve quantitative and qualitative data

References

Alivernini, F. (2012). Mixed methods research on learning. In N. M. Seel (Eds.), Encyclopedia of the Science of Learning (pp. 1-4). New York: Springer.

Almeida, F. (2018). Strategies to perform a mixed methods study. European Journal of Education Studies, 5(1).

Bhandari, P. (June 22, 2023). What Is Qualitative Research? | Methods & Examples https://www.scribbr.com/methodology/qualitative-research/

Bowen, P. W., Rose, R., & Pilkington, A. (2017). Mixed methods-theory and practice. Sequential, explanatory approach. International Journal of Quantitative and Qualitative Research Methods, 5(2), 10.

Bryman, A. (2007). Barriers to integrating quantitative and qualitative research. Journal of mixed methods research, 1(1), 8-22.

Burt, J. (2015). Following the mixed methods trail: some travel advice. British Journal of General Practice, 65(634), 264-265.

Clark, V. L. P., Creswell, J. W., Green, D. O. N., & Shope, R. J. (2008). Mixing quantitative and qualitative approaches. In S. N. Hesse-Biber, & P. Leavy (Eds) Handbook of emergent methods, (pp. 363-387). The Guilford Press.

Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage

Creswell, J. W., & Hirose, M. (2019). Mixed methods and survey research in family medicine and community health. Family medicine and community health, 7(2).

Flynn, S. V., & Waterhouse, S. (2021). Mixed Methods Research: Transformative and Multiphase Designs. In Flynn, S. V. (Ed), Research Design for the Behavioral Sciences An Applied Approach. Springer Publishing, Connect.

George, T. (August 13, 2021). Mixed Methods Research | Definition, Guide & Examples https://www.scribbr.com/methodology/mixed-methods-research/

Guest, G. (2013). Describing mixed methods research: An alternative to typologies. Journal of mixed methods research, 7(2), 141-151.

Ivankova, N. V., Creswell, J. W., & Stick, S. L. (2006). Using mixed-methods sequential explanatory design: From theory to practice. Field methods, 18(1), 3-20.

Matsiola, M., Spiliopoulos, P., & Tsigilis, N. (2022). Digital Storytelling in Sports Narrations: Employing Audiovisual Tools in Sport Journalism Higher Education Course. Education Sciences, 12(1), 51.

Schoonenboom, J., & Johnson, R. B. (2017). How to construct a mixed methods research design. Kolner Zeitschrift fur Soziologie und Sozialpsychologie, 69(Suppl 2), 107.

Sonnenberg, L. K., Pritchard-Wiart, L., Hodgson, C. S., Yu, Y., & King, S. (2017). Assessment of resident physicians’ communicator and collaborator competencies by interprofessional clinicians: a mixed-methods study. Teaching and Learning in Medicine, 29(4), 392-401.

Sreekumar, D. (March 23, 2023). What is quantitative research? Definition, methods, types, and examples https://researcher.life/blog/article/what-is-quantitative-research-types-and-examples/

Subedi, D. (2016). Explanatory Sequential Mixed Method Design as the Third Research Community of Knowledge Claim. American Journal of Educational Research, 4(7), 570-577.

Tashakkori, A., & Teddlie, C. (2008). Introduction to mixed method and mixed model studies in the social and behavioral science. In V.L. Plano-Clark & J. W. Creswell (Eds.), The mixed methods reader, (pp. 7-26). Sage.

Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Sage.

Wolf, L., Stidham, A. W., & Ross, R. (2015). Predictors of stress and coping strategies of US accelerated vs. generic baccalaureate nursing students: An embedded mixed methods study. Nurse education today, 35(1), 201-205.