The previous chapters have focused on how social workers use social science research methods to understand the world. But what about social workers who aren’t researchers? Social workers in practice may not have time or interest in conducting experiments or focus groups for the purposes of scholarly publication. While the preceding chapters should provide you the information you need to understand the research conducted by professional researchers, social workers in practice still must use research skills to help their clients. This chapter will review three approaches to research that social workers will use while in practice.

Chapter Outline

• 11.1 Evaluation research
• 11.2 Single-subjects design
• 11.3 Action research

Content Advisory

This chapter discusses or mentions the following topics:  substance abuse and intimate partner violence.

As you may recall from the definition provided in Chapter 1, evaluation research is research conducted to assess the effects of specific programs or policies. Evaluation research is often used when some form of policy intervention is planned, such as welfare reform or school curriculum change. The focus on interventions and social problems makes it natural fit for social work researchers. It might be used to assess the extent to which intervention is necessary by attempting to define and diagnose social problems in social workers’ service areas, and it might also be used to understand whether their agencies’ interventions have had their intended consequences.  Evaluation research is becoming more and more necessary for agencies to secure and maintain funding for their programs.  The main types of evaluation research are needs assessments, outcomes assessments, process assessments, and efficiency analyses such as cost-benefits or cost-effectiveness  analyses.  We will discuss two types in this section:  outcomes assessments and process assessments.

Outcomes Assessments

An outcomes assessment is an evaluation designed to discover if a program achieved its intended outcomes. Much like other types of research, it comes with its own peculiar terminology.  Inputs are the resources needed for the program to operate. These include physical location, any equipment needed, staff (and experience/knowledge of those staff), monetary funding, and most importantly, the clients. Program administrators pull together the necessary resources to run an intervention or program. The program is the intervention your clients receive—perhaps giving them access to housing vouchers or enrolling them in a smoking cessation class. The outputs of programs are tangible results of the program process. Outputs in a program might include the number of clients served, staff members trained to implement the intervention, mobility assistance devices distributed, nicotine patches distributed, etc. By contrast, outcomes speak to the purpose of the program itself.  Outcomes are the observed changes, whether intended or unintended, that occurred due to the program or intervention. By looking at each of these domains, evaluation researchers can obtain a comprehensive view of the program.

Let’s run through an example from the social work practice of the wife of Matt DeCarlo who wrote the source material for much of this textbook. She runs an after-school bicycling club called Pedal Up for children with mental health issues. She has a lot of inputs in her program. First, there are the children who enroll, the volunteer and paid staff members who supervise the kids (and their knowledge about bicycles and children’s mental health), the bicycles and equipment that all clients and staff use, the community center room they use as a home base, the paths of the city where they ride their bikes, and the public and private grants they use to fund the program. Next, the program itself is a twice weekly after-school program in which children learn about bicycle maintenance and bicycle safety for about 30 minutes each day and then spend at least an hour riding around the city on bicycle trails.

In measuring the outputs of this program, she has many options. She would probably include the number of children  participating in the program or the number of bike rides or lessons given. Other outputs might include the number of miles logged by the children over the school year, the number of bicycle helmets or spare tires distributed, etc. Finally, the outcomes of the programs might include each child’s mental health symptoms or behavioral issues at school.

Process Assessments

Outcomes assessments are performed at the end of a program or at specific points during the grant reporting process. What if a social worker wants to assess earlier on in the process if the program is on target to achieve its outcomes? In that case a process assessment is recommended, which evaluates a program in its earlier stages. Faulkner and Faulkner (2016) describe three main goals for conducting a process evaluation.

The first is program description, in which the researcher simply tries to understand how the program looks like in everyday life for clients and staff members. In our Pedal Up example, assessing program description might involve measuring in the first few weeks the hours children spent riding their bikes, the number of children and staff in attendance, etc. This data will provide those in charge of the program an idea of how their ideas have translated from the grant proposal to the real world. If, for example, not enough children are showing up or if children are only able to ride their bikes for ten minutes each day, it may indicate that something is wrong.

Another important goal of process assessment is program monitoring. If you have some social work practice experience already, it’s likely you’ve encountered program monitoring. Agency administrators may look at sign-in sheets for groups, hours billed by clinicians, or other metrics to track how services are utilized over time. They may also assess whether clinicians are following the program correctly or if they are deviating from how the program was designed. This can be an issue in program evaluations of specific treatment models, as any differences between what the administrators conceptualized and what the clinicians implemented jeopardize the internal validity of the evaluation. If, in our Pedal Up example, we have a staff member who does not review bike safety each week or does not enforce helmet laws for some students, we could catch that through program monitoring.

The final goal of process assessments is quality assurance. At its most simple level, quality assurance may involve sending out satisfaction questionnaires to clients and staff members. If there are serious issues, it’s better to know them early on in a program so the program can be adapted to meet the needs of clients and staff. It is important to solicit staff feedback in addition to consumer feedback, as they have insight into how the program is working in practice and areas in which they may be falling short of what the program should be. In our example, we could spend some time talking with parents when they pick their children up from the program or hold a staff meeting to provide opportunities for those most involved in the program to provide feedback.

Needs Assessments

A third type of evaluation research is a needs assessment. A needs assessment can be used to demonstrate and document a community or organizational need and should be carried out in a way to better understand the context in which the need arises. Needs assessments focus on gaining a better understanding of a gap within an organization or community and developing a plan to address that gap. They will often precede the development of a program or organization and are often used to justify the necessity of a program or organization to fill a gap. Needs assessments can be general, such as asking members of a community or organization to reflect on the functioning of a community or organization, or they can be specific in which community or organization members are asked to respond to an identified gap within a community or agency.

Needs assessments should respond to the following questions:

• What is the need or gap?
• What data exist about the need or gap?
• What data are needed in order to develop a plan to fill the gap?
• What resources are available to do the needs assessment?
• Who should be involved in the analysis and interpretation of the data?
• How will the information gathered be used and for what purpose?
• How will the results be communicated to community partners?

In order to answer these questions, needs assessments often follow a four-step plan. First, researchers must identify a gap in a community or organization and explore what potential avenues could be pursued to address the gap. This involves deciphering what is known about the needs within the community or organization and determining the scope and direction of the needs assessment. The researcher may partner with key informants within the community to identify the need in order to develop a method of research to conduct the needs assessment.

Second, the researcher will gather data to better understand the need. Data could be collected from key informants within the community, community members themselves, members of an organization, or records from an agency or organization. This involves designing a research study in which a variety of data collection methods could be used, such as surveys, interviews, focus groups, community forums, and secondary analysis of existing data. Once the data are collected, they will be organized and analyzed according to the research questions guiding the needs assessment.

Third, information gathered during data collection will be used to develop a plan of action to fill the needs. This could be the development of a new community agency to address a gap of services within the community or the addition of a new program at an existing agency. This agency or program must be designed according to the results of the needs assessment in order to accurately address the gap.

Finally, the newly developed program or agency must be evaluated to determine if it is filling the gap revealed by the needs assessment. Evaluating the success of the agency or program is essential to the needs assessment process.

Conclusion

Evaluation research is a part of all social workers’ toolkits. It ensures that social work interventions achieve their intended effects. This protects our clients and ensures that money and other resources are not spent on programs that do not work. Evaluation research uses the skills of quantitative and qualitative research to ensure clients receive interventions that have been shown to be successful.

Single-subjects design is distinct from other research methodologies in that, as its name indicates, only one person, group, policy, etc. (i.e., subject) is being studied. Because clinical social work often involves one-on-one practice, single-subjects designs are often used by social workers to ensure that their interventions are having a positive effect. While the results will not be generalizable, they do provide important insight into the effectiveness of clinical interventions. Single-subjects designs involve repeated measurements over time, usually in two stages. But what exactly are we measuring in single-subjects design? The behavior or outcome that we expect will change as a result of the treatment is the dependent variable in a single-subjects research design.  The dependent variable is measured repeatedly during two distinct phases: the baseline stage and the treatment stage.

The baselinestage is the period of time before the intervention starts. During the baseline stage, the social worker is collecting data about the problem the treatment is hoping to address.  For example, a person with substance use issues may binge drink on the weekends but cut down their drinking during the work week.  A social worker might ask the client to record the number of drinks that they consume each day.  By looking at this, we could evaluate the level of alcohol consumption.  For other clients, the social worker might assess other indicators, such as the number of arguments the client had when they were drinking or whether or not the client blacked out as a result of drinking.  Whatever measure is used to assess the targeted problem, that measure is the dependent variable in the single-subjects design.

The baseline stage should last until a pattern emerges in the dependent variable.  This requires at least three different occasions of measurement, but it can often take longer.  During the baseline stage, the social worker looks for one of three types of patterns (Engel & Schutt, 2016).  The dependent variable may (1) be stable over time, (2) exhibit a trend where it is increasing or decreasing over time, or (3) have a cycle of increasing and decreasing that is repeated over time.  Establishing a pattern can prove difficult in clients whose behaviors vary widely.

Ideally, social workers would start measurement for the baseline stage before starting the intervention. This provides the opportunity to determine the baseline pattern.  Unfortunately, that may be impractical or unethical to do in practice if it entails withholding important treatment. In that case, a retrospective baseline can be attained by asking the client to recollect data from before the intervention started.  The drawback to this is the information is likely to be less reliable than a baseline data recorded in real time. The baseline stage is important because with only one subject, there is no control group. Thus, we have to see if our intervention is effective by comparing the client before treatment to and during and after treatment.  In this way, the baseline stage provides the same type of information as a control group — what it looks like when there is not treatment given.

The next stage is the treatmentstage, and it refers to the time in which the treatment is administered by the social worker. Repeated measurements are taken during this stage to see if there is change in the dependent variable during treatment.

One way to analyze the data from a single-subjects design is to visually examine a graphical representation of the results.  An example of a graph from a single-subjects design is shown in Figure 11.1.  The x-axis is time, as measured in months. The y-axis is the measure of the problem we’re trying to change (i.e., the dependent variable).

In Figure 11.1, the y-axis is caseload size. From 1998 to July of 1991, there was no treatment. This is the baseline phase, and we can examine it for a pattern. There is upward trend during the intervention phase, but it looks as if the caseloads began to decrease during the baseline (October 1989).  Once the intervention occurred, there is a clear pattern of a downward trend, indicating the treatment may be associated with the reduction in caseload.

Figure 11.1 Example x-y graph for single subjects design

In single-subjects design, it is possible to  begin a new course of treatment or add a new dimension to an existing treatment.  This is called a a multiple treatment design.  The graphing would continue as before, but with another vertical line representing the second intervention, indicating a new treatment began.

Another option would be to withdraw treatment for a specified time and continue to measure the client, establishing a new baseline. If the client continues to improve after the treatment is withdrawn, then it is likely to have lasting effects. This is called a withdrawal design and is represented as A-B-A or A-B-A-B.

Single-subjects designs, much like evaluation research in the previous section, are used to demonstrate that social work intervention has its intended effects.  Single-subjects designs are most compatible with clinical modalities such as cognitive-behavioral therapy which incorporate as part of treatment client self-monitoring, clinician data analysis, and quantitative measurement. It is routine in this therapeutic model to track, for example, the number of intrusive thoughts experienced between counseling sessions. Moreover, practitioners spend time each session reviewing changes in patterns during the therapeutic process, using it to evaluate and fine-tune the therapeutic approach. Although researchers have used single-subjects designs with less positivist therapies, such as narrative therapy, the single-subjects design is generally used in therapies with more quantifiable outcomes. The results of single-subjects studies are not generalizable to the overall population, but they help ensure that social workers are not providing useless or counterproductive interventions to their clients.

Action research is defined as research that is conducted for the purpose of creating social change. When conducting action research, scholars collaborate with community stakeholders at all stages of the research process with the aim of producing results that will be usable in the community and by scientists. Stakeholders are individuals or groups who have an interest in the outcome of your study. Social workers who engage in action research never just go it alone; instead, they collaborate with the people who are affected by the research at each stage in the process. In action research, stakeholders, particularly those with the least power, are consulted on the purpose of the research project, research questions, design, and reporting of results.

Action research also distinguishes itself from other research in that its purpose is to create change on an individual and community level. Kristin Esterberg (2002) puts it quite eloquently when she says, “At heart, all action researchers are concerned that research not simply contribute to knowledge but also lead to positive changes in people’s lives” (p. 137).  As you might imagine, action research is consistent with the assumptions of the critical paradigm, which focuses on liberating people from oppressive structures. Action research has multiple origins across the globe, including Kurt Lewin’s psychological experiments in the United States and Paulo Friere’s literacy and education programs (Adelman, 1993; Reason, 1994). Over the years, action research has become increasingly popular among scholars who wish for their work to have tangible outcomes that benefit the groups they study.

Action research does not bring any new methodological tricks or terms, but it uses the processes of science in a different way from traditional research. What topics are important to study in a neighborhood or with a target population? A traditional scientist might look at the literature or use their practice wisdom to formulate a research question. An action researcher, on the other hand, would consult with the target population itself to see what they thought were the most pressing issues and best solutions. In this way, action research flips traditional research on its head. Scientists are more like consultants who provide the tools and resources necessary for a target population to achieve their goals and address social problems.

According to Healy (2001), the assumptions of participatory-action research are that (a) oppression is caused by macro-level structures such as patriarchy and capitalism; (b) research should expose and confront the powerful; (c) researcher and participant relationships should be equal, with equitable distribution of research tasks and roles; and (d) research should result in consciousness-raising and collective action. Coherent with social work values, action research supports the self-determination of oppressed groups and privileges their voice and understanding through the conceptualization, design, data collection, data analysis, and dissemination processes of research.

There are many excellent examples of action research. Some of them focus solely on arriving at useful outcomes for the communities upon which and with whom research is conducted. Other action research projects result in some new knowledge that has a practical application and purpose in addition to the creation of knowledge for basic scientific purposes.

One example of action research can be seen in Fred Piercy and colleagues’ (Piercy, Franz, Donaldson, & Richard, 2011) work with farmers in Virginia, Tennessee, and Louisiana. Together with farmers in these states, the researchers conducted focus groups to understand how farmers learn new information about farming. Ultimately, the aim of this study was to “develop more meaningful ways to communicate information to farmers about sustainable agriculture” (p. 820). This improved communication, the researchers and farmers believed, would benefit not just researchers interested in the topic but also farmers and their communities. Farmers and researchers were both involved in all aspects of the research, from designing the project and determining focus group questions to conducting the focus groups and finally to analyzing data and disseminating findings.

Perhaps one of the most unique and rewarding aspects of action research is that it is often interdisciplinary. Action research projects might bring together researchers from any number of disciplines, from the social sciences, such as sociology, political science, and psychology; to an assortment of physical and natural sciences, such as biology and chemistry; to engineering, philosophy, and history (to name just a few).

Anyone interested in social change can benefit from having some understanding of social scientific research methods. The knowledge you’ve gained from your methods course can be put to good use even if you don’t have an interest in pursuing a career in research. As a member of a community, perhaps you will find that the opportunity to engage in action research presents itself to you one day. Your background in research methodology will no doubt assist you in making life better for yourself and those who share your interests, circumstances, or geographic region.

Spotlight on UTA School of Social Work

Dr. Maxine Davis shares experiences with action research

There are various types of action research. Although the degree to which stakeholders are involved may vary across different stages of the research and dissemination process, each type is valuable and aims to accomplish shared decision-making, responsibility, and power between the researcher and the researched. I will share with you a few examples of recent research that I have had the pleasure of being involved in.

Case 1 (St. Louis, MO) Community based participatory research (CBPR)

As a community organizer, activist, and Missionary, Ms. Johnson is well connected to her community in North St. Louis city. She has worked in partnership with a number of clergy members throughout St. Louis on improving the overall well-being of African-Americans for a number of years. From education to political engagement, she has her pulse on the many issues of local residents and a wide network of clergy and ministers who trust her. In 2014, I partnered with Ms. Johnson to explore clergy perceptions on religious or spiritual (R/S) related abuse within intimate partner violence (IPV). Ms. Johnson conducted more than half of the interviews (many of which occurred only because of the trust clergy members had with her, not due to my recruitment efforts). We coded the data independently and analyzed it as a team. As a result, Ms. Johnson gained the skills to conduct basic qualitative data analysis that may be applicable to her other work. The study results revealed that R/S abuse in IPV was a serious issue that Black clergy often faced in ministry. Furthermore, they desired training to help them to better prepare in responding to this problem. The project did not end at manuscript development, rather the efforts to address this issue continue as we develop and plan to implement R/S specific IPV training for Black clergy in St. Louis.

Case 2 (Chicago, IL) Community-engaged research using a Community Collaborative/Advisory Board (CCB)

A colleague who knew of my interest in the intersection of religious faith and IPV connected me with a priest at St. Pius V parish who was looking for someone to evaluate a portion of the church’s domestic violence program.  The project combined evaluation research and action research. I sought and obtained funding tosupport the first step of a multi-phase project involving process evaluation in preparation for a longitudinal impact (i.e. outcome) evaluation. I convened a collaborative board of relevant stakeholders from different organizations and relocated to Chicago (Pilsen neighborhood) to embark upon the research. Over the course of one year, I lived in the community and collected various types of data from a variety of sources while the CCB and I developed an evaluation plan that would meet the organization’s needs. The primary research questions explored were: “What is The Men’s Group (TMG)?” and “Why do participants attend and remain engaged in TMG?” We discovered that TMG was a trauma-informed, culturally-tailored (to Latino men), spirituality and group based partner abuse intervention program (PAIP) aiming to stop violence perpetration and help participants become self-aware. Men remained engaged in the PAIP because they were met with respect by staff/facilitators, reported gaining benefits because of participation, and connected with other group members through a brotherhood. A quasi-experimental design using quantitative data is currently underway.

Case 3 (Grand Prairie, TX) Youth-led CBPR

The Grand Prairie Storm Track & Field Association (GP Storm) reached out to me after their founders saw me present on the potential of hip-hop music influencing public perceptions about IPV. Our shared interest on increasing Black/African-American representation in health-related research careers brought us together. I invited high school students who were affiliated with the program to join me in examining this area, but also encouraged them to develop a set of their own research questions that they were excited to explore. We met weekly over the course of 3 months in the summer of 2019 and analyzed the lyrics of 7 hip-hop songs. The youth-led research team consisted of six Black/Multiracial young women (5 high school; 1 middle school), the organization founder/director, a PhD student, and myself. The findings revealed that hip-hop music brings awareness to IPV/A by discussing Death, Denial, Freedom, and Physical violence/various types of consequences. Partnering with the GP Storm and affiliated students (the community researchers) allowed the research team to examine research questions that were of interest to a wider audience and do so by drawing on multiple perspectives, thereby improving the rigor of the study. The research did not end here; rather next steps involve hosting a listening party as an intervention to reduce violence and acceptability thereof amongst youth and adults.

Lessons learned

I have learned a few lessons through conducting community-engaged research that I think are worth sharing. It is imperative that you are comfortable openly discussing race and diversity if you plan on engaging in action research with communities of color. This applies, regardless of your own identity, but is especially relevant for those who are an “outsider” in terms of gender or race/ethnicity. The second lesson is that trust need not be earned once, rather you must continuously build and maintain trust in order to conduct sound research. You must also plan to nurture and intend to maintain these relationships in a humanistic manner, beyond that of “a research product.” If your intentions are genuine and you are honest with any trepidations, that plus meaningful project delivery will carry you far.

References

Refer to following articles for more exploration into this research:

Davis, M., ^Johnson, M., Bowland, S. (In Draft) “I hate it…but it’s real”: Black Clergy Perspectives on Intimate Partner Violence related Religious/Spiritual Abuse

Davis, M., ^Dahm, C., Jonson-Reid, M., Stoops, C., Sabri, B. (Revisions Submitted-Awaiting Final Decision). “The Men’s Group” at St. Pius V: A Case Study of a Parish-Based Voluntary Partner Abuse Intervention Program.

^denotes community partners