Hello, SOCW 6311 Scholars,
This week you will evaluate how a program is providing its services and evaluate the quality of those services with a process evaluation. Please review the difference between a process (performance ) and product (outcome) evaluation, because if you start using the wrong type of evaluation, the entire assignment will be wrong.
You will be conducting a process evaluation of an actual program, indicated in the assignment instructions. I attempted to provide a link, but Blackboard has been temperamental and the link has not been working consistently. If you have problems with it, please enter the title into the Walden Library search bar, and the article will be pulled up immediately.
Note that different people will evaluate the program differently. There is a subjective component to all evaluations. Do not change your responses if someone answers differently. If your reading of the material leads you to answer differently from someone else, that’s life! When have you ever been in a meeting where everyone saw a challenge exactly the same way? If you can defend your view, don’t let anxiety force you to give it up.
Since you are evaluating the methods used in this study, attention to detail is paramount. People become invested in their programs. They can become very defensive and disappointed if an evaluator suggests that their performance or planning is inadequate. However, if you point out specific tasks or behaviors that could be done differently, they may accept the results much more readily. For example, telling the staff that their program is scheduled at an inconvenient time may not be well accepted, but stating, \\\”starting your program at 3 PM instead of 4 PM might reach more students at the end of the school day\\\” may be a suggestion they might consider. Therefore, your evaluation must contain precise action-oriented recommendations rather than general statements.
The assignment seems long because there are a number of questions from which you can choose, but the answers should be in the form of accurate, clear, and concise statements. See the examples in the RAND handouts. I hope that you should be able to complete it in two pages.
Please contact me if you have any questions about the assignment. I am specifically asking that you do this assignment so I can assess your knowledge of the evaluation process most objectively.
Dudley, J. R. (2020). Social work evaluation: Enhancing what we do (3rd ed.) Oxford University Press.
- Chapter 8, “Improving How Programs and Practice Work” (pp. 171–208)
Becker, L. A. (1999). Statistical and clinical significance. Retrieved from https://www.uccs.edu/lbecker/
Man-Son-Hing, M., Laupacis, A., O’Rourke, K., Molnar, F. J., Mahon, J., Chan, K. B., & Wells, G. (2002). Determination of the clinical importance of study results. Journal of general internal medicine, 17(6), 469-476.
Plummer, S.-B., Makris, S., & Brocksen S. (Eds.). (2014b). Social work case studies: Concentration year. Baltimore, MD: Laureate International Universities Publishing. [Vital Source e-reader].
Read the following section:
- “Social Work Research: Qualitative Groups” (pp. 68–69)
Document: Bliss, M. J., & Emshoff, J. G. (2002). Workbook for designing a process evaluation. Retrieved from http://beta.
roadsafetyevaluation.com/ evaluationguides/info/ workbook-for-designing-a- process-evaluation.pdf (PDF)
Georgia Department of Human Resources, Division of Public Health.
Week 9 Assignment: Conducting a process evaluation
Purpose of process evaluation
Good ideas do not always yield good results. Therefore, to understand the tasks to be performed and the scope of effort, a clear description of the implementation process is required. This step also will aid in replicating the effort in other environments. (Stipak, (n.d.)
Process evaluations are conducted during and at the end of programs to view how the services or program components were delivered. A process evaluation helps program developers to identify and address problems that reduce the effectiveness of the program. The process evaluation focuses on the factors that enable or obstruct the smooth delivery of services, determines if the program functions as intended, and demonstrates accountability (or lack thereof) of the organization to the agreements made to funding organizations and stakeholders.
As the Dashboard notes, the procedures for conducting a process evaluation are similar to those used in other types of evaluations such as engaging stakeholders, asking meaningful questions, using appropriate data collection methods, and reporting results.
It is the purpose and timing of a process evaluation most distinguish[es] it from other types of evaluation. […] A process evaluation can also be useful in supporting an outcome evaluation by helping to determine the reason behind program outcomes.
Since accountability to stakeholders and responsibility to deliver the expected outputs with quality and efficiency are essential elements of process evaluations, “documentation from the planning stage [particularly the logic model] may prove useful when planning a process evaluation” to learn the original vision for the program and discover whether it was achieved.
Week 9 Assignment, SOCW 6311, Walden University.
Please take a moment to return to the information in the Week 6 discussion to refresh your memory about the differences between product and process evaluations. You will see both types in the social work literature. In this assignment, you will want to be sure that you are providing the right kind of evaluation.
For an excellent example of some worksheets used in process evaluations, skim pages 76-83 in Getting to Outcomes (Ebener et al., 2017). These worksheets take only a few minutes to skim and will clarify the process for you. Getting to Outcomes is attached to the email that contains this assignment. Alternatively, you can access it at https://www.rand.org/pubs/tools/TL259.html. The title is in very large font, but next to it is a small icon of the workbook with the link Read Online. In the right margin, you can also open it as a webpage. Look for Chapter 7, Process Evaluations, to find pages 76-86. From pp. 81-86, you will find a booklet containing a planning document for the evaluation. It includes the questions to be asked in the evaluation, the data collection tools, a schedule for collecting data, and the name of the person assigned to collect the data. At the end is an example of an evaluation summary provided to stakeholders.
If there is any possibility that you would be asked to evaluate programs in your practice as a social worker, I suggest downloading this free workbook as it is very practical and aligned with current best practices.
In the assignment for this week, you will conduct a process evaluation from information regarding an actual anti-bullying program that was conducted in thirteen elementary schools, described in a research article entitled Process Evaluation of a Bullying Prevention Program: A Public School-County Health Partnership by Lynne Edmonson and John Hoover: (https://eds-a-ebscohost-com.ezp.waldenulibrary.org/eds/pdfviewer/pdfviewer?vid=14&sid=83000077-1890-41e4-8198-9aa9e0f1d49f%40sessionmgr4008). *** If this link does not open, place the title in the Walden Library search bar. Caution!! If you copy the title into Google, a different article comes up! Make sure that you have the right article by the right authors.
Your task for the assignment:
Envision yourself as part of the team that was assigned to conduct a process evaluation of the program. As a member of the team, you are asked to obtain answers to questions regarding the delivery of the program services. Specifically, you are asked to describe the program and find answers to the following questions in the following categories, expanded in the list on the next page:
- Program participants/recipients
- Services/activities performed by staff or participants within the program
- Program staff
- Quality and satisfaction
- Suggestions for improvements and/or changes in program services and activities
To conclude your report, you are asked to comment on the strengths of the program and to identify one area for improvement.
Fortunately, you do not have to develop the questions for the evaluation. Your task involves locating answers to the questions on the following two pages from the article and basing your judgments on the information.
- No title page or abstract. This request is important. Please put your name and page number in the upper margin.
- Give a name for your report at the top of the first page, then proceed with your responses to the questions immediately below it.
- Use numbers, bullet points, or other types of organization schemes (your choice) to make it very easy for readers to find your responses. Remember that stakeholders that seek the information in your report usually prefer to get the information quickly in a brief and well-organized manner.
YOUR REPORT MUST CONTAIN RESPONSES TO THE FOLLOWING QUESTIONS on Pages 3 and 4.
Section I. Program description.
In three sentences or less, describe the rationale or purpose of the program and the intended outcomes.
Section II. Program components
- Identify the resources that the program leaders and staff were able to use to meet the goals of the program. (1-2 sentences)
Did the process of acquiring the resources go smoothly? If not, what obstacles prevented the program staff from getting what was needed?
- Who were the intended participants for the program activities?
- Did the program reach the people that it was intended to reach?
- If the intended participants did not attend, was a reason provided?
- What are the demographic characteristics of the participants?
- Does the report indicate how many people dropped out of the program or declined to participate? If so, were you able to get any reasons for dropping out or declining to participate? Could you find any patterns of participation or dropping out? Do those patterns, if any, reveal external factors that encourage or discourage participation, or pose obstacles to participation?
- Services and activities provided/performed.
- List the specific interventions, activities, services or events that were provided in the program.
- What evidence is provided in the report to show that the services were delivered?
- What information, if any, was provided about problems that arose in delivering the program as planned?
- Program staff.
- Did the report indicate the qualifications of the program leaders or assume that the leaders were adequately qualified? (1 sentence).
- Quality and satisfaction.
- What information was reported about how the recipients, staff, or community members viewed the quality of the program? If so, what was reported?
- Did each participating group have similar views of the quality of the program?
- Did most respondents state that they were satisfied with the program? If not, what were the most common complaints? (1-3 sentences).
- Improvements and changes.
- What changes to the original program plans were recommended, if any, to enable increased achievement of the program goals?
- Were any recommendations made about discontinuing parts or all of the program?
Section III. Your judgments
- List two strengths of the program as described and/or services that encourage participation and assist participants in achieving the intended goals. (2-3 sentences, based on the article and your judgment)
- Make one substantive suggestion for improvement in the operation of the program or manner that the services or activities were provided (2-3 sentences).
Assignments are due by 11:59 p.m. Mountain Time (MT) on the day assigned (which is 1:59 a.m. Eastern Time (ET) the next day). The time stamp in the classroom will reflect Eastern Time (ET), regardless of your time zone. As long as your submission time stamp is no later than 1:59 a.m. Eastern Time (ET), you have submitted on time.
Bliss, M. J. & Emshoff, J. G. (2002). Workbook for conducting a process evaluation. University of Georgia. https://www.roadsafetyevaluation.com/docs/workbook-evaluation.pdf
***Blackboard may be fussy with links. If this link does not open, you can locate the resource in Dashboard or you may paste the url into a search engine to access the article easily.
Ebener, P. A., Hunter, S. B., Adams, R. M., Eisenman, D., Acosta, J. D., & Chinman, M. (2017. Getting to outcomes: Guide for community emergency preparedness. RAND Corporation. https://www.rand.org/pubs/tools/TL259.html
Edmondson, L., & Hoover, J. (2008). Process evaluation of a bullying prevention program: A public school-county health partnership. Reclaiming Children and Youth, 16(4), pp 25-33. (https://eds-a-ebscohost-com.ezp.waldenulibrary.org/eds/pdfviewer/pdfviewer?vid=14&sid=83000077-1890-41e4-8198-9aa9e0f1d49f%40sessionmgr4008). *** If this link does not open, copy the title into the Walden library search bar. Make sure that you have the right authors.
Stipak, B. (n.d.) Process evaluation and impact evaluation. http://web.pdx.edu/~stipakb/download/PA555/ProcessEvalOverview.htm