SOCW 6311-Provide a research question that would be appropriate for this study of the employment program for the parolees described in Plummer et al. (2013), Social Work Research: Chi Square

https://conjointly.com/kb/internal-validity/

 

This week we will review a statistical test, the chi-square, and see how it applies to a social work research study from the textbook.

I think that this is the hardest week of the term, even if you have studied statistics before.

First, there is a LOT to read and comprehend.  You may read the assigned learning resources. In particular, there is a handout on a short course in statistics.  I suggest printing out the handout and tucking it in a safe place to review for your licensing exam. Alternatively, you can read the information I have placed in the top discussion post and the assignment instructions, which contains all you will need to answer the questions in the discussion and assignment.  I urge you to read my information in both  documents before starting the discussion because you will have a firm grasp on the concepts by the time you read it all.

If you limit your reading to what I have placed in the discussion and assignment,  I advise you to read the assigned textbook material and handout when you have some extra time.  I do not suggest reading it all first.  Of course, you are free to read all the assigned material before tackling the material – it\\\’s good stuff – but if time is an issue, you will get what you need from my post and the assignment instructions.

Second, I think that reading off a monitor is very hard, especially when there are graphs and other aids. Scrolling back and forth breaks the continuity of thought.   I strongly advise printing out the discussion instructions and assignment instructions.

Here\\\’s how I suggest tackling the reading:

In grade school, you might have been introduced to the SQ3R method of reading – survey (skim), question, read, review, and report (or write).  At the time, I thought the method was b-o-o-o-o-o-r-i-n-g.   Now I think it is genius.

Here\\\’s how it works:

1. S:  Skim the pages so you get an idea what the topic is.   Watch any videos and skim material from any book, webpage, or other source linked in the discussion post and assignment instructions. These are short and will save you a lot of time and anxiety.   Then  take the printed copies of the discussion post and assignment instructions, lay them out, and grab a highlighter and a pen.

2. Q:  Read the questions I present in the discussion and the assignment. You will find them under \\\”By Day 3\\\”, \\\”Your Task\\\” or some similar prompt.

4. R: Go back through the discussion and assignment instructions and look for the answers to the questions.  Since you skimmed the material earlier, you will have a general idea where to look.  Read around the answer for context. By the time you have found all the answers, you will have read the entire set of instructions and it won\\\’t seem so overwhelming.  Mark up your pages with your highlighter, make notes, draw arrows – whatever works to help you find the answers again when you write the report.

5.  R: Now re-read the questions so you are sure you found all the details.

6.  R: Look for the requested format in the instructions and answer the questions, fill in the blanks or do as instructed to complete the report for the discussion or assignment.

Note: I am attaching a handout on threats to validity and confounding variables.  I think it is a better list than the one by Trochim in the learning resources, but you are free to have a different opinion.

Done.

My feedback comments in the discussion might help you in the assignment, so look for them after you post to the discussion board.

Your assignment should not exceed one page, including references, but if they spill over to the second page, there is no penalty.  Just be clear and concise.

This is a crazy week for me but I will be checking my email frequently. If you are stumped, don\\\’t agonize, grab the chocolate, or honk at lousy drivers. Email me. You will survive, and what\\\’s more, things will smooth out once we turn the corner on this week.

Week 4 Assignment Details and Aids for Successful Completion

 

Information from the Walden University dashboard for the assignment:

Statistical analysis software is a valuable tool that helps researchers perform the complex calculations. However, to use such a tool effectively, the study must be well designed. The social worker must understand all the relationships involved in the study. He or she must understand the study’s purpose and select the most appropriate design. The social worker must correctly represent the relationship being examined and the variables involved. Finally, he or she must enter those variables correctly into the software package.

This assignment will allow you to analyze in detail the decisions made in the “Social Work Research: Chi Square” case study and the relationship between study design and statistical analysis. Assume that the data has been entered into SPSS and you’ve been given the output of the chi-square results.  (Walden University, SOCW 6311 Week 4 Assignment, 2021).

To prepare for this Assignment:

  1. Review the information I placed in my discussion post regarding the statistical analysis for this case, and contact me at yank@mail.waldenu.edu if you are confused about the meaning of the numbers in the tables for the statistical analysis described there.

 

  1. Watch the video on chi-square tests of independence presented by Dr. Chris Duke: https://www.youtube.com/watch?v=ZjdBM7NO7bY

 

  1. See Rod Pierce’s web page Math is Fun! for a very good description (with pictures!) of the relationship between chi-square results and p-values: http://www.mathsisfun.com/data/chi-square-test.html. It’s only one page and you will never be confused again! You will need to understand the use of p-values in chi-square analysis for this assignment.

 

Don’t worry about the math – both videos show how the numbers in the tables are obtained, but you will never have to calculate these in this course or in your MSW program!  It’s presented so you can understand what the numbers in the table mean.

 

 

Some basics about chi-square tests of independence (chi-square tests, for short)

 

Chi-square tests are appropriate when the data are in the form of counts that fall into two or more discrete, non-overlapping nominal categories. Examples include these and many more:

  • gender (male, female, non-binary);
  • political views (liberal vs. conservative);
  • locale (urban vs. rural);
  • ratings (good, better, best)

When the outcome data or groups fall into nominal categories (such as no employment, part-time employment, and full-time employment the chi-square test is the most appropriate test to use for data analysis. Chi-square tests are not used for continuous measurements such as  heights, scores on tests, time to complete a task  where the values do not reflect nominal categories.

 

A note about null and alternative hypotheses

 

Researchers start out with the assumption that there is no relationship between the factors in the study  (the intervention and employment status, in this case study.) This is the null hypothesis.

 

The reason for starting with a null hypothesis is buried in the mists of statistical history.  Most people, including statisticians, think this approach is backwards. But since it’s been done like this for so long, it’s hard to change ingrained habits.  In most cases, researchers hope that their findings permit them to reject the null hypothesis in favor of the alternative hypothesis.

 

The alternative hypothesis proposes that the program has a positive effect on employment status, meaning that those who get the treatment (participation in the program) are more likely to be employed, and much more likely to get full-time employment.

__________________________________________________

The interpretation of the chi-square value

 

The value of the chi-square statistic is calculated mathematically. It can be done by hand, but most of the time, we use statistical software programs to do it quickly for us.

 

The software compares the results of the study to the results that would be expected if the intervention had no relationship to the results. In chi-square language, when there is no relationship between the intervention and the outcome in a chi-square test, we say that the variables (intervention and outcome) are independent of each other.

 

If the differences between the results obtained and the values that would be expected if there was no effect of the intervention are large enough,  we can conclude (within a margin of error) that the results obtained depend upon the intervention.

 

But how do you know if the difference between groups and the chi-square value are large enough to be statistically significant?  We look for the very important p-value.

 

The p-value tells us whether there is a significant difference between the results obtained and the results expected if the intervention had no effect. A significant p-value would tell us that the outcomes from the employment program had a statistically significant relationship to the intervention.

 

Pesky little detail: a significant p-value does not mean that the intervention causes the outcome in a chi-square testInstead, a significant p-value in a chi-square test means that the independent and dependent variables are linked.  In our case study, it would mean that the intervention has a significant effect on the employment outcomes.

 

Statisticians have decided that the chi-square test is not powerful enough for us to say that the intervention  causes the effects on the employment outcomes. This may seem like splitting hairs, but it has to do with some details in chi-square analysis that are too complex for our purposes.

 

Some basics about interpreting data

  1. Your first task is to understand the purpose of the research. In most research articles, the purpose is described as the research question, the aim of the research, goal of the research, or some similar description.   It tells us why the researchers did the study and what they wanted to learn from it.

 

The purpose of the research dictates everything about the study: the participants, the research design, the intervention, the data collection methods, and the statistical analysis.

  1. Our next task is to understand the purpose of the statistical analysis. In our case study for the week, we are determining if there is a relationship between two factors: an intervention factor and an employment status factor:
  • The intervention factor has two levels: treatment and wait list status
  • The employment status factor has three levels: not employed (job = none), part-time employment, and full-time employment.

 

We want to know if there is a statistically significant relationship between the intervention and the employment status (remember this – it may come in handy when answering the questions below!).

  1. Finally, the third task is analyzing the data. Most researchers analyze data with plots and statistical tests.

In the case study, Table 1 shows the results of the study in terms of the number of people from the program group and the waiting list group in each employment category (none, part-time, full-time) at the end of the program.

 

 

 

Table 1.

Table 2 shows the results of the chi-square analysis with the accompanying p-value (in blue)

Table 2.

Chi-Square Test
  Value df Asymp. Sig. (2-sided)
Pearson Chi-Square 11.748a 2 .003
N of Valid Cases 59    
a.      No cells (0.0%) have expected

count less than 5. The minimum

expected count is 6.88.

Red arrow points to the chi-square statistic              Blue arrow points to the p-value

____________________________________________________________________

Reporting results

Use the APA format (see the box below) to report results. Any description of the results must contain a brief description of the purpose of the analysis, how the study was set up (comparison between conditions – treatment and waiting list, in this case), a statement that the test met the assumptions for the test (cell frequencies greater than 5 for chi-square), and the statistical results. These are presented in four brief but informative sentences. (Lund, 2018).  This information is what other researchers expect to read, in this order, with as few words as possible to convey the complete meaning. 

The chi-square statistic is represented by χ2(2) which is the Greek letter “chi” (pronounced K-eye, which rhymes with my or tie). If you cannot reproduce the symbol χ2 in your assignment, write out “chi square”.  The superscript above the chi symbol, 2, stands for “squared”.  (2) in parentheses means that there are two degrees of freedom (that is, the number of categories of employment- full-time, part-time, or none – minus 1). 3 categories -1 = 2, which is the number placed in parentheses. See how the numbers in Table 2 are placed in the sentence giving the results in the box below.

A chi-square test for independence was conducted between intervention and waiting list participants on their employment status. The rates of employment for a group receiving the intervention were compared to the employment rates of a group of wait-list members who received no intervention. All expected cell frequencies were greater than five.  A statistically significant association was found between intervention and employment status, χ2(2) = 11.748, p = .003.

You may copy and paste this into your report without quoting a source because this is one specific format endorsed by the APA.  Alternatively, you may write something very similar, but do not expand on the results or use metaphors.  Readers look for this sentence to determine if they want to continue reading about the study.

__________________________________________________________________________________

YOUR TASK for this assignment:  A one page report on the statistical analysis of the case study as directed by the FORMAT provided below.  If references spill over to the next page, there is no penalty, but try to be concise so that everything fits on one page. Please put your name and Week 4 Assignment in the header of your paper, but do not include a title page or abstract, please.  Number your answers.

 

Writing style. Apply all the scholarly writing goals that we have covered so far: academic integrity, appropriate format in the reference lists and in the in-text citations (capitals and italics, in particular).

Make your paper readable and well organized with numbers to identify the prompt to which you are responding.

At least 80% of your report should be your own synthesis or paraphrases of the material in the articles; no more than 20% of the paper should consist of quotes.

If you quote a source, please cite it carefully and accurately. Copied material much be enclosed in quotation marks. Without them, you are passing off the writing of experts as your own. Points are deducted for this error!

Beginning this week, your reference list and in-text citations will be subject to guidelines for formatting as indicated in the discussion format which are found in the Publication Manual of the American Psychological Association (7th ed). In subsequent weeks, additional APA formatting guidelines will be demonstrated in the weekly Scholarly Writing Goal. Once they are introduced, you will be graded on them.

Points are deducted for late submissions according to Walden policy as stated in the syllabus,  grammar and composition errors, lack of scholarly expression, or missing essential details as prescribed in the report.

FORMAT FOR THE ASSIGNMENT

  1. Provide a research question that would be appropriate for this study of the employment program for the parolees described in Plummer et al. (2013), Social Work Research: Chi Square. In other words, why did the agency conduct this research? State what the agency wanted to learn regarding the relationship of the intervention to the outcome (1-2 accurate, brief, and clear sentence(s))
  2. Explain why the social work agency in the case study used a chi square statistic instead of some other statistical method to evaluate whether there is a statistical difference among the outcomes for the people who participated in the program and those who remained on the waiting list by completing the following sentence. You may add one more sentence to complete your thought if necessary :

A chi-square test of independence instead of a different statistical method was used to analyze the results because____________ (1-2 sentences)

  1. Describe the data collection methods used to obtain the employment status of each participant after the end of the program as described in the case study. Assume that the data were collected several months after the end of the study, and report problems that might arise with these methods. Identify a process of data collection that might have made an apples-to-apples comparison difficult or compromised the validity in another way. Consult the handout regarding threats to validity sent with this assignment. Respond by completing the following sentences.

Methods for data collection used in this study included _____________________________.

Potential threats to validity that might arise with these methods are______ because _________________ [ (1-2 sentences)

  1. State the results of the statistical analysis of the data in this study, using the conventional APA format as listed above.   Include Pearson’s chi-square value, degrees of freedom (df) and p-value (under Asymp. Sig.).

 

  1. Explain the meaning of the p-value, which is the number found under Asymptotic Significance in the second table (circled). (1 sentence). The Math is Fun! webpage linked above may be helpful with this question.
  2. Interpret the results for the reader by completing the sentences below. Provide your rationale from the data obtained in Tables 1 and 2 above:

Results: These findings suggest that the program [did/did not] have an effect on employment status because _____ % of those in the intervention group obtained full-time employment compared to _____ % in the waiting list group.   _____ % in the intervention group remained unemployed compared to ____ % in the waiting list group. Differences in the number of parolees who obtained part-time employment [were/were not]  different.

Rationale: The results for this finding [are/are not] significant because the [select a statistic from the Chi-Square Test Table]  for the chi-square analysis is _____, which [is/is not] a significant value. 

  1. Explain whether the findings answered the research question you listed in #1. Base your explanation on the findings you reported in 5. (1-2 sentences).
  2. 8. In one or two sentences, identify something beneficial that you learned about writing research reports from this assignment.

Include a reference list for sources that are cited in your paper.

__________________

If you get stuck on the statistical analysis, contact me for help. Don’t struggle. This is a lot of information to digest this week.  

 

Assignments are due by 11:59 p.m. Mountain Time (MT) on the day assigned (which is 1:59 a.m. Eastern Time (ET) the next day). The time stamp in the classroom will reflect Eastern Time (ET), regardless of your time zone. As long as your submission time stamp is no later than 1:59 a.m. Eastern Time (ET), you have submitted on time.

 

References

 

Duke, C. (2019, June 3). StatsCast: What is a chi-square? [Video]. YouTube. https://www.youtube.com/watch?v=ZjdBM7NO7bY

 

Lund, A. (2020). Learn how to accurately report and interpret your results. In Laird Dissertation: an online textbook. https://statistics.laerd.com/features-writing-up.php

Pierce, R. (13 Oct 2016). Chi-Square Test. Math is Fun. http://www.mathsisfun.com/data/chi-square-test.html

Plummer, S.B., Makris, S., Brocksen, S. (2013). Social work research: Chi-Square. In Social Work Case Studies: Concentration Year. Laureate Publishing, 10/21/13. [VitalBook file]