Generalizing Study Results

Generalizability is the extent to which research findings from your sample population can be applicable to a larger population. There are many best practices for ensuring generalizability. Two of those are making sure the sample is as much like the population as possible and making sure that the sample size is large enough to mitigate the chance of differences within the population.

• For this Discussion, read the case study titled “Social Work Research: Program Evaluation” and consider how the particular study results can be generalizable.
• Post your explanation of who the sample is. Also explain steps researchers took to ensure generalizability. Be sure to discuss how the study results could possibly be generalizable. Please use the Learning Resources to support your answer.

Social Work Research: Program Evaluation

Major federal legislation was enacted in 1996 related to welfare reform. Financial assistance programs at the national level for low-income families have been in place since the mid-1960s through the Aid to Families with Dependent Children (AFDC) program. The Personal Responsibility and Work Opportunity Reconciliation Act of 1996, or welfare reform, created TANF (Temporary Assistance for Needy Families). Major components of the new TANF program were to limit new recipients of cash aid to no more than 2 years of TANF assistance at a time and to receive no more than 5 years of combined TANF assistance with other service programs during their lifetimes. The goal was to make public assistance a temporary, rather than a long-term, program for families with children. Beyond these general rules, each of the 50 states was given substantial latitude to adopt requirements to fit their own objectives. The new law also allowed states that reduced their public assistance expenses to keep whatever support was already being provided by the federal government for use at their own discretion. This was seen as a way to encourage states to reduce welfare dependency.

In response, the state of California decided to call its new program CalWORKs, the California Work Opportunity and Responsibility to Kids program. CalWORKs is California’s application of the new TANF federal law. Like most of the other states, CalWORKs provided its 58 counties with a fair amount of discretion in how to implement the new provisions. Some counties chose to develop strong upfront “employment-first” rules that mandated recipients be employed as soon as possible. Others chose a response that included testing and assessment and the provision of education and training services.

One of the largest counties in the San Francisco Bay Area developed several options for CalWORKs recipients, including immediate job readiness (Job Club) help, remedial education for recipients lacking basic skills, and vocational training at local community colleges and adult education centers for those seeking higher level education and skills. Recipients could take up to 5 years to complete these activities and even longer in certain circumstances to maximize their chances of success. Recipients were predominantly single mothers. If recipients fully complied with the rules, they received a variety of financial incentives, while those who did not comply received sanctions that often resulted in reduced benefit levels. The county provided grants to a wide array of education, training, and service programs to work as partners in serving the needs of participants.

In 1996, the county’s CalWORKs program enrolled approximately 22,000 families in various forms of public assistance programs. Of these, approximately 10,000 elected to participate in one of the education and training programs, 9,000 elected to attend intensive job placement (Job Club) classes, and the remaining 3,000 opted to not comply with the new program and accepted reduced benefit sanctions.

To meet its state and federal mandates, the county carefully tracked the progress of all program participants and compiled comprehensive quarterly reports that summarized assignments and outcomes at each of the contracted partner sites as well as countywide trends. During the first 11 years of the program, from 1996 through 2007, the county’s public assistance roles were reduced by approximately 40%, from more than 22,000 to about 13,000 families. The best results were obtained among participants in education and training programs, who accounted for about two-thirds of long-term outcome success, although this group was also found to be more costly to the local CalWORKs program during their years of study. These costs, in addition to the longer period of monthly benefits received, also included the cost of education and training and, in some cases, childcare expenses. Among the participants who were placed in the immediate job search (Job Club) program, total costs to the county were somewhat less per year, but more than 50% were still not successful in gaining employment, and those that did find a job received a much lower salary and fewer benefits, and another 23% fell back on CalWORKs after later losing their employment.

Although the results of the CalWORKs program in this county seemed to be following a mostly positive trend from 1996 through 2007, the situation changed dramatically in the opposite direction during the national economic downturn from 2007 through 2011. Total public assistance rolls more than doubled to about 30,000 during this time as the local and state unemployment rate rapidly grew from about 7% to more than 12%. The county was initially successful in getting the state to grant it waivers to allow recipients to extend their period of benefits during education and training, but these waivers were considerably restricted after 2011 due to major state budget cuts. Between 2011 and early 2013 the total number of recipients began to decline again by about 10% from its peak 2 years earlier. However, the total number of CalWORKs recipients is at 27,000, still about 5,000 recipients higher than when the program started in 1996.

Compounding the difficulty of more people becoming eligible for CalWORKs’ benefits due to poor economic conditions, the state’s budget crisis prompted a reduction in state allocations to counties and recipients. Nonetheless, county administrators were still pleased to report that more than more than 16,000 recipients during the program were able to obtain employment or other support that eliminated their dependency on cash public assistance.