socw 6301 Wk 10 Assignment: Article Review and Critique

By now, you should be aware that the findings from a research study are only part of the story. As a consumer, hoping to inform practice by use of an evidence base, you want to know much more. A sound research study includes all the steps highlighted in previous weeks: reviewing existing literature, focusing a research question, choosing a qualitative or quantitative method for answering the question, designing the study including selection of data collection procedures and/or measures, procedures used, data analysis plan, and findings. In addition, the study commonly discusses how ethical concerns were addressed and acknowledges the limitations of the study. For this assignment, you review a published research study with two purposes in mind:

• Observing the structure and content of the article
• Comparing the content of the article to the recommended content of sections for a research study By Day 

Submit a 7-10 page critique and review of the article, which includes the title page and the reference list. Follow the guidelines below:

1. Use the quantitative or qualitative research article that you located and that your instructor approved as part of the Week 5 assignment.
2. Provide an APA reference for the article you select.
3. If you selected a quantitative research study, use the “Quantitative Article Review and Critique. If you selected a qualitative study, use the “Qualitative Article Review and Critique.” Respond to all the questions.

Be sure to include the questions in your critique. This will cause your Turnitin report to show high similarity to other students’ papers. However, do not be concerned about that. Do, however, appropriately paraphrase and cite specific details from the article you review.

EDITORIAL

Thoughts on Social Work
Knowledge Development Activities
v^ithin a Quantitative Framev^ork

Andrew Grogan-Kaylor and Jorge Delva

A
s a result of many years of serving as review-
ers for numerous journals from multiple
professions and disciplines and through our

own experience as researchers and authors, we of-
fer the following thoughts on conducting research
within a quantitative framework. We hope these
ideas can further strengthen knowledge develop-
ment activities hy social work researchers who rely
on quantitative methods.The focus on quantitative
methods is largely because this is the work we do,
and hence we helieve we are in a strong position to
offer ideas that can serve to strengthen this line of
work. Colleagues who can make suggestions along
the lines of qualitative research will he sought out to
speak on these issues in a subsequent editorial.

In this editorial, we make a two-stage argument.
First, we argue that quantitative research in social
work must increase in its statistical sophistication if
social work research is to make robust and widely
read conclusions about the social problems and issues
that social workers care about. Second, we argue that
technical sophistication is not enough. Social work-
ers and social work researchers must think carefully
about the research questions, methodologies, and
conclusions that underlie social work research.

For many research questions, simple univariate
statistics such as means, medians, standard devia-
tions, and percentages, or hivariate statistics such
as correlations, t tests, and chi-squares are often
not sufficient to address the research question of
interest. Such univariate and bivariate statistics
provide critical pieces of information about the
study sample and about basic relationships among
variables, but they are only a first step in the pro-
cess of uncovering more complex relationships. In
the case of bivariate statistics alone, for example,
these estimates do not afford the ability to control

for the effects of other variables that might affect,
or account for, the relationships of interest. For
example, in a study of the relationship of a par-
ticular kind of parenting with children’s behavior
problems, it would be important to control for
other variables. Without such statistical controls
it would be possible that any observed bivariate
relationship could be attributed to an unobserved
third factor. As an illustration, in a recent article,
Grogan-Kaylor (2004) used regression methods to
demonstrate that a relationship between parental
use of corporal punishment and children’s antiso-
cial behavior persisted even when factors such as
children’s age, or initial levels of antisocial behavior
were accounted.Through the use of more sophisti-
cated statistical techniques, the author was able to
account for a number of factors that are sometimes
suggested as explanations for the observed relation-
ship between parental use of corporal punishment
and higher levels of children’s behavior problems
and to provide stronger evidence that parental use
of corporal punishment has undesirable effects
on children’s behavior. The particular regression
models were ftxed-efFect regression models, an
extension of ordinary least squares regression that
is able to account for both observed and some
unobserved variables.

We acknowledge that the use of more sophisti-
cated statistical methods that can rule out alternative
explanations in social work research is hindered by
several factors. First of all, authors may not always
have the necessary expertise to carry out the ap-
propriate statistical analyses. Graduate programs in
social work at both the master’s and doctoral levels
are encouraged to provide social work students with
at least some training in multivariate methods. At
the same time, we recognize that it may often be

CCC Code: 0037-8046/08 J3.00 ©2008 National Association of Social Workers 293

beneficial for social workers to collaborate with
researchers such as statisticians, among others, who
have the necessary complementary expertise. In
fact, this is yet another example of the importance
of engaging in multidisciplinary or interdisciplinary
collaborations. It creates a win—win situation for
all disciplines involved. A second limitation on the
use of multivariate methods in social work research
is that it is preferable to use multivariate methods
with larger study samples. For example, a common
if somewhat limited rule of thumb suggests that
between 10 and 20 additional cases are needed for
every additional variable that is added to a model.
Although we believe that the cost of collecting
larger study samples is offset by the more rigorous
conclusions that can be produced with multivariate
research, we recognize that larger sample sizes may
be more difFtcult and expensive to produce.

As the importance of more sophisticated quan-
titative research becomes clearer to the social work
community, more sophisticated methods are enter-
ing the journal literature. With that in mind, we
offer the reminder that good statistics can provide
a foundation for good thinking, but good statistics
cannot replace good thinking. Complicated statistics
bereft of solid conceptualization and research design
methods do little to advance the social work litera-
ture. In that spirit, we offer four recommendations
to authors, to journal reviewers, and to ourselves as
scholars in this field that we believe can substantially
strengthen a researchers work.

First, it is important to have a solid understanding
of the literature and of the competing or comple-
mentary conceptual models. By “solid understand-
ing,” we mean a critical understanding of the
strengths and weaknesses of the theoretical models
that guide prior research and of all the aspects of
the research methods of prior studies upon which a
subsequent study may build. For example, we submit
that it is insufficient to write a literature review in
which fmdings of prior research are presented with-
out a critical analysis of at least one or more of the
following components: the adequacy of the study’s
sampling strategy, sample size, contact and response
rates, conceptualization and operationalization of
measures, research design (for example, posttest
only, pre- and posttest, experimental), and analytic
strategies, among others. We think that it is more
informative for readers to know that even in the
case of a particular cross-sectional study in which
a significant association is found between variables

that no definitive statements can be made about the
temporal and causal directions of these associations
or, that despite being significant, the magnitude of
the association, commonly known as the effect size,
is so small that the substantive importance of the
association is suspect.

Another example of information that readers
may find useful when reading a literature review is
that low participation rate or distinct differences in
characteristics between those who participated and
those who did not may limit the generalizability of
the findings to the sample itself Given the realities
of page limitations that journals have and the large
number of areas in which studies can be criticized,
we suggest that authors who elect to highlight one
or two of the most salient strengths or weaknesses
of the studies they review significantly strengthen
the quality of their manuscripts.

Second, regardless of statistical sophistication, the
interpretation of a study’s findings is bound by the
research methods used in the study. For example, if
a study’s participation rates are low or the sample
is cross-sectional or nonexperimental and the
researchers use structural equation modeling to
analyze associations among variables, the implica-
tions of the findings could be less significant than
the findings of a study that uses simpler statistics,
such as multivariate analyses of variance, and a more
rigorous design such as a longitudinal or a quasi-
experimental or experimental design.Through this
example, we imply three points about being critical
about one’s work: (1) Researchers should not trust
one statistic over another but should remind them-
selves that any analytic approach is bound by the
context of the research method; (2) there is a need
for researchers to use more rigorous research designs
in their knowledge development activities; and (3)
if it is not feasible to use a more rigorous research
design, one that can provide more confidence in
the findings, then it is important for researchers to
critically describe their findings within the context
of these limitations. Of course, if the limitations are
well highlighted there is a greater risk that journal
reviewers will completely agree with the author
and decide that the manuscript should not be pub-
lished. We believe that this is a risk authors should
take because honest and critical analysis better
contributes to the process of knowledge develop-
ment. However, if authors are not persuaded by this
argument, knowledge that reviewers are likely to
raise concerns about the study limitations should

294 Social Work VOLUME 53, NUMBER 4 OCTOBER 2008

be a strong motivator. In addition, as the method-
ological rigor of social work literature improves, we
believe that social work research will become more
frequently cited by other disciplines, increasing the
positive effects of social work research.

Third, it is not enough to focus on statistical
significance. Authors should also pay attention to
the size of the effects under consideration. Statistical
significance, as measured by the ubiquitous “p value,”
is a measure of the likelihood that a given result is
due to chance. Generally, we only see results with
small p values as scientifically meaningful. At the
same time, it is worth repeating the statistical truth
that a result might be statistically significant but that
it might represent a relationship whose size is not
meaningful. Researchers would do well to pay as
much attention to the effect sizes as to their statisti-
cal significance. Several procedures for commenting
on the size of statistical effects exist. In our opinion,
the most basic procedure is to think carefully about
the units in which two variables are measured. For
example, if an income variable is measured in dol-
lars and a mental health variable is measured in the
frequency of certain beliefs, then thinking carefully
about the units in which each variable is measured
may lead to important conclusions about the amount
of change in a mental health variable that is likely to
be associated with a particular magnitude of change
in income. Imagine two sets of results, both of which
are statistically significant. Imagine further that in
the first set of results changes in income are associ-
ated with large changes in mental health, whereas
in the second set of results, the magnitude of the
association is small. In each case, the results would
be statistically differentiable from chance results,
but the substantive implications for policy, practice,
and intervention might be quite different. Although
more sophisticated procedures for assessing effect
sizes can be used, we argue that at the most basic
level we should at least strive toward more careful
thinking.

Fourth, the use of multivariate analyses requires
careful consideration in the choice of a model
that conforms to the major features of the data. A
plethora of multivariate models are in existence.
The ordinary least squares regression models that
are usually the beginning (and too often the end)
of coursework in multivariate models are not ap-
propriate for every situation. For example, ordinary
least squares regression models make the assumption
that dependent variables are continuous and that

every step of that continuum is equivalent. This
may be plausible when trying to predict income,
when every additional unit of income is an extra
dollar. However, in other cases in which the de-
pendent variable may have a range of values, it may
be less plausible that every step of the continuum
is equivalent. For example, in a study of depressive
symptoms, participants might be asked whether
they never (coded as 0), sometimes (coded as 1),
or frequently (coded as 2) experience a particular
symptom. Such a study would afford an outcome
with a range of values (that is, 0, 1, or 2), but many
might find it implausible to assume that the distance
between “never” and “sometimes” is the same as the
distance from “sometimes” to “frequently.” In such
cases, a more appropriate choice would be an ordinal
regression model that captures the ordered nature of
the outcome of interest but that recognizes that the
distances between the levels of the outcome may be
nonequivalent. In still other cases, outcomes may be
clearly noncontinuous. For example, social w^orkers
are likely to be interested in many outcomes that
are categorical in nature. At the simplest level, such
categorical outcomes are binary in nature. For ex-
ample,”Did the respondent smoke tobacco or not?”
or “Did the respondent become homeless or not?”
Several methods for correctly analyzing such binary
outcomes exist. Categorical outcomes can also be
multinomial when outcomes can be classified into
several groups, but there is no distinct ordering of
those outcomes. For example, respondents might
choose to become affiliated with any one of a num-
ber of religious organizations, and there is no way
to rank or order religions in terms of intrinsic value.
For such situations, more complicated multinomial
regression models exist. This discussion has not ex-
hausted the range of possible types of outcomes. For
example, space does not permit us to engage in a
discussion of count outcomes or censored outcomes.
Our point, however, is that accurate specification of
outcomes of interest to social workers may require
use of advanced statistical models.

Dependent variables are not the only statistical
constructs subject to a wealth of possible specifica-
tions. Modeling requires careful thinking about
both the independent and the dependent variables.
One situation in which this commonly occurs,
which is likely to be of interest to social workers,
is when data are nested or clustered. For example,
families and children may be clustered inside com-
munities, schools, health care centers, hospitals, and

G R O G A N – K A Y L O R AND DELVA / Thoughts on Social Work Knowledge 295

organizations, and so forth. As Raudenbush and
Bryk (2002) have pointed out, failure to account for
this clustering or nesting may lead us to mistakenly
infer statistically significant results. For example, in a
study of the effects of neighborhood social climate
on outcomes for children and youths, the failure to
account for the fact that two residents live in the
same neighborhood might overstate the importance
of some effects. One advantage of the use of so-
called multilevel models is that they lead to more
modest, but more accurate, conclusions with data
that are clustered according to the cluster unit (for
example, neighborhoods, schools, organizations).

Another situation that may call for more advanced
statistical expertise is when the research question,
the design, and the data lend themselves to the
modeling of unobserved variables, as in the fmdings
of the study by Grogan-Kaylor (2004) described
earlier. Note that the common ordinary least squares
regression model, which is often the starting point
for coursework in multivariate methods, can only
account for observed variables that are entered
into the model. In the Grogan-Kaylor study, one
explanation for an observed relationship between
parental use of corporal punishment and increased
children’s behavior problems might be unobserved
aspects of context, such as a higher level of neighbor-
hood violence that contributed both to parental use
of corporal punishment and to children’s behavior
problems. If this factor had not been observed and
not entered into a model, the results of the model
could have been biased as a result. In the Grogan-
Kaylor (2004) example, the fixed effects regression
was able to control for a number of such unobserved
variables, strengthening the conclusions provided. As
with many of the other statistical questions we have
discussed, there is a rich and growing literature on
the modeling of unobserved factors, including the
techniques of fixed-effects regression (Wooldridge,
2002) and propensity scores (Morgan & Harding,
2006).

In this editorial, we offer a number of thoughts
on approaches that we think could help knowledge
development activities of social work researchers.
We recognize that we have been inconsistent in
our own work in following these ideas. Therefore,
we offer these thoughts in the spirit of guidelines
to which we aspire. We are certain that if we and
our colleagues follow these guidelines, the quality
of all of our knowledge development activities will
increase.

REFERENCES
Grogan-Kaylor, A. (2004). The effect of corporal punish-

ment on antisocial hehavior in children. Social Work
Research, 28, 153-162.

Morgan, S. L., & Harding, D. J. (2006). Matching estimators
of causal effects: Prospects and pitfalls in theory and
practice. Sociological Methods and Research, 35, 3 – 6 0 .

Raudenhush, S.W., & Bryk,A. S. (2002). Hierarchical linear
models:Applications and data analysis mei/iorfs. T h o u s a n d
Oaks, CA: Sage Publications.

W o o l d r i d g e , J . M . (2002). Econometric analysis of cross section
and panel data. Cambridge, MA: MIT Press.

Andrew Grogan-Kaylor, PhD, is associate professor, and Jorge
Delva, PhD, is professor. School of Social Work, University of
Michigan, Ann Arbor, MI 48109; e-mail: agrogan@umich.edu

and jdeha@umich.edu.

NASW PRESS POLICY ON
ETHICAL BEHAVIOR

The NASW Press expects authors to ad-here to ethical standards for scholarship
as articulated in the NASW Code of Ethics
and Writing for the NASW Press: Information
for Authors. These standards include actions
such as

• taking responsibility and credit only for
work they have actually performed

• honestly acknowledging the work of
others

• submitting only original work to
journals

• fuUy documenting their own and others’
related work.

If possible breaches of ethical standards have
been identified at the review or publication
process, the NASW Press may notify the au-
thor and bring the ethics issue to the attention
of the appropriate professional body or other
authority. Peer review confidentiality will not
apply where there is evidence of plagiarism.

As reviewed and revised by
NASW National Committee on
Inquiry (NCOI), May 30, 1997

Approved by NASW Board of
Directors, September 1997

296 SocidWork VOLUME 53, NUMBER 4 OCTOBER 2008