first step in the construction of CCKM’s Research Guide for
Child Care Decision Making was to find and retrieve
possibly-relevant research publications. Possibly-relevant
reports were defined as those that empirically studied the
relationship of child care conditions to children’s
development. Most reports were found in peer-reviewed journals
but some were found in commissioned government reports. Rather
than limiting ourselves to peer-reviewed database searching,
the CCKM review team cast a wider net with a more efficient
and targeted strategy of iterative Reference searching.
reports were first identified through a search of the most
prominent and relevant database of early childhood research,
ERIC (Educational Resources Information Center), using the
Cambridge Scientific Abstracts database platform. Resultant
citations were then subjected to the Web of Science database
search engine to obtain collateral reports. Internet database
search engines were used to obtain national and international
government reports and other position and opinion papers.
Although these documents were rarely in themselves empirical
studies, their lists of reference citations provided more
opportunities for discovering possibly-relevant research
reports. Canadian governmental and non-governmental documents
helped locate research conducted in Canada but external to ERIC’s database.
Using these procedures, a working reference list was created
for the Research Guide.
searching was followed by reference searching. The reference
list of each article retrieved from the databases was examined
for citations. As the Research Guide’s working reference list grew, it
was possible to identify two research journals that published
a significant number of child care articles. They were:
Child Development and Early Childhood Research
Quarterly. The review team searched the Tables of Contents
of 45 issues of these journals and located only 13 articles
that had not already been identified. Finally, six reviews of
research literature including meta-analyses and other reports
were found and retrieved. The citation lists of the reviews
were examined for possibly-relevant articles. Sixteen
additional articles were identified and added to the Research
Guide’s working reference list.
iterative process of reference searching was continued. Each
new article was retrieved and examined for additional
citations. By the end of the search process, possibilities for
the discovering new articles were essentially exhausted, and
the search for research reports was considered complete.
of the steps taken to identify research reports for possible
inclusion in the Research Guide are presented below.
of Online Databases
The first search of the literature was conducted using the
ERIC CSA database and the following command line:
(KW= child care or KW= day care or KW= child-care or
KW=day-care) and (KW=child* or preschool*) and (KW= school
readiness or KW= social development or (KW= cognitive
development or cognitive ability)) and (KW= quality)
Results = 235 citations
Articles were considered relevant for possible inclusion
in the Research Guide if their investigations empirically linked
child care factors to measures of children’s development
or behaviour. Of the 235 articles identified in the
ERIC database, 45 journal articles were retrieved as
possibly-relevant for the Research Guide based on titles and
The second search used the Web of Science search
engine to locate articles that cited those already
identified, as well as cited works by prominent child care
Fifteen new articles were identified and added to
the Reference list.
The third search used World Wide Web search engines to
locate national and international governmental and
non-governmental documents. Canadian documents identified
resources such as You Bet I Care!, government
publications from Statistics Canada, and child care
survey resources such as the National Longitudinal
Survey on Children and Youth (NLSCY), National
Institute of Child Health and Human Development (NICHD),
and Understanding the Early Years. Research articles
were either found directly by the search or found as
articles linked to government (e.g., Human Resources and
Social Development Canada) or early child development websites (e.g.,
Centre of Excellence for Early Childhood Development,
www.excellence-earlychildhood.ca). Twenty-one empirical research
studies were retrieved from this search.
The searches listed above resulted in a working
reference list of 81 relevant citations.
Search of Reference
The references cited in each article in the working
reference list were located and examined. Possibly relevant, as determined by the title and
abstract, were retrieved and added to the
working reference list. The reference lists cited in the
added articles were also examined for possibly-relevant
citations, and reference searching continued. As a result,
17 citations were added to the working Reference
Citations of national and international government reports
and position papers were examined; If relevant, based on
the articles’ titles and abstracts, articles were
retrieved and added to the working Reference list.
Citations of 19 articles were added to the working
Search of Journals and
Two academic peer-reviewed journals were selected for
examination. They were selected based on their high yield
of articles retrieved the steps above. Tables of Contents
of a total of 45 issues were reviewed for possible
Volume 1 (1986) through Volume 20 (2005) Early
Childhood Research Quarterly: Volume 51 (1980) through
Volume 76 (2005)
Three relevant articles, that did not already exist in the
working Reference list, were retrieved and added from
Child Development, and ten were retrieved and added
from Early Childhood Research Quarterly.
Only primary research articles were included in the
reference list, and therefore reviews of the literature
including meta-analyses and other such reports were
excluded. These documents, however, were located by
database searching and used to find additional relevant
articles. Twelve reviews, meta-analyses, and reports were
retrieved, but due to the extensive and overlapping
citation lists, six documents which contained the
seemingly most fruitful reference lists were examined.
Ceglowski, D., & Bacigalupa, C. (2004). Keeping current in
child care research. Annotated bibliography: An update.
Early Childhood Research & Practice, vol. 4. Retrieved
May 27 from http://ecrp.uiuc.edu/v4n1/ceglowski.html
Doherty, G. (1996). The great child care debate:
The long-term effects of non-parental child care.
Toronto: Child Care Resource and Child Care Unit.
Dunn, L. (1994). Ratio and group size in day care
programs. Child and Youth Care Forum, 22, 193-226.
Fiene, R. (2002). 13 indicators of quality child care:
Research update. Presented to: Office of the Assistant
Secretary for Planning and Evaluation and
Department of Health and Human Services, Washington DC.
Love, J.M., Harrison, L., Sagi-Schwartz, A., van
Ijzendoorn, M.H., Ross, C., Ungerer, J.A., Raikes, H.,
Brady-Smith, C., Boller, K., Brooks-Gunn, J., Constantine,
J., Kisker, E.E., Paulsell, D., & Chazan-Cohen, R. (2003).
Child care quality matters: How conclusions may vary with
context. Child Development, 74, 1021-1033.
Vandell, L. D., Wolfe, B. Child Care Quality: Does It
Matter and Does it need to be improved? University of
Wisconsin-Madison: Institute for Research on Poverty.
Retrieved May 27, 2005 from http://aspe.hhs.gov/hsp/child
Sixteen additional citations were identified from the
above six reports.
At the end of the search, the Research Guide’s working
reference list contained 146 citations.
inclusion of citations in the working reference list
was based on information contained in the titles and abstracts
of the articles. The inclusion of research reports in CCKM’s
Research Guide for Child Care Decision Making was determined by
examining the full-text of the 146 citations.
were selected if they included data on children’s development
and met one or more of the following criteria.
study was conducted in Canada.
The study was conducted outside Canada and the US.
The study was conducted by a frequently cited child-care research
The sample size was
The design included follow up assessments at school age.
The objectives that guided these criteria included interest in
data collected in Canada, interest in international research,
interest in studies that had been recognised by governments
and other child care researchers, studies of significant
magnitude to suggest reliability of data, and studies that
addressed long-term impact. These criteria reflected the idea
that national context influences the research goals, and they
minimized the possible influence of the large number of
small-sample idiosyncratic studies conducted in the United
States. Such studies by individual university researchers are
subject to possible publication-bias due to the fact that many
journals in which child care research reports typically reside
(e.g., Child Development, Developmental Psychology,
etc.) are owned and operated in the United States, and may be
more likely to publish small-scale US reports. We sought an
international perspective of research on child care factors
and their impact on children’s development. Our strategies
were consonant with the Research Guide’s objective of providing a
landscape of research findings for child care decision making
of the steps taken to include research reports in the Research
are presented below.
included data on children’s development and
met at least one of the criteria
below were included in the Research Guide.
Conducted in Canada
Conducted outside Canada and the US
Conducted by an acknowledged child-care research team
A research assistant, trained in systematic review
methods, applied the inclusion/exclusion criteria to the
146 articles in the working Reference list using
both title and abstract and full-text examination. To
ensure the reliable application of the inclusion/exclusion
criteria, a second trained research assistant also applied
the criteria to each of the 146 articles.
Inclusion/exclusion disagreements where they occurred were
resolved through discussion with the research team. The inclusion/exclusion
consensus process resulted in the assignment of 66
articles to the Research Guide.
The 66 research articles were coded in terms of one or more
of the seven child care factors they addressed:
adult-child ratio, time spent in care, age of
entry, type of care, stability of care, and
quality of care. An article could contain more than one
study and the research could address more than one factor.
Time Spent in Care
Age of Entry
Caregivers’ years of education and/or amount of specific
child care training.
Adult-child ratio and class size results are grouped
under the same factor heading. Higher adult-child ratio
and smaller class size are on the same end of the
Number of hours, days, weeks, months, or years in a
child care setting.
Age at which children entered child care.
Type of Care
Stability of Care
Quality of Care
Primary child care setting: parental
care, non-parental child care in home setting, or centre
Number of different caregiving arrangements a child
Definitions of quality of child care differed
across studies. For example, quality of care was
defined as sensitive caregiving, and/or as physical
and structural aspects of care.
Researchers used various standard assessment scales of
child care quality or created their own measurement
Information from each article was systematically extracted and
entered in a Microsoft Excel database by two trained research
assistants. The following information was extracted from each
study in each article.
Demographic Characteristics of Studies
Author(s), year, title,
journal, volume pages
Location of data collection
Years & months of children’s ages at time of assessment
Infancy, Preschool, or School-age at time of assessment
Characteristics of Sample
Jurisdiction of Sample
Databases & Measures
Type of Child Care
Municipality, provincial, state, inter-provincial/state,
Number of children who participated in the study.
Database source if applicable; measures of children’s
Parental home care
Care in non-parental family home setting with
non-relative, relative, other
Non-parental centre care
Characteristics of Methods
Time of Child Care Data Collection
Design of Study
Time of Assessments
Prospective or retrospective collection of data concerning
factors of child care and children's development
Observational or experimental (including method of
participant assignment and experimental variables)
Contemporary with child care (No Follow-up); longitudinal
Characteristics of family and child that were measured and
accounted for either by selection of groups (e.g., low-income vs high
income) or by statistical procedures
relating to the association of child care factors with child
development outcomes were counted as positive, negative, or
zero relative to their direction and whether they met the
statistical convention of p ≤.05. A result was given a value
of 1 if the effect was found for the composite measure. For
example: If a positive relationship of Teacher Education to
the summary score of the Peabody Picture Vocabulary Test was
reported, a count of 1 was entered as a ‘+’ in the Language
Category cell of the row for the article on the Teacher
positive, negative, or zero result found for a subscale
of an assessment tool was assigned a fractional
(proportion of 1) value of the assessment scale. For example: If a
positive result for Teacher Education and the Woodcock-Johnson
Tests of Cognition on the letter-identification subscale
was reported and a null results was found for the memory
sentences subscale, .5 was added to the ‘+’ column, and .5
is added to the ‘0’ column.
Subscales of tests were given fractional scores for two
reasons. Studies in which children happened to be measured in
finer detail (many subscales) would not overly influence the
summary of results for the factor studied. After all, any one
report might be unique in other ways (country in which it was
conducted, regulations concerning parental leave,
characteristics and quality of research methods). By using
fractional scores for subtests and confining the overall test
to a value of 1, multiple testing procedures did not give any
one report undue sway over the conclusions. Secondly,
differences in results across subscales of the same test
should not be ignored. Fractional scores allowed the results
of the subtests to be accurately credited as +, 0, or
The disadvantage of this procedure was cosmetic in that the
final tally of test results was not always an integer (see
results for the impact of type of care on tests of language
Direction of Result
Tables and Graphs
Positive (+), null (0), or negative
child care factor and children’s category of
development. Results for each measurement, including
subscales of tests, were recorded (N=563). Results were
based on the outcome of the statistical tests (p >.05 =
null) used in the report.
Summary of results of tests of cognitive, language, and behavioural development across
studies for each factor.
CCKM prepared the Research Guide as a quantitative and
qualitative, easy to understand summary of findings within and
across the 66 reports. The Research Guide is meant for
parents, practitioners, policy analysts, and others who want
to consider accumulated research findings. The preferences of
stakeholders, as well as the complexities and vagaries of the
study designs and analyses, did not advocate for meta-analyses
of the data.
Nevertheless, the methods used in the Research Guide were not
seen as a substitute for meta-analyses. Indeed, the citations
and where possible full-text of reports are available in the
Research Guide for interested readers who want to conduct
meta-analyses for the seven factors and three categories of
child development. Such analyses will require their own
compromises in calculating and synthesising effect sizes, but
will add to the compendium of knowledge that can inform
decisions about the work-time care of young children.
Confidence in conclusions was provided by the large sample
sizes and the many test results obtained in the reports.
Citations and descriptions of all database surveys of
children’s development relating to child care (e.g., NLSCY)
identified in the reports were recorded. Assessment scales and other tools used in the 66 articles to
measure the characteristics, development, and behaviour of
children and families were identified and described. Assessment scales and other tools used to measure
the construct of child care quality were also
identified and described. Citations of the 66 articles
reviewed in the Research Guide and
citations from which the included articles were drawn as well
as other references cited in the Background text of the
Research Guide were listed.
The spreadsheets were used to construct CCKM's Research Guide
for Child Care Decision Making. Information from the
spreadsheets and the full text of the reports was drawn to
complete the Summary of Evidence.
Scorecards for each child care factors were constructed by
compiling data derived from the Microsoft Excel file. The citations for
each research reports that address each factor were
annotated with tables that listed the country of data collection,
the sample size and age group of the children in the study,
the large-scale database survey from which the data were drawn
if applicable, and the methods of the study. Published abstracts were added to the annotated citations, and
full-text files were linked. Finally, the Resources sections
of the Research Guide were compiled from the spreadsheets.