Highlights

  • Research reports were identified by searching library databases, the Internet, scientific journals, and reference lists.

  • The 66 reports included in the Research Guide met at least one of the following criteria: Conducted in Canada, conducted outside Canada and the US, conducted by an acknowledged research team, studied at least 100 children, studied effects on school achievements.

  • The child care factors represented in the reports were identified: type of care, age of child care entry, time spent in care, stability of care, quality of care, teacher education, adult-child ratio.

  • Demographic characteristics of the studies, characteristics of the sample, and research methods were extracted and coded from each study.

  • Results were represented as the relationship of each child care factor to scores on tests of children's cognition, language, and behaviour.

  • Across the 66 reports there were 563 tests of the relationship of child care factors to children's development.

  • The results were quantified, summed, and displayed as positive, null, or negative based on statistical significance.

The first step in the construction of CCKM’s Research Guide for Child Care Decision Making was to find and retrieve possibly-relevant research publications. Possibly-relevant reports were defined as those that empirically studied the relationship of child care conditions to children’s development. Most reports were found in peer-reviewed journals but some were found in commissioned government reports. Rather than limiting ourselves to peer-reviewed database searching, the CCKM review team cast a wider net with a more efficient and targeted strategy of iterative Reference searching.

Research reports were first identified through a search of the most prominent and relevant database of early childhood research, ERIC (Educational Resources Information Center), using the Cambridge Scientific Abstracts database platform. Resultant citations were then subjected to the Web of Science database search engine to obtain collateral reports. Internet database search engines were used to obtain national and international government reports and other position and opinion papers. Although these documents were rarely in themselves empirical studies, their lists of reference citations provided more opportunities for discovering possibly-relevant research reports. Canadian governmental and non-governmental documents helped locate research conducted in Canada but external to ERIC’s database. Using these procedures, a working reference list was created for the Research Guide.

Database searching was followed by reference searching. The reference list of each article retrieved from the databases was examined for citations. As the Research Guide’s working reference list grew, it was possible to identify two research journals that published a significant number of child care articles. They were: Child Development and Early Childhood Research Quarterly. The review team searched the Tables of Contents of 45 issues of these journals and located only 13 articles that had not already been identified. Finally, six reviews of research literature including meta-analyses and other reports were found and retrieved. The citation lists of the reviews were examined for possibly-relevant articles. Sixteen additional articles were identified and added to the Research Guide’s working reference list.

The iterative process of reference searching was continued. Each new article was retrieved and examined for additional citations. By the end of the search process, possibilities for the discovering new articles were essentially exhausted, and the search for research reports was considered complete.

Details of the steps taken to identify research reports for possible inclusion in the Research Guide are presented below.

Search of Online Databases

(a) The first search of the literature was conducted using the ERIC CSA database and the following command line:

(KW= child care or KW= day care or KW= child-care or KW=day-care) and (KW=child* or preschool*) and (KW= school readiness or KW= social development or (KW= cognitive development or cognitive ability)) and (KW= quality)

Results = 235 citations
(b) Articles were considered relevant for possible inclusion in the Research Guide if their investigations empirically linked child care factors to measures of children’s development or behaviour. Of the 235 articles identified in the ERIC database, 45 journal articles were retrieved as possibly-relevant for the Research Guide based on titles and abstracts.
(c) The second search used the Web of Science search engine to locate articles that cited those already identified, as well as cited works by prominent child care researchers. Fifteen new articles were identified and added to the Reference list.
(d) The third search used World Wide Web search engines to locate national and international governmental and non-governmental documents. Canadian documents identified resources such as You Bet I Care!, government publications from Statistics Canada, and child care survey resources such as the National Longitudinal Survey on Children and Youth (NLSCY), National Institute of Child Health and Human Development (NICHD), and Understanding the Early Years. Research articles were either found directly by the search or found as articles linked to government (e.g., Human Resources and Social Development Canada) or early child development websites (e.g., Centre of Excellence for Early Childhood Development, www.excellence-earlychildhood.ca). Twenty-one empirical research studies were retrieved from this search.
(e) The searches listed above resulted in a working reference list of 81 relevant citations.

 

Search of Reference Lists
(a) The references cited in each article in the working reference list were located and examined. Possibly relevant, as determined by the title and abstract, were retrieved and added to the working reference list. The reference lists cited in the added articles were also examined for possibly-relevant citations, and reference searching continued. As a result, 17 citations were added to the working Reference List.  
(b) Citations of national and international government reports and position papers were examined; If relevant, based on the articles’ titles and abstracts, articles were retrieved and added to the working Reference list. Citations of 19 articles were added to the working Reference List.
Search of Journals and Literature Reviews
(a)

Two academic peer-reviewed journals were selected for examination. They were selected based on their high yield of articles retrieved the steps above. Tables of Contents of a total of 45 issues were reviewed for possible relevant articles.   

Child Development: Volume 1 (1986) through Volume 20 (2005) Early Childhood Research Quarterly: Volume 51 (1980) through Volume 76 (2005)

Three relevant articles, that did not already exist in the working Reference list, were retrieved and added from Child Development, and ten were retrieved and added from Early Childhood Research Quarterly.

(b)

Only primary research articles were included in the reference list, and therefore reviews of the literature including meta-analyses and other such reports were excluded. These documents, however, were located by database searching and used to find additional relevant articles. Twelve reviews, meta-analyses, and reports were retrieved, but due to the extensive and overlapping citation lists, six documents which contained the seemingly most fruitful reference lists were examined.
 
Ceglowski, D., & Bacigalupa, C. (2004). Keeping current in child care research. Annotated bibliography: An update. Early Childhood Research & Practice, vol. 4. Retrieved May 27 from http://ecrp.uiuc.edu/v4n1/ceglowski.html

Doherty, G. (1996). The great child care debate: The long-term effects of non-parental child care. Toronto: Child Care Resource and Child Care Unit.
Dunn, L. (1994). Ratio and group size in day care programs. Child and Youth Care Forum, 22, 193-226.

Fiene, R. (2002). 13 indicators of quality child care: Research update. Presented to: Office of the Assistant Secretary for Planning and Evaluation and U.S. Department of Health and Human Services, Washington DC.

Love, J.M., Harrison, L., Sagi-Schwartz, A., van Ijzendoorn, M.H., Ross, C., Ungerer, J.A., Raikes, H., Brady-Smith, C., Boller, K., Brooks-Gunn, J., Constantine, J., Kisker, E.E., Paulsell, D., & Chazan-Cohen, R. (2003). Child care quality matters: How conclusions may vary with context. Child Development, 74, 1021-1033.


Vandell, L. D., Wolfe, B. Child Care Quality: Does It Matter and Does it need to be improved? University of Wisconsin-Madison: Institute for Research on Poverty. Retrieved May 27, 2005 from http://aspe.hhs.gov/hsp/child carequality00/child carequal.htm.


Sixteen additional citations were identified from the above six reports.

At the end of the search, the Research Guide’s working reference list contained 146 citations.

The inclusion of citations in the working reference list was based on information contained in the titles and abstracts of the articles. The inclusion of research reports in CCKM’s Research Guide for Child Care Decision Making was determined by examining the full-text of the 146 citations.

Articles were selected if they included data on children’s development and met one or more of the following criteria.

The study was conducted in Canada.
The study was conducted outside Canada and the US.
The study was conducted by a frequently cited child-care research team.
The sample size was
100.
The design included follow up assessments at school age.

The objectives that guided these criteria included interest in data collected in Canada, interest in international research, interest in studies that had been recognised by governments and other child care researchers, studies of significant magnitude to suggest reliability of data, and studies that addressed long-term impact. These criteria reflected the idea that national context influences the research goals, and they minimized the possible influence of the large number of small-sample idiosyncratic studies conducted in the United States. Such studies by individual university researchers are subject to possible publication-bias due to the fact that many journals in which child care research reports typically reside (e.g., Child Development, Developmental Psychology, etc.) are owned and operated in the United States, and may be more likely to publish small-scale US reports. We sought an international perspective of research on child care factors and their impact on children’s development. Our strategies were consonant with the Research Guide’s objective of providing a landscape of research findings for child care decision making in Canada. 

Details of the steps taken to include research reports in the Research Guide are presented below.

Inclusion/Exclusion Criteria

Reports that included data on children’s development and met at least one of the criteria   
 below were included in the Research Guide. 

    1. Conducted in Canada

    2. Conducted outside Canada and the US

    3. Conducted by an acknowledged child-care research team

    4. Sample size 100

    5. Longitudinal assessments

A research assistant, trained in systematic review methods, applied the inclusion/exclusion criteria to the 146 articles in the working Reference list using both title and abstract and full-text examination. To ensure the reliable application of the inclusion/exclusion criteria, a second trained research assistant also applied the criteria to each of the 146 articles. Inclusion/exclusion disagreements where they occurred were resolved through discussion with the research team. The inclusion/exclusion consensus process resulted in the assignment of 66 articles to the Research Guide.

The 66 research articles were coded in terms of one or more of the seven child care factors they addressed: teacher education, adult-child ratio, time spent in care, age of entry, type of care, stability of care, and quality of care. An article could contain more than one study and the research could address more than one factor.
 

Child Care Factors

Teacher Education Adult-Child Ratio Time Spent in Care Age of Entry
Caregivers’ years of education and/or amount of specific child care training. Adult-child ratio and class size results are  grouped under the same factor heading. Higher adult-child ratio and smaller class size are on the same end of the continuum. Number of hours, days, weeks, months, or years in a child care setting. Age at which children entered child care.
Type of Care Stability of Care Quality of Care
Primary child care setting: parental in-home care, non-parental child care in home setting, or centre setting, etc. Number of different caregiving arrangements a child experienced. Definitions of quality of child care differed across studies. For example, quality of care was defined as sensitive caregiving, and/or as physical and structural aspects of care. Researchers used various standard assessment scales of child care quality or created their own measurement scales.

Information from each article was systematically extracted and entered in a Microsoft Excel database by two trained research assistants. The following information was extracted from each study in each article.
 

Demographic Characteristics of Studies

Citation Year Country Age Age Group
Author(s), year, title, journal, volume pages Publication year Location of data collection Years & months of children’s ages at time of assessment Infancy, Preschool, or School-age at time of assessment

 

Characteristics of Sample

Jurisdiction of Sample Sample Size Databases & Measures Type of Child Care

Municipality, provincial, state, inter-provincial/state, national, international

Number of children who participated in the study.

Database source if applicable; measures of children’s development

Parental home care
Care in non-parental family home setting with non-relative, relative, other
Non-parental centre care
Other

 

Characteristics of Methods

Time of Child Care Data Collection Design of Study Time of Assessments Control Variables

Prospective or retrospective collection of data concerning factors of child care and children's development

Observational or  experimental (including method of participant assignment and experimental variables) Contemporary with child care (No Follow-up); longitudinal (Follow-up) Characteristics of family and child that were measured and accounted for either by selection of groups (e.g., low-income vs high income) or by statistical procedures

Results relating to the association of child care factors with child development outcomes were counted as positive, negative, or zero relative to their direction and whether they met the statistical convention of p ≤.05. A result was given a value of 1 if the effect was found for the composite measure. For example: If a positive relationship of Teacher Education to the summary score of the Peabody Picture Vocabulary Test was reported, a count of 1 was entered as a ‘+’ in the Language Category cell of the row for the article on the Teacher Education spreadsheet.

Each positive, negative, or zero result found for a subscale of an assessment tool was assigned a fractional (proportion of 1) value of the assessment scale. For example: If a positive result for Teacher Education and the Woodcock-Johnson Tests of Cognition on the letter-identification subscale was reported and a null results was found for the memory sentences subscale, .5 was added to the ‘+’ column, and .5 is added to the ‘0’ column. 

Subscales of tests were given fractional scores for two reasons. Studies in which children happened to be measured in finer detail (many subscales) would not overly influence the summary of results for the factor studied. After all, any one report might be unique in other ways (country in which it was conducted, regulations concerning parental leave, characteristics and quality of research methods). By using fractional scores for subtests and confining the overall test to a value of 1, multiple testing procedures did not give any one report undue sway over the conclusions. Secondly, differences in results across subscales of the same test should not be ignored. Fractional scores allowed the results of the subtests to be accurately credited as +, 0, or -. The disadvantage of this procedure was cosmetic in that the final tally of test results was not always an integer (see results for the impact of type of care on tests of language development).

Results

Direction of Result Tables and Graphs
Positive (+), null (0), or negative (-) relationship of child care factor and children’s category of development. Results for each measurement, including subscales of tests, were recorded (N=563). Results were based on the outcome of the statistical tests (p >.05 = null) used in the report. Summary of results of tests of cognitive, language, and behavioural development across studies for each factor.

CCKM prepared the Research Guide as a quantitative and qualitative, easy to understand summary of findings within and across the 66 reports. The Research Guide is meant for parents, practitioners, policy analysts, and others who want to consider accumulated research findings. The preferences of stakeholders, as well as the complexities and vagaries of the study designs and analyses, did not advocate for meta-analyses of the data.

Nevertheless, the methods used in the Research Guide were not seen as a substitute for meta-analyses. Indeed, the citations and where possible full-text of reports are available in the Research Guide for interested readers who want to conduct meta-analyses for the seven factors and three categories of child development. Such analyses will require their own compromises in calculating and synthesising effect sizes, but will add to the compendium of knowledge that can inform decisions about the work-time care of young children. Confidence in conclusions was provided by the large sample sizes and the many test results obtained in the reports.

Citations and descriptions of all database surveys of children’s development relating to child care (e.g., NLSCY) identified in the reports were recorded. Assessment scales and other tools used in the 66 articles to measure the characteristics, development, and behaviour of children and families were identified and described. Assessment scales and other tools used to measure the construct of child care quality were also identified and described. Citations of the 66 articles reviewed in the Research Guide and citations from which the included articles were drawn as well as  other references cited in the Background text of the Research Guide were listed.

The spreadsheets were used to construct CCKM's Research Guide for Child Care Decision Making. Information from the spreadsheets and the full text of the reports was drawn to complete the Summary of Evidence.

Scorecards for each child care factors were constructed by compiling data derived from the Microsoft Excel file. The citations for each research reports that address each factor were annotated with tables that listed  the country of data collection, the sample size and age group of the children in the study, the large-scale database survey from which the data were drawn if applicable, and the methods of the study. Published abstracts were added to the annotated citations, and full-text files were linked. Finally, the Resources sections of the Research Guide were compiled from the spreadsheets.

   2006 © Canadian Centre for Knowledge Mobilisation