ACF Banner
[NCCIC header image][NCCIC Header Image]

Quality Assurance

As discussed in Section 2 of this toolkit, within professional development systems are several interconnected components. These components fall under five broad elements: (1) core knowledge; (2) access and outreach; (3) qualifications, credentials, and pathways; (4) funding; and (5) quality assurance. As seen in the following table, this section will provide information about how to ensure and measure achievement with professional development systems.

System Question

System Element

What is it?

Core knowledge

Why does it matter and what is available?

Access and outreach

How can we work toward it?

Qualifications, credentials, and pathways

How can we afford it?

Funding

How do we ensure and measure achievement?

Quality assurance

This section provides an overview of quality assurance and its components, key points from provider and policy perspectives, a State Story about its quality assurances, brief State examples, and related resources. Appendix J includes a quality assurance-related planning tool.

Element Overview

Through the quality assurance element of professional development systems, States and Territories ensure the quality of professional development and measure achievement. Quality assurance activities are vital to monitor, approve, and evaluate professional development offerings and the overall professional development system. Quality assurance strategies include establishing qualifications for trainers and standards for training; conducting participant, activity, and system evaluations; engaging diverse advisory groups; monitoring adherence to licensing and other regulations or program requirements; and investment in local, State, or national accreditation.

Quality assurance components

  • Approval Processes

    Approval of training and trainers is one method of ensuring appropriate and meaningful professional development activities are occurring, in line with the philosophy and direction of the professional development system. A combination of approved content and deliverers can help ensure that trainings and coursework are of high quality.

    At least 23 States have implemented some type of trainer and/or training approval process or a trainer registry: Arizona, Arkansas, Connecticut, Delaware, Florida, Georgia, Idaho, Maine, Maryland, Missouri, Montana, Nevada, New Jersey, New Mexico, Oklahoma, Oregon, Pennsylvania, South Carolina, Texas, Washington, West Virginia, Wisconsin, and Wyoming. Several States include a practitioner registry and/or a trainer directory as a component of their professional development systems.

    • Trainer Approval

      A trainer approval (e.g., standards and registries) system is defined as a set of standards and qualifications for those who offer training. A trainer registry is a database of trainers and their qualifications. Competent, approved trainers are well equipped to support the various needs of adult learners. Most trainer approval systems include a combination of requirements specific to early childhood content expertise and an understanding of adult development, facilitation, and/or instruction methods. These systems may also have requirements related to work experience in training adults and/or education in adult development and learning. They may also require direct service work experience as one method of ensuring that delivery will be relevant to provider participants.

    • Training Approval

      A training approval system is defined as a set of standards that training must meet, usually linked to core knowledge and principles of adult learning. The approval of training is critical to ensure that the content delivered is appropriate, up to date, and linked to other standards as prioritized by the professional development and overall early childhood system. Some States develop specific curricula that make up their approved training; others approve individual trainings based on a set of criteria for each knowledge or content area, level of training, and modality considerations.

  • Evaluation Processes

    Evaluation plays an essential role in effective professional development systems. Ideally, evaluations are planned for an overall system, built into each element and/or component, and are part of an iterative process that continually informs and improves the system. State professional development systems often set varied goals for the overall system and its inter-related components, and employ a range of evaluation methods to assess achievement of these goals. Logic and theory of change models can be useful tools for system evaluation. Performance-based contracts for operation of professional development systems or specific elements or components embed evaluations and goal achievement. As system-level or component plans are developed, initial data can be gathered to serve as benchmarks for later collections. In addition, data and evaluations from those involved (both participants and trainers) can also instruct planning and revision processes. While participant evaluations have a long history in professional development activities, system, element, and component evaluations are still being refined. Part of this refinement includes developing standard definitions and measurements of professional development—a local and national challenge.

  • Monitoring Processes

    Monitoring adherence to staff qualification and professional development requirements specified in licensing regulations, program or funding standards, accreditation, and other quality improvement systems (e.g., quality rating systems) provides a concrete foundation for overall quality assurance. Unlike other quality assurance processes, monitoring adherence focuses on the documentation that requirements have been met. Practitioner registries, described further in Section 6 of this toolkit, can also play a significant role in verifying achievement of qualifications and ongoing training.

  • Accreditation

    Accreditation is a voluntary process designed to improve the quality of early care and education programs. Accreditation systems require early care and education programs to meet standards that exceed minimum State regulatory requirements. Achieving accreditation involves extensive self-study and validation by professionals outside the program to verify that quality standards are met. Most accreditation processes include staff-specific standards that address qualifications and professional development.

Perspectives on Quality Assurance

Professional development systems impact people in different roles in unique ways. The following provides some key points about the importance of the quality assurance element from both the provider and policy perspectives.

Provider perspective

  • With low compensation and limited time and funds, each training or other professional development activity that providers participate in should be high quality.
  • Providers need to know that the training and professional development they participate in will help them with their work.
  • Providers want their professional development to help them meet requirements (e.g., licensing, program, accreditation, etc.).
  • Providers want to receive professional development from people they can trust, who know the subject area, and who can relate to their day-to-day work.

Policy perspective

  • Accountability is key. Limited funds and high expectations from funders, other agencies, practitioners, families with young children, and the public make quality assurances particularly important. Funded professional development services and systems need to show that they are making an impact on provider practices and eventually on child outcomes.
  • With the stress on accountability, there is a constant tension between summative and formative evaluations. To get to a summative evaluation that points to changes in practice and potentially in child outcomes, the implementation of the intervention needs to be evaluated (i.e., a formative evaluation is needed).
  • If a program or system is not performing well, policy-makers want to know how to make it better or whether funds might be better invested elsewhere.

State Story: Oklahoma

The following describes how Oklahoma developed its trainer approval process. A brief overview of its professional development system provides some context for Oklahoma’s quality assurance efforts. Also included is a description of how its specific work began, its evolution, successes, challenges, lessons learned, and future plans.

The National Child Care Information and Technical Assistance Center (NCCIC) would like to thank Jill Soto, Oklahoma’s Center for Early Childhood Professional Development (CECPD) project director, and Judy Collins, former committee member and adviser to CECPD, for their contributions to the following State Story.

CECPD, which houses the components of Oklahoma’s professional development system, supports those who work in licensed child care settings, including family child care homes, child care centers, and Head Start programs. CECPD is supported by a grant from the Division of Child Care within the Oklahoma Department of Human Services (DHS), and operates under the auspices of the University of Oklahoma’s College of Continuing Education. The Oklahoma Early Childhood Professional Development Council serves as an advisory body to CECPD and consists of several workgroups that provide guidance and assistance regarding training, the director and school-age credentials, compensation supplements, articulation agreements, career advising, model observation sites, and a literacy initiative.

How it began

Development of Oklahoma’s professional development system was first funded in 1976 with a contract between DHS and the Oklahoma State University (OSU) extension program. The efforts of a graduate student, who maintained a small library of professional development materials under this original contract, led to institutionalization of support for professional development in the State.

From the onset, planning and delivering training and quality assurance was factored into Oklahoma’s professional development system. Initially there was no training available that was specific to the needs of center-based staff and family child care providers. Ten-hour courses developed by OSU staff became the first approved courses offered specifically for the child care field. The courses, developed for workshops and conferences, evolved around specific content. At one point there were 40 courses, which became the foundation for the professional development system’s career lattice.

Once the content for each of the approved courses was developed, it became apparent that trainers who were competent in delivering the training materials were critical to maintaining consistent and reliable content delivery. In response to this need, training and trainer approval processes were developed. These processes then evolved into the Oklahoma Training for Child Care Careers (OTCCC) project—a new, nonprofit organization governed by a board of directors. The OTCCC training and trainer approval system included a content committee along with other committees and workgroups. To be an approved trainer in this system, an individual had to meet requirements in three areas: formal education, direct experience working with children, and direct experience teaching adults. The process of approving trainers included reviewing three written personal references, a resume, college transcripts, an Oklahoma State Bureau of Investigation check, a criminal background check, and a sex offender stamp (proving they were not on a list/registry).

In the OTCCC system, members of the trainer and training content committee reviewed the applications and made recommendations to the board of directors for each applicant to be approved at one of four levels: provisional trainer, conditional trainer, team trainer, or facilitator. Approved team trainers were matched with more qualified trainers to present a few courses and were then re-evaluated. The trainers, including college faculty and trainers with advanced degrees, were all unpaid volunteers, dedicated to the system and supportive of the approval process.

Evolution

In 1992 Oklahoma identified 10 individual agency representatives to attend the first professional development preinstitute meeting at the National Association for the Education of Young Children Professional Development Institute, facilitated by the Wheelock College Center for Career Development in Early Care and Education. Those individual agency representatives formed a team that established a model for cooperation and collaboration around professional development.

With consultation from Wheelock College to establish a full professional development system, OTCCC was absorbed into CECPD. CECPD then set up a new trainer approval system because there were so many more people training, and they were only receiving a nominal fee for this work. To manage the influx of applications, the process was changed to a paper review, the requirements were revised, and approval was completed by an individual staff person. In the course of these changes, CECPD staff became concerned about the quality of training and consulted with the designers of the original approval system for insight. As with many efforts, the approval system cycled back to be more like its original design.

The current Oklahoma Training Approval System (TAS) includes requirements for three levels of educator (i.e., trainer) roles—apprentice, practitioner, and specialist—and a content expert role. Apprentice and practitioner educators must have at least an associate’s degree, experience working with young children, and professional growth training for adults. Specialists must have at least a bachelor’s degree, experience working with young children, and professional growth training for adults. Content experts are required to have a current license, certificate, or credential in a specific training topic. All educators must also complete 16 clock hours of CECPD adult education courses within 1 year of the application approval. All coursework used to fulfill the education requirements must be from an accredited college or university recognized by the Oklahoma Department of Education. Further details about the educator and content expert requirements are available on the CECPD Web site at www.cecpd.org/OK%20TAS/becomeEducator_new.html.

In the transition to TAS, references were again required and the committee review was reinstituted. Two people on the committee conduct an indepth check of applicants’ qualifications, including a check of the validity of transcripts and whether the schools were accredited, as there were some issues with a diploma mill. Transcripts are also checked for evidence of at least 12 hours of early childhood coursework. A syllabus is used as documentation of content. Reference letters are reviewed with follow-up phone calls, if deemed necessary. The full committee reviews candidates’ entire files. If applicants have been trainers in the past, all of the course/workshop evaluations are also reviewed.

Currently, each trainer starts as a Level I apprentice educator. A mentor coach works with the apprentice trainer. The length of time a mentor works with a trainer apprentice varies depending on performance from one 10-hour course to a maximum of 30 hours. After this time, the coach provides a recommendation that the trainer be approved as a Level II practitioner educator, Level III specialist educator, or that he/she continue at the apprentice level for continued support. Currently there are approximately 130 active educators, not including the apprentice trainers.

The process also includes continuous compliance, such as follows:

  • An educator evaluation is sent out for every fourth course taught and returned directly to CECPD;
  • Surprise onsite observations are conducted using an instrument designed by the University of Oklahoma to look at the effectiveness of trainers and ensure that that they are meeting course criteria, following the curriculum, conducting paperwork, being appropriate in class, etc.; and
  • Every 3 years, approved trainers go through a renewal process. They must update their application and training information and complete professional portfolios. All evaluations from sessions they have facilitated/taught are reviewed. Trainers also must stay current with their adult training methods coursework. On a quarterly basis, CECPD offers training on adult development, ethics, completing paperwork, setting up workshops, etc.

Keys to success

The success of Oklahoma’s system is in large part due to the involvement of an active stakeholder group, which eventually became the OTCCC board. Representative participation continues to include higher education faculty; Oklahoma Health Department staff, including licensing staff; child care providers; and representatives from the State Department of Education, Head Start, Tribes, and other community leaders.

A spirit of common purpose and clear goals sustained the teamwork and collaboration and resulted in a high level of ongoing stakeholder engagement. As with all efforts, new leadership has been developed as people move in and out of the process and as changes in the training delivery system have occurred.

Challenges

An early and ongoing challenge has been having enough people to conduct trainings in every area of the State. Like many States, Oklahoma has diverse geography and many rural areas. Transitioning to the new associate’s degree requirements, a new curriculum, and a revised approval system has resulted in temporary reduction in the number of approved trainers. However, as of August 2007, there is a trainer educator in 70 of the State’s 77 counties. The 7 counties that do not have an educator are all located in far northwest Oklahoma, including the panhandle, where there are 64 child care facilities.

Lessons learned

The Oklahoma trainer approval process has gone through many changes in the past 25 years. One lesson learned is that to have a strong training delivery system, there must be quality control in place for the trainer as well as for the training. The focus tends to be on the training curriculum; however, having a strong curriculum will not have an impact if there are no people who can deliver it competently. The challenges come with a lack of energy and resources. Training approval is a time-consuming process, but it is key to maintaining the integrity of the career lattice. When the process for reviewing applications was streamlined for efficiency, the trainer quality seemed to wane. The process has gone back to include a more careful review and includes mentoring and support to ensure higher quality trainers.

Balancing the increased educational requirements for trainers and maintaining a supply of trainers has been a challenge, especially in some areas of the State. Some of the experienced trainers do not have associate’s degrees. A grandfather clause that allows for the requirements to be phased in could help resolve this issue.

Another lesson learned is that it is important to have bridges built in from one generation of the system to the next—whether the new generation is the way the system looks, a new curriculum, or in the traditional sense of age and leadership. It is important to be intentional about transitions.

Future plans

The trainer approval process will remain a key component of Oklahoma’s professional development system. The State is committed to this process, which contributes to the quality assurance of the full system. It is part of a movement to embed outcomes and measurement throughout the State’s entire system. Even though it has limited resources, Oklahoma is focused on measuring what is making a difference in practice and program quality. A valid system must continually evolve and can never be thought of as done. It must be sensitive to changes in society, the culture, and the profession.

Sources

NCCIC gathered information included in Oklahoma’s story via interviews with Judy Collins on July 10, 2007, and Jill Soto on August 16, 2007. Several additional sources were also used, including various materials from CECPD’s Web site at www.cecpd.org.

State Examples

The following are some examples of States’ quality assurance efforts. They do not include all States that have quality assurances, but are meant to represent a range of approaches States have taken to develop this system element.

Approval processes

Training Approval

  • Connecticut

    Connecticut Charts-A-Course (CCAC) is the statewide professional development system for early education and school-age care. CCAC has a quality assurance system to document the credibility and reliability of core knowledge curriculum content and delivery, create a more highly specialized training experience for participants, as well as provide an ongoing professional development experience for CCAC approved trainers. In this system, random, unannounced observations of trainers are conducted, and trainers must submit a self-evaluation form and participate in meetings to discuss the results. Details about the system are in Training Approval Board Policies and Procedures (March 2005), which is available on the Web at www.ctcharts-a-course.org/forms/TAB.pdf. Additional information is available by visiting the Web at www.ctcharts-a-course.org, or calling 800-832-7784 or 203-397-4036.

  • Georgia

    The Georgia Child Care Training Approval System is funded by Bright from the Start: Georgia Department of Early Care and Learning, and housed at the University of Georgia Center for Continuing Education. Trainers apply for approval to conduct in-state trainings and submit information about training sessions that require approval for meeting licensing requirements. Guidelines based on trainer expertise in content areas, adult learning theory, and current theory and best practices in early care and learning have been set and are used to evaluate training applications. The requirements for trainers include 60 hours in adult education/learning. A reference guide for the system is available on the Web at https://www.training.decal.state.ga.us/gccta/information.do?page=rg. Additional information is available by visiting the Web at https://www.training.decal.state.ga.us/gccta/welcome.do or calling 706-542-6999.

  • Montana

    In 1998 the Early Childhood Project at Montana State University was funded to manage and oversee the early care and education career development system in Montana. The training approval system is designed to ensure quality by approving noncredit training that relates to the State’s core knowledge areas. All agencies, organizations, and people sponsoring noncredit training for early childhood practitioners must apply for approval to be included in the statewide training calendar. Additional information about the system is available on the Web at www.montana.edu/ecp/training.html. Additional information is available by visiting the Web at www.Montana.edu/ecp or calling 406-994-4746.

Trainer Approval

  • Maine

    The Maine Roads Trainer Registry establishes standards for trainers who deliver the core knowledge training. Trainers are required to meet certain standards that are recognized by national organizations and meet higher education requirements. Additional information about the Maine Roads to Quality Trainer Registry is available by visiting the Web at http://muskie.usm.maine.edu/maineroads/pages/trainerregistry.htm, or calling 888-900-0055 or 207-780-5846.

  • Missouri

    The Opportunities in a Professional Education Network (OPEN) Initiative’s Trainer Registry is a database for trainers in the field of early childhood and school-age/after-school or youth development education in Missouri. It collects and verifies trainers’ education and experience, and assists in the development of trainer criteria for a trainer approval system through the Missouri Department of Health and Senior Services. Information about the registry is available on the Web at https://www.openinitiative.org/content.aspx?file=TrainerRegistry.txt. For additional information, call the OPEN Initiative at 877-782-0185 or 573-884-3373, or visit the Web at www.openinitiative.org.

  • New Jersey

    Professional Impact NJ (formerly the New Jersey Professional Development Center for Early Care and Education) is a comprehensive statewide system of professional development for educators. It includes a career lattice linked with educational opportunities and resources. The Instructor Approval System is a statewide system whereby instructors in early care and education, out-of-school time, and primary education may submit their applications to the New Jersey Registry of Childhood Professionals. They may then be approved as an associate instructor, instructor, or master instructor, depending on their educational level and amount of work experience. All child care centers, agencies, and parent groups looking for speakers or trainers for a particular topic have access to the Approved Instructor Database, located on the Web at www.njpdc.com/1/findinst.asp. Additional information about the Instructor Approval System is available by visiting the Web at www.njpdc.org/1/Main_IAS.htm or calling 908-737-4240.

Evaluation processes

  • California

    More than 40 counties in California have implemented compensation programs based on the CARES (Compensation and Recognition Enhances Stability) model. Lessons from CARES and Other Early Care and Education Workforce Initiatives in California, 1999-2004: A Review of Evaluations Completed by Fall 2004 (2004), by Marcy Whitebook and Dan Bellm, published by the Center for the Study of Child Care Employment Institute of Industrial Relations University at the University of California at Berkeley, offers an overview of evaluations of CARES programs and discusses findings, lessons learned, and possible directions for the future. This resource is available on the Web at www.iir.berkeley.edu/cscce/pdf/cares.pdf. Additional information about CARES is available on the Web at www.w4qcc.org.

  • Kentucky

    The Kentucky Professional Development Framework Research Collaborative (PDFRC) is a 3-year, $1.2 million grant from the Administration for Children and Families, U.S. Department of Health and Human Services, awarded in 2004. It supports a collaboration among the Interdisciplinary Human Development Institute and Department of Special Education and Rehabilitation Counseling at the University of Kentucky; the University of Louisville; the Cabinet for Health and Family Services, Division of Child Care; and the Kentucky Department of Education, Division of Early Childhood Development. The research team will build on the current statewide evaluation of the KIDS NOW Initiative by conducting research about the degree to which a statewide unified professional development system impacts the educational level of early care and education providers and subsequent classroom quality. Moreover, the influence of these indicators will be examined to determine their impact on child outcomes.

    The research design includes a multisite, mixed-methods design with 79 centers and 330 classrooms and teachers who represent three types of classroom settings (i.e., child care, Head Start, and State preschool). Child-level outcome data will be collected from 395 children across these settings. Data collection will be coordinated with members of the KIDS NOW evaluation team to maximize resources.

    The primary objective of PDFRC is to determine (1) the degree to which a unified professional development system developed at the State level results in positive child outcomes and (2) the degree to which the educational level of early childhood and education providers enhances the quality of classroom environments. Additional information is available by visiting the Web at www.ihdi.uky.edu/pdfrc or calling 859-257-2083.

  • North Carolina and Multi-State Evaluation Information

    In 1990 the Child Care Services Association created the T.E.A.C.H. (Teacher Education and Compensation Helps) Early Childhood® Project to address the issues of low education levels, poor compensation, and high turnover within the early childhood workforce. Additional information about the project is available on the Web at www.childcareservices.org/ps/teach.html. The Child Care WAGE$® Project provides education-based salary supplements to low-paid teachers, directors, and family child care providers working with children between the ages of birth to 5 years. Additional information about the Child Care WAGE$ Project is available at www.childcareservices.org/ps/wage.html. See Section 7 for additional information about these projects.

    T.E.A.C.H and WAGE$ evaluation data can be found in Early Childhood Workforce Investments: A National Strategy: The T.E.A.C.H. Early Childhood & Child Care WAGE$ Projects, 2005-2006 Annual Program Report (2006), by the Child Care Services Association. This report is available on the Web at www.childcareservices.org/_downloads/TEACH_annual_report_06.pdf. Additional information is available by calling the association at 919-967-3272.

Selected Resources

The following are a sample of resources covering topics related to quality assurance. These resources are categorized by components of the quality assurance element, and are listed in alphabetical order by title. Additional resources are available via NCCIC’s Library, which can be accessed at http://nccic.acf.hhs.gov/index.cfm?do=oll.search.

Evaluation Process

Title:

Critical Issues in Early Childhood Professional Development

Author:

Martha Zaslow and Ivelisse Martinez-Beck (Eds.)

Publisher:

Paul H. Brookes Publishing Company

Date:

2006

URL:

Information about how to order this book is available on the Web at  www.brookespublishing.com/store/books/zaslow-8256/index.htm

 

This book identifies what the field needs to learn about early childhood professional development, specifically in knowledge of the characteristics and size of the early childhood workforce. The field still needs to understand how early childhood professional development contributes to the quality of early care and education and school readiness; the evidence about strategies to strengthen the qualifications of this workforce; and the methods available to assess the costs and benefits as well as market effects of differing approaches to strengthen professional development.


Title:

Evidence-Based Practice Empowers Early Childhood Professionals and Families, A FPG Snapshot #33

Author:

Frank Porter Graham Child Development Institute

Publisher:

Frank Porter Graham Child Development Institute

Date:

September 2006

URL:

www.fpg.unc.edu/%7Eimages/pdfs/snapshots/Snap33.pdf

 

This resource recommends a five-step process for evidence-based practice decision-making for the early childhood field: (1) pose the question, (2) find the best available research evidence, (3) appraise the evidence quality and relevance, (4) integrate research with values and wisdom, and (5) evaluate. Web sites are listed that provide information about early childhood intervention research.


Title:

"Impact of Training and Education for Caregivers of Infants and Toddlers," in Research-to-Policy Connections No. 3

Author:

J. Lee Kreader, Daniel Ferguson, and Sharmila Lawrence, National Center for Children in Poverty

Publisher:

Child Care & Early Education Research Connections

Date:

August 2005

URL:

www.researchconnections.org/SendPdf?resourceId=6874

 

This report summarizes research about the approaches to caregiver training and education that are most likely to improve the quality of care for children younger than age 3 years in family child care homes and centers.


Title:

Roundtable on Measuring Quality in Early Childhood and School-age Settings: At the Junction of Research, Policy and Practice

Author:

Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services

Publisher:

Office of Planning, Research and Evaluation

Date:

July 2007

URL:

www.researchconnections.org/SendPdf?resourceId=12621

 

This resource is a synopsis of the proceedings of a roundtable discussion, convened by the Child Care Bureau in December 2006 in Washington, DC, about emerging issues in the area of State child care quality measures.

Title:

Working Towards a Recommended Common Core of Measures of Early Childhood Professional Development: Issues and Preliminary Recommendations, White Paper

Author:

Martha Zaslow, Tamara Halle, Michelle McNamara, Debra Weinstein, and Ayonda Dent

Publisher:

Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services

Date:

July 2007

URL:

www.researchconnections.org/SendPdf?resourceId=12685

 

This resource presents issues related to the limitations of existing measures of early childhood professional development, with preliminary recommendations for addressing areas of difficulty.