Stage 4: Support and Monitor

ESSA's evidence provisions specify four levels of evidence and encourage states and districts to use the most rigorous type of evidence available. SEAs and LEAs using Title IIA funding for professional learning can use all four levels of evidence. The top three levels of evidence require findings of a statistically significant effect on improving student outcomes or other relevant outcomes:

• Level 1 - Strong evidence: At least one well-designed and well-implemented experimental (i.e. randomized) study.
• Level 2 - Moderate evidence: At least one well-designed and well-implemented quasi-experimental (i.e. matched control group) study.
• Level 3 - Promising evidence: At least one well-designed and well-implemented correlational study with statistical controls for selection bias

The fourth level is useful when the research base on a particular type of intervention is not well developed and few or no rigorous studies are available. It consists of interventions that are developing and promising but do not yet have evidence qualifying for the top three levels.

• Level 4 - Evidence-building and under evaluation:

  • Demonstrates rationale based on high-quality research or positive evaluation that such activity, strategy, or intervention is likely to improve student outcomes;
  • Includes ongoing efforts to examine the effects of such activity, strategy, or intervention.

More evidence on what works to increase student success is available now than ever before, but education leaders may not have extensive experience accessing that information. Recognizing the time and effort involved in this process, some states are providing research summaries or hiring organizations to make reviewing and selecting interventions easier for districts (Klein, 2018).

For state and district education leaders conducting reviews, the following tips can provide a sound start:

1. Conduct a review of the literature and educational research.
2. Select studies that are not older than 1990.
3. Focus on studies in which the participants match or are similar to the students you serve. For example, evidence about a program that has been effective in elementary schools may not have the same impact on high school teachers and students. More evidence about the impact on high school students would be needed before selecting that program for implementation.

When reviewing studies, district leaders keep in mind that student achievement (usually measured on standardized tests) is an important metric of success, but not the only one. Educators can also consider the strategies' alignment with state or national standards, consistency with current statements of experts or professional associations, and other indicators of quality.

A continuous improvement cycle is one of the best ways to use to determine where and when to collect and use evidence. Continuous improvement cycles can use and build evidence in multiple ways. District and school leaders, school staffs, and others can use questions associated with each phase of the cycle to begin using evidence to help solve their most vexing problems.

 

Questions that go with each cycle

Plan:

• What is our goal?
• What is our challenge?
• What does the strongest available evidence recommend for our context?
• How will we evaluate our impact?
• What can we learn from studies about prior implementations?

Implementation:

• How will we know how we're doing along the way?
• How will we address issues throughout implementation?
• What do the data tell us?
• What explains our successes and setbacks?

Evaluation:

• What changes should we make?
• Have our needs changed since we began addressing this?
• What can others learn from our experience?

 

During the monitoring phase, the local education agency (LEA) staff should consider looking for data about changes in teacher knowledge, skills, behaviors, attitudes, and aspirations (KASAB) associated with professional learning. They should also consider identifying changes in classroom practices associated with professional learning. Using and sharing data strategically helps sustain momentum and inform continuous improvement.

Federal regulations require that state educational agencies (SEAs) monitor the implementation of program requirements and the expenditure of federal funds at the LEA level. Section 1111(h)(1)(C)(x) of the Elementary and Secondary Education Act (ESSA) requires the reporting of:

The per-pupil expenditures of Federal, State, and local funds, including actual personnel expenditures and actual non-personnel expenditures of Federal, State, and local funds, disaggregated by source of funds, for each local educational agency and each school in the State for the preceding fiscal year. (p. 34578 )

 

Monitoring of federal programs is conducted to ensure that all children have a fair, equal, and significant opportunity to obtain a high-quality education. However, monitoring traditionally has emphasized accountability for the wise use of resources and little monitoring has been conducted on the impact of the expenditure of resources or the quality of services provided. ESSA provides the vehicle for LEAs and states to achieve high-quality implementation of educational programs. Tool 4.1: Rethinking progress monitoring can help LEA staff plan their monitoring programs.

Under ESSA, states are required to publish school-level spending data in report cards. Such information is intended to shine a light on the education investments in communities and how those investments are used to serve students. Data Quality Campaign's infographic illustrates what it looks like when leaders - from state and local leaders to principals and community advocates - use school-level spending data in their roles to ensure successful, well-resourced classrooms (see next page).

After SEAs have determined and communicated their monitoring and reporting requirements, the next step is for LEAs to communicate a monitoring schedule for schools, including details about what they need to have in place for monitoring. The ultimate goal of monitoring the use of Title IIA funds is to measure whether the uses of the funds make an impact on teaching and learning and the plan is being followed or revised as needed.

An essential component of the monitoring plan is the identification of the data LEAs and schools need to collect before implementation begins. The monitoring process is more difficult, expensive, and time-consuming if LEA staffs begin looking for data on progress and results after the fact. Other attributes of good monitoring plans describe how districts will: 1) measure progress, 2) improve accountability and management of resources, 3) use data efficiently and effectively for decision making and measuring impact, 4) improve collaboration and coordination with partners, and 5) collect complete and timely information (see Tool 4.2: Reporting progress).

Monitoring is also a way to signal to educators the importance of continuous learning and improvement. By following the principle of continuous learning and improvement, districts can revisit their measures of impact, evaluate the success of their Title IIA investments, and recalibrate their strategy to better support teacher and leader effectiveness and increased student learning. ESSA gives states and districts the opportunity to involve stakeholders in decisions about the most important things to monitor. The TALIS Survey offers one monitoring model that identifies leading indicators related to the quality of professional learning, professional capital, and leadership. An excerpt from the 2018 international TALIS survey of educators is included in Resource 4.1: How LEAs monitor and support Title IIA-funded activities.