After a program evaluation, put the reported effect beside the proposed renewal cost on one page. The 2012 Head Start follow-up found that early benefits and later results differed. Before renewing a program, check whether its promised benefit matches the population and period the evaluation actually examined.
A program evaluation systematically examines a program to assess its operation or results and support decisions about its future.
Head Start findings changed across the follow-up period
The completed report needs to be read against the renewal proposal. Start with the exact result that the sponsor wants to buy again. Is the purpose an immediate improvement, a lasting change, or access to a service? Those purposes require different evidence. A report can support one purpose while leaving another unresolved. Changing the purpose after reading the results conceals that difference. When an evaluation supports an accreditation claim, the self-study follow-up also checks what must be operating now and what depends on future work.
The 2012 Head Start third-grade follow-up report illustrates why the time period matters. Children were randomly assigned to access to Head Start or to a control group. The control group could use other early childhood services. The study found initial benefits, but few effects remained by the end of third grade across the full samples. Some subgroup findings differed.
The comparison therefore concerned access to Head Start against the alternatives available to those families. It was not a comparison against every child receiving no early education. Nor did the later results establish that the entire program was ineffective. The report examined particular cohorts and measured outcomes over a stated period. Its conclusions need those limits attached.
The relevant question for a sponsor is whether the proposed renewal requires effects that last that long. If the purpose is early school readiness, the early findings matter. If the renewal promises a sustained later advantage, early findings alone cannot establish that promise. Neither conclusion should be selected merely because it makes the funding recommendation easier to defend.
Choose the outcome and follow-up period that the renewal decision actually depends on. Then return to the report and find the estimate that addresses them. Record its uncertainty beside it. Keep the population and comparison conditions on the same page. A summary that omits these details can turn a limited finding into an unsupported claim about everyone served.
An average also needs a denominator. Check who was eligible, who entered the study and who remained available at follow-up. Ask the evaluator whether missing observations could alter the conclusion. Do not substitute participant satisfaction for an effect estimate. Satisfaction may answer a useful question, but it cannot silently replace the outcome the sponsor promised to achieve.
For an internal renewal review, copy the relevant result into a short evidence note. Under it, write what the finding does not establish. This makes the boundary visible to people who have not read the technical appendix. The discipline also helps when reading a completed systematic review: evidence needs to match the question being decided.
A finding can be credible and still be too narrow to justify the proposed commitment. The response is to adjust the commitment or obtain the missing evidence. It is not to dismiss the study because it is inconvenient. Where the proposed population differs, name that difference explicitly before transferring the result.

Read the follow-up horizon against the sponsor’s promise. A renewal paper should retain the early result where it matters, while identifying the later effect that remains unsupported. This lets the sponsor consider a narrower purpose without claiming the original longer-term promise has been established. Keep that change visible in the recommendation.
Your evaluation measured past results; the next funding decision needs a clear reason to expect enough benefit under the renewal terms. Start the Walk →
Does the evidence support the proposed renewal?
A useful evaluation can show whether delivery followed the plan and whether intended outcomes occurred. CDC distinguishes outcome evaluation from impact evaluation: observing an outcome does not by itself show that the program caused it. Read the design before treating a change as an effect attributable to the service.
This is where the completed theory of change can help. Find the connection between the service delivered and the outcome being claimed. Check which parts the evaluation examined. A sound account of delivery may explain why participants attended. It may leave the effect of attendance unresolved. Keep those two conclusions separate in the renewal paper.
Now compare the evaluated program with the proposed operating terms. Staff capacity may change. Eligibility may widen. The provider may propose a different delivery format. Mark each material change and ask what supports carrying the result across it. A renewal at a different scale is a new commitment, even when the program name stays the same.
Renewal needs an acceptable relationship between expected benefit and future cost. Include the resources needed to deliver the evaluated service properly. Excluding supervision or follow-up can make a cheaper proposal look equivalent when it is materially different. Assess future costs rather than using money already spent as a reason to continue.
Check whether the cost estimate covers the same intensity of service as the evaluation. A lower price can depend on shorter sessions or fewer trained staff. Ask the provider to explain any difference before treating the measured effect as the expected effect. Where the proposed service changes materially, describe renewal as a changed option and assess it on that basis.
The next funding period also has alternatives. Compare renewal unchanged with a targeted change or an orderly closure. Include what happens to participants if the service stops. A positive result does not automatically make renewal the best use of available resources. A weak result does not remove the need to manage a responsible exit.
The CDC evaluation framework includes acting on findings. That action needs a named user and a decision to inform. In this case, the sponsor must say what evidence would justify renewal. This connects evaluation to data-driven decision making without treating the evaluator as the person who authorises funding.
Set the evidence threshold before renewing the program
Write the renewal decision in one sentence. State the eligible population and the outcome sought during the next funded period. Then identify the finding that most affects the choice. This creates a clear point of comparison. The task differs from monitoring an approved business case, because the sponsor is deciding again after completed evaluation evidence is available.
Set the required confidence according to the consequences of being wrong. A short extension with a controlled cost may justify acting with unresolved questions. A long contract that displaces an essential service may require stronger evidence. Explain which uncertainties matter to this commitment. Do not demand complete certainty about every question the program raises.
A weak evaluation design can leave effectiveness unknown. Closing a service on that basis still requires a decision about unmet need and available alternatives.
Ask for additional work only when its answer could change the choice. Specify the missing question and who will resolve it. A request to expand the report without a decision question can consume the renewal window. If the answer cannot arrive in time, compare a limited extension with the consequences of waiting. Record why the selected option is acceptable.
The Universal Decision-Making Method places sufficient certainty within a decision about purpose and available options. Here, apply it to the actual funding terms. Keep one evidence note with the renewal recommendation. Show the result relied on, its limits and the cost of the proposed service. Make any conditions of approval visible to the person who will sign.
Record what would make the renewal decision change. This might be a failure to reach the eligible group or a change in the cost required to deliver the service. Use a signal that can be observed during the funded period. Give someone authority to return the decision for review before the next automatic renewal date.
The final recommendation should explain why the evidence is sufficient for the chosen option. It should also identify the remaining uncertainty the sponsor accepts. This is the practical stopping rule in evidence-based decision making. The report remains evidence about what happened. The signed renewal record explains why that evidence justifies what happens next.
How much evidence does this program need before another funding period is justified?
Work through your decisionNo sign-up. Just pick your decision and start.
Grant Purdy is the co-author, with Roger Estall, of Deciding (2020), and the architect of the Universal Decision-Making Method.