After a pilot study, compare the proposed rollout conditions with the conditions in your completed pilot. England's rental e-scooter trials began in July 2020. The Department for Transport's December 2022 response welcomed evidence of useful journeys while identifying safety concerns and changes already made. The immediate task is to decide which expansion the evidence supports before approving a wider service.

A pilot study tests a proposed activity on a limited scale to learn whether and how it can be delivered more widely.

England’s e-scooter trials tested a bounded service

The Department for Transport commissioned an independent evaluation of rental e-scooter trials in England. The work examined who used the service and its wider effects. In its December 2022 response, the department reported 14.5 million rental trips by December 2021. It welcomed findings about commuting and changing travel choices.

That volume established substantial use of the service. It did not establish that every proposed extension would deliver acceptable outcomes. The same response recorded concerns about collisions and pavement riding. It also noted that users' experience complicated comparisons with cycling. Evidence of demand cannot settle a separate question about harm. A sponsor needs to keep those judgments separate when reading a positive summary.

The department reported changes already made through trial guidance and regulation. These included mandatory unique identification numbers and stronger encouragement for operators to provide helmets. The response therefore described both useful findings and changes to delivery. It was a dated policy response, not randomized evidence of a commercial service's effectiveness. It does not establish present legal status or a later national rollout.

The evaluation's delivery chapter makes the operating arrangements visible. Interviews with local stakeholders described the importance of partnerships and clear responsibilities. Local authorities and operators used information about their areas to change the service. In the West Midlands, parking moved from a free-floating approach to designated bays following concerns about where scooters were left.

These findings came from stakeholder interviews, rather than a controlled test of each management practice. They still identify something a rollout proposal must address: the people and arrangements that kept the service operating. The service being assessed includes the work needed to manage it. Removing that work creates a different proposition, even if the equipment stays the same.

For a business sponsor, the useful question is what would happen when the pilot team leaves. Does the permanent team have the authority to change a rule? Can it resolve a complaint without asking the project leader? This is a proposed application of the case, not a claim that the transport evaluation used a particular decision method. Compare those duties with the staffing and budget in the expansion request.

A completed pilot report should preserve changes made during delivery. Otherwise, its final result can be mistaken for evidence about the original design. Record which version produced the outcome being cited. Then check whether the proposed rollout includes that version. This goes beyond the early prototype question in a design sprint: an operating service also needs a sustainable support model.

Before relying on a successful operating period, ask who provided help during it. Read the incident log alongside the outcome summary. Look for extra shifts or manual corrections that the permanent service will lose. These are proposed checks for your own pilot, not findings about hidden work in the e-scooter trials. Their purpose is to identify the conditions your evidence actually covers.

After a pilot study, check whether the normal team can sustain the required result before expansion
The comparison separates observed pilot results from the evidence needed for normal operation.Click to expand

The case supports a limited conclusion: evaluation can justify changes as well as further use. A pilot can justify a changed next step without justifying the whole rollout. For your decision, state which finding supports the next commitment and which unresolved condition limits it. That distinction belongs in the approval request, where the sponsor can act on it.

Use the Walk to decide which expansion your pilot evidence can support. Start the Walk →

What does the pilot establish about wider use?

A pilot makes a proposed service concrete. Staff attempt the work, participants encounter it, and the team records what happens. This is valuable evidence for data-driven decision making. It can reveal an impractical process before the organization commits to delivering it everywhere. The limit is the claim attached to the result.

NCCIH guidance on research pilots distinguishes feasibility from proof of intervention effects. That guidance concerns research studies, so its statistical advice should not be copied wholesale into an operational rollout. Its useful distinction is between evidence that an approach can be delivered and evidence that it achieves the intended benefit. Check which question your pilot was designed to answer.

The required certainty depends on what the next approval makes difficult to change. A limited extension with retained support is a different decision from replacing an existing service. State the consequence of a wrong conclusion before judging whether the pilot gives enough confidence. This keeps the review focused on information that could change the decision.

The following cards are proposed review categories, not measured results from the transport trials. Use them to label the evidence in your own completed pilot.

Observed
The outcome recorded during the pilot.
Assumes: the record includes unsuccessful attempts.
Supported
The people and help used to achieve it.
Assumes: that support remains available.
Transferable?
The claim made about the wider service.
Assumes: changed conditions will preserve the result.

Check who entered the pilot and who did not. Volunteers may be willing to tolerate difficulties that other users will reject. This is a possibility to investigate, not a reason to dismiss volunteer evidence. A different participant group needs an explicit reason for expecting a similar result. Compare eligibility and actual participation with the people the expansion will serve.

Also inspect what the pilot counted as success. Completion can be useful without establishing the benefit the sponsor wants. The government's process evaluation guidance explains why delivery and context matter when interpreting outcomes. Ask whether the reported result depends on a delivery condition the expansion request leaves out.

Compare trial support with the proposed rollout

Bring the pilot report and the expansion request to the same review. Beside each material outcome, write what changes under the proposed service. Include the support that withdraws when the project ends. Approve only the commitment for which the evidence is sufficient. A concern about one condition should lead to a check of that condition, rather than an automatic repeat of the entire pilot.

The following fork is a proposed decision aid. It shows what happens when the same positive pilot result supports different commitments.

Pilot meets its stated operating target
Proposed choices
Expand as proposed
Approve the wider service using the pilot result.
Assumes: normal staffing can sustain that result.
Check changed support
Test the handover before increasing the commitment.
Tests: whether the permanent team can deliver without pilot-only help.

Write the unresolved issue as an answerable question. For example, can the permanent team handle requests without the pilot manager intervening? This is an illustrative question, not a result from a real project. Review the handover record for evidence before commissioning another trial. If no record exists, arrange a limited handover check under the staffing proposed for expansion. State what result would support proceeding and what result would require a change.

The Universal Decision-Making Method provides a sequence for this review. Apply it to the expansion decision rather than using the pilot's completion as permission to proceed.

  1. Frame: State the outcome the expansion must achieve and the people it must serve.
  2. Tentative Elements: Describe the proposed expansion, including the option of retaining a smaller service.
  3. Assumptions: Identify which changed operating condition could prevent the required outcome.
  4. Sufficient Certainty: Decide what evidence about that condition is enough for the commitment being considered.
  5. Implement and Monitor: Name who checks the condition and what finding requires another decision.

Check operational readiness separately from technical feasibility. Working equipment does not establish that the permanent team can manage the service. If the handover remains uncertain, specify the evidence needed before expansion. If that evidence is already adequate, record why further testing would not change the choice.

The approval should name the condition that would require another decision. Put that condition in the operating handover and assign the check to someone who stays. Use monitoring tied to the decision to detect a loss of the support the result depended on. The completed pilot then supports a defined commitment with a clear limit.

How confident are you that the permanent service can sustain the result achieved with pilot support?

Work through your decision

No sign-up. Just pick your decision and start.


Grant Purdy is the co-author, with Roger Estall, of Deciding (2020), and the architect of the Universal Decision-Making Method.