Tools and workflows

Python automation: dry runs and safe retries

Design a Python script that previews proposed changes, detects destination conflicts, and resumes correctly after a partial failure.

Give a Python automation one function to plan changes and another to apply them. In a dry run, print the plan and skip the writes. During a real run, confirm which items are already in the intended state and leave them alone. That makes a retry easier to reason about after an interruption.

Decide what the script should do with duplicates

Imagine organizing finished reports into month folders. Decide which field identifies a finished report and which date determines the folder. Try two distinct reports named march.txt before you run the job. If both land at 2026-03/march.txt, the plan needs a stable way to distinguish them or a clear reason to skip them.

Build a plan with source, destination, and reason for each proposed item. Keep rejected items visible. A good preview says that a report was skipped because its date could not be verified. “Moved zero files” leaves the operator guessing whether there was no work, bad input, or silent refusal to act.

Make one code path produce the plan

Offer a dry run that uses the real selection and naming logic. If dry run and apply independently recalculate the work with different rules, the preview cannot be trusted. Preflight the same destination checks for both. The apply phase still needs to recheck important conditions immediately before each write, because someone can change a directory between preview and execution.

A clean design can have a plan function that accepts observed inputs and an apply function that uses the resulting plan. Keep file operations explicit. Work in a temporary test directory with made-up files until you know what survives failures. An automation's first run should not have to be the first time you learn whether it overwrites a name.

Define a safe repeated run

Idempotence means repeating a completed operation preserves the intended result. It is possible when the work has a stable identity and can tell what already happened. A produced output named only with the current time will look new on every run. A result derived from a stable input identity can be checked before creating a second copy.

Do not use an existing destination filename as proof that it contains the correct item. Compare suitable evidence, such as the input ID or the required file properties. If the destination conflicts with the plan, stop or ask a human to resolve it rather than silently treating the conflict as complete.

Decide what a user should see before approving work

An informative preview might say that four eligible reports will be placed in specific month folders, one is already present, and one cannot be classified. If the user runs the preview again on the same input, it should reach the same selection. This is easier when the program sorts items deterministically and records their stable identifiers instead of using a clock value to decide the names on every run.

A preview should account for whether changes can be reversed. Creating a distinct output is a simpler operation to review than renaming or erasing originals. If the desired action is irreversible, design a stronger authorization step and test a restore plan before applying it to real inputs.

Define a small fixture with realistic conflicts

Use a temporary directory for file tests. Create one new report, one already in its intended location, and one whose destination contains a different report. The first should be ready to apply, the second already complete, and the last blocked with both paths visible.

Check the difference between identical content and identical intent. Two source files can contain the same bytes but represent distinct records a reporting team needs. Conversely, the same work item can be encoded slightly differently by two runs. Use the identifier and equality check appropriate to the real outcome.

Test a partial failure and a second run

Interrupt the test after its first successful change and before the next one. Ask the program to produce a fresh plan. It should identify what is already settled and what still needs attention. If a step might have happened before the program recorded it, reconcile the actual state before retrying.

Record enough information to reconstruct the outcome without copying unnecessary private data. A useful run report counts completed, already completed, skipped, and failed items separately. Give an operator an actionable error when intervention is needed, such as a conflicting destination or an unreadable source. These details make a repeated job understandable to a person who did not write it.

Sources and further reading

Related reading

Browse Tools and workflows · Explore project tutorials

Search PythonProjects.org