How Sample Scout is tested

Sample Scout works on Logic projects that represent months of somebody's work. That sets a higher bar than "it seems to run". This is how I put it through its paces, and why the testing is built to try to prove the app wrong rather than prove it right.

The rule everything else hangs off

Your original project is never written to. Every repair and every consolidation happens on a copy, and the copy is what you open. If Sample Scout gets something wrong, the worst case is a copy you delete.

That is not a claim I make and hope for. It is checked mechanically on every release across every code path that touches a file, and it is the first thing any change is measured against.

Testing against damage on purpose

A project that will not open is the case that matters most, and it is not one you can wait around for. So I build broken projects deliberately, in every distinct way I have seen a real one break, and keep them as a permanent set.

Testing missing samples honestly

Finding a missing sample is easy to fake. Point a test at a file that is sitting somewhere obvious and it passes while proving nothing. Each case is therefore built so it can only pass one way:

• Moved — the file still exists, somewhere else entirely.
• Renamed — the file is there but no longer called what the project expects.
• Ambiguous — several files share a name and are genuinely different audio. The app must not choose for you.
• Gone — the file does not exist anywhere. The only honest answer is to say so.
• Cloud — the file exists but has not been downloaded yet.
• Many at once — whole folders of projects, processed unattended.

Isolation, so a test cannot cheat

Because Sample Scout searches your whole Mac, a test project can accidentally find a copy of a sample belonging to a different test, and pass for the wrong reason. I measured this: nearly every filename in the test set also exists somewhere else on a working machine.

So the suite runs sealed off from the rest of the computer, verified in both directions before each run. A test that says a sample was found has genuinely found the right one.

Three ways every change is checked

Saying only what is true

A repair tool that overstates what it did is worse than one that does less. A rollback to a backup is not a repair. A run that copied nothing has not consolidated anything. A project I could not read is not a project I have checked.

Several of the fixes I am proudest of changed no behaviour at all. They changed what the app says about itself, so that a green tick always means the same thing.

The app also ships a plain list of what it cannot do, and how to solve those cases yourself. If a limit is real, I would rather tell you before you rely on it.

Tested on real work

Alongside the built cases, Sample Scout is run across my own working library: hundreds of projects going back years, of every size, with the plugin sprawl and moved folders and half-migrated sample libraries that come with real use. Several of the most valuable fixes came from there, because a real 10 GB project exposes things a tidy test project never will.

What it can and cannot do

Lifted straight from the app, so it matches what you get.