How a photo moves through the pipeline

A Tauri shell hosts a React UI and spawns a bundled FastAPI server bound to 127.0.0.1, guarded by a per-install X-Local-Token header. The UI never touches the filesystem itself: it posts a job config and then watches an SSE stream for progress. Exactly one job runs at a time — a second request is rejected rather than queued, because both would be mutating the same tree.

File discovery is os.walk wrapped in walk_ignoring(). Two details make it correct rather than merely working.

Pruning, not filtering. Ignored subtrees are removed by slice-assigning dirnames[:], so the walk never descends. A naive if dirpath in ignore_set: continue excludes nothing useful here — sorted output always nests, so an ignored folder holds zero direct files while its children's paths are absent from the set. Rebinding the name instead of slice-assigning is silently a no-op, since os.walk reads back that same list object.

The self-ingest guard. effective_ignore_set() adds the destination to the ignore set whenever it nests strictly inside the source. Without it, sorting into a subfolder of the source makes the next run read its own output. It is deliberately not added when dest == source, which would exclude everything.

Admission is a suffix test against one canonical tuple — f.lower().endswith(SUPPORTED_EXTENSIONS), where the tuple form lets endswith test all 19 at once. It lives in face_engine so the sorter and the enroller cannot drift apart; when they did, enrolment accepted only JPEG and PNG, so a user enrolling HEIC selfies got an empty gallery and no explanation.

Decode is a separate concern one rung later, and it is layered: Pillow for the standard formats, pillow-heif registered as an opener for HEIC/HEIF, and rawpy with a Wand/ImageMagick fallback for RAW. Every loader is imported lazily and degrades to a logged skip, so a missing optional dependency disables one format rather than the backend.

Image.open() does not apply the EXIF Orientation tag, so a portrait JPEG arrives at the detector on its side. ImageOps.exif_transpose() runs first, before convert('RGB') — the order matters, because transpose reads from image.info. Measured on 35 real enrolment photos: 33 of 35 faces detected upright, 1 of 35 with the same photos rotated 90°.

EXIF comes from Image._getexif(), with PIL.ExifTags.TAGS and GPSTAGS mapping numeric IDs to names. Byte-valued tags are decoded with errors='replace' and stripped of NULs rather than being allowed to raise.

Location scanning is gated behind the sort method: loading the geocoder unconditionally was measured taking file locks that affected unrelated sort types. The country segment is dropped from the output path when every photo resolves to a single country — Uttar-Pradesh/Lucknow/ rather than a pointless IN/ root.

This is the piece most worth understanding. Face detection is not one operation with a quality dial — it is an ordered list of increasingly expensive attempts, and each photo exits at the first pass that returns an encoding. An ordinary photo pays for pass 1 only; the escalation cost falls entirely on the photos that would otherwise have been filed as faceless. The three modes are prefixes of the same list, not different algorithms.

dlib's CNN allocates per-image buffers proportional to pixel count and offers no way to cap that from Python, so concurrency is the only available lever. Parallel CNN detections were measured exhausting an 8 GB machine, and an OOM kill loses the entire job — strictly worse than the job being slow. The cap is deliberately a fixed number rather than a memory heuristic, and is marked in the source as a known ceiling: a 64 GB box and an 8 GB box both land on 2.

face_recognition is a thin Python convenience layer. The work is dlib, a C++ computer-vision library exposed through pybind11 — which is why installation compiles from source when no wheel matches, and why the heavy paths are not bound by the GIL the way pure-Python code would be.

• HOG — Histogram of Oriented Gradients plus a linear SVM over a sliding window (Dalal & Triggs, 2005). Gradient orientations are binned per cell into a descriptor robust to lighting but not to rotation. Cheap, CPU-only, and the reason passes 3 and 4 exist.
• CNN — dlib's MMOD (Max-Margin Object Detection) detector. Substantially better on small, angled and poorly-lit faces; expects CUDA, and its memory scales with input pixels.
• The encoding — dlib crops and aligns the detected face to a 150×150 chip and runs a ResNet that emits 128 floats. Same identity ⇒ small Euclidean distance. Recognition reduces to nearest-neighbour search in a 128-dimensional space.

Matching is one vectorised operation per photo rather than per face:

dlib recommends 0.6. This uses 0.55, and the number survived scrutiny for a specific reason: measured against a cleanly rebuilt gallery, 0% of same-person pairs exceed it. It only looked too tight against the old gallery, where 12.6% did — the fix was the gallery, not the threshold. A runner-up margin test and per-person distance aggregation were both implemented, measured, and removed as neutral-or-worse.

Enrolment runs once per person over a handful of photos, so it can afford the expensive detector. It uses CNN with an upsampled HOG fallback — 3 of the 35 measured enrolment photos are undetectable by HOG at any scale and would silently drop out of the gallery otherwise. Detection runs on an 800px copy and the box is scaled back into a 1600px array for encoding, with num_jitters=10.

Three guards keep one bad sample from poisoning the gallery, because nearest-neighbour matching means a single wrong face is worse than a missing one:

• Largest face wins. The previous code took face_encodings(...)[0] — dlib's arbitrary order. On a two-person enrolment photo that files a stranger under the subject's name.
• Ambiguity is refused, not guessed. A second face at ≥60% of the subject's area means the folder name is unresolvable, so the photo is skipped with a message naming the fix.
• Outliers are pruned. Per person, mean pairwise distance with a median + 2×MAD threshold, floored at 0.60. Needs ≥3 samples to mean anything.

Self-consistency cannot catch one person enrolled twice under two names — each name's photos agree with themselves perfectly — so that case is detected separately and reported as a warning. Which name to keep is the user's call, so nothing is deleted.

Recognition used to run inline, one photo at a time, on a single core — the reason a People sort over a large library took hours. It is also pure compute with no shared state, which makes it the one part of the loop that parallelises cleanly.

So it was lifted into a single pass that completes before any file operation begins. That ordering is doing real work: the pool lives and dies inside one function, so an abort or an error tears the workers down there rather than orphaning them mid-move — and because nothing has been written yet, aborting during the expensive phase needs no rollback at all.

A SQLite table keyed (path, engine_version), validated against (file_size, file_mtime). Two decisions in it are worth stating.

It caches encodings, not names. Names depend on who is enrolled and go stale the moment someone is added. Matching an encoding against the gallery is a single numpy operation, so it is re-run every time and stays correct for free.

A cheap miss is not evidence for an expensive question. A "no face" result from a fast scan must not satisfy an accurate request. The row records passes_run, and reuse is gated on depth:

That comparison is only sound because the ladders are strict prefixes of one another — fast ⊂ balanced ⊂ accurate. The bracket structure in §05 is not presentation; it is the invariant this rule depends on. Bumping ENGINE_VERSION invalidates every row rather than trusting encodings from an older recipe, and only the parent process touches the cache, so there is a single writer and no lock contention.

The write is shutil.copy2 or shutil.move into a path assembled with os.path.join, followed by os.utime() stamping the destination with its EXIF date so the file manager's own sort agrees with the folder structure. Name collisions are resolved by numbering, never by overwriting. Move mode branches on st_dev, and the two branches have genuinely different safety properties:

A People sort files one group photo under every person in it, so a photo with four enrolled faces is written four times — and in copy mode the originals stay too. On a nearly-full volume that is the difference between the feature being usable and not.

Every figure below came from a measurement on real photos, and each drove a decision that is still in the code.

A folder scan is os.walk pruned by a path-prefix ignore set; files are admitted by extension against one canonical tuple, then decoded by Pillow, with pillow-heif for HEIC and rawpy/Wand for RAW. Pillow's _getexif yields the shutter date and GPS DMS, which is converted to decimal degrees and resolved to Country/State/City by an offline k-d tree reverse geocoder — no network. Destination paths are built with os.path.join, then shutil.copy2/shutil.move writes the file, os.utime back-dates it, and on APFS clonefile(2) via ctypes shares blocks so fan-out copies cost no extra disk.

People sorting runs a separate pipeline: enrolled faces are encoded once into a versioned 128-D gallery, then every source photo goes through one parallel Pool pass that completes before any file is touched, so an abort mid-scan needs no rollback. Each photo climbs a detection ladder — HOG at 1024, HOG at 1600 upsampled, ±90° rotations, then dlib's MMOD CNN — and exits at the first pass returning an encoding; fast/balanced/accurate select ladder depth and nothing else. Matching is nearest-neighbour Euclidean distance under a 0.55 tolerance. Results are cached in SQLite keyed on (path, engine_version), invalidated by (size, mtime), storing encodings rather than names — because names go stale whenever someone is enrolled, and re-deriving them is one numpy operation.