Glass to Silicon - Optics
New month, new topic! 🙂
I did some thinking about this around the same time as the SPIRIT series, but decided to start with that first, which also then lead to the previous series.
The intersection of software developers who dabble in photography (or photographers who knows software development) has to be significant, and yet I don't see much discourse about this. 🤔
Out-or-in
The modern photography hardware has two approaches to capture a subject:
Capture everything at high resolution and crop
- Phones' quad-Bayer main sensors, say focal length of 24mm (35mm equivalent of course - will skip this onwards) at 200MP.
Tighter framing with fewer, higher quality pixels
- Phones' non-quad-Bayer telephoto sensors, focal length of 100mm with just 12MP resolution.
Some phones keep ultrawides over the telephotos, probably because social group settings are more common and it's a simpler, cheaper hardware after all.
Digital zoom is increasingly replaced by 'AI zoom' and both are poor substitutes for the right sensor-lens combination: either inventing pixels or guessing them wrongly.
It's a similar choice when it comes to coding agents and their context selection:
Ingest broadly and focus afterwards
Working understanding of complete code repositories; doubly good for monorepos perhaps.
Pro: Any scoping decisions run less risk of excluding crucial files, changing the context reduces new retrieval round trips.
Con: Crop-in doesn't recover perspective; a repo-wide skim-then-focus builds a different context than an intentional, calibrated read. See next section on bokeh.
Con: Tinier pixels gather less light each, noise and per-pixel dynamic range weakens - comparable to attention dilution, or the lost-in-middle syndrome at scale.
Retrieve exactly what the task requires
Deep-cuts of specific submodules and services, detailed reasoning of not just the how (code) but the why (history).
Pro: The intentional, calibrated read yields better context framing.
Con: Framing must be decided before the shot, a wrong scope and the cost of doing retrieval round trips will start to escalate.
When needed, modern photography hardware can pixel-bin high-resolution into right-resolution with the right fidelity, which is basically compaction at that point. What's similar to both is also the potential token churn: Skim-then-focus-again, or repeated adjacent look ups.
Bokeh
To produce pleasing bokeh, there are once again two approaches:
Using telephoto lenses - having a wide aperture matters less here due to the inherent background separation.
Background is also magnified due to lens compression, making out-of-focus areas larger and more blended.
Aperture then controls the quality of the blur.
Using a wide aperture on wider lenses - the wider aperture becomes the factor affecting the amount of background blur.
A wide aperture reduces the depth-of-field, an effect that will seem more pronounced at wider focal lengths.
There is little lens compression to work with, so the background separation is more reliant on the depth-of-field control.
Think of the background blur - the out-of-focus areas - as the parts where the context is treated to be less important. In that lens (pun intended), telephoto lens - and tightly bounded agents - can make precise code changes without distraction by irrelevant libraries or cause unintended regressions. Agents with broader context - a deeper depth-of-field - keep more of the code repository in focus.
Gear Acquisition Syndrome
An often-discussed subject, some photographers build up a collection (sometimes significant) of lenses that they think they will use, eventually. The other group seek to suppress this gear acquisition syndrome by sticking with a few favorites that they eventually gain mastery with - say with a bag of primes or just one well-balanced superzoom lens.
In the agentic context, stronger agents can be calibrated according to the task - a 'prime lens' for that one specific well-described bug, or the 'superzoom' that gives them a broad base for understanding system architecture. Using the same 'prime lens' agent might not be suitable for a broad understanding of adjacent services though.
We will look at the lens kit (not kit lens hehehe), next.
AI disclaimer: Have bounced ideas off, so while text is still my own, the way I arrived at some of them may contain minor influences from such interactions.