A recurring failure deserves an explanation that crosses the layers.
The visible error may be the point where tolerance ran out. The cause may sit elsewhere: in an interaction, assumption, resource, dependency, or condition nobody has examined together.
Repeatedly repairing the symptom can leave the operating pattern intact.
A useful diagnostic effort makes the next observation more informative than the last.
Separate what happened from why people think it happened.
An incident generates observations and interpretations. Both matter, but they do different work.
What was actually seen? Which explanation accounts for it? What else would be expected if that explanation were correct? What observation would challenge it?
These questions help an investigation move beyond an expanding collection of plausible remedies.
Follow the behavior across boundaries.
Concurrency, caching, communication, resource ownership, and recovery can interact. A team organized around components may need an investigation organized around behavior.
Howard Pearlmutter combines hands-on development with architecture and systems perspective. He can examine several layers and connect the finding with the operating result the organization needs.
In an ecommerce and distribution environment, he resolved intermittent degradation and crashes involving a mission-critical distributed caching grid. The problem had persisted for nine months before his arrival. Read the cache case.
Make the system easier to question.
Instrumentation and diagnostic tools are useful when they distinguish among explanations. More data is not automatically more understanding.
Howard has also architected and implemented a diagnostic framework for a video-streaming network, including client-side agents, a browser-based administrative console, server-side services, and persistence.
The tool and the investigation can inform one another: what does the team need to observe in order to understand the behavior?
Connect resolution with future architecture where useful.
An incident may reveal an architectural constraint. It may also reveal a gap in the team's shared model or diagnostic capability.
A bounded investigation can identify the next useful step. Further architecture or implementation should follow what the findings justify.
Explore architecture assessment, qualification, and technical transfer. Start with the recurring pattern and the point where the current explanation stops being convincing.
