How I Find Problems to Solve as a Staff Engineer

Note: this post was revised after publishing for increased clarity, based on
reader feedback.

“How do you find problems worth working on?” a senior engineer I mentor asked me
recently. He’s trying to make the jump to staff engineer and realized that the
role isn’t just about doing the work he’s assigned. He also needs to get
involved in figuring out what his team and org should be building.

Someone else had suggested blocking out time in his calendar to think about the
bigger picture. He’d tried that, but hadn’t found it productive, so he asked if
I had any alternatives.

I told him I rarely find good problems by staring at a blank page and trying to
“think strategically.” Instead, I act like a sponge. I listen to the stream of
day-to-day noise, absorb the problems people are having and let them sit in the
back of my mind. Over time, some fade away while connections begin to appear
between others that initially seemed unrelated. Eventually, I start to see
what’s really slowing people down and what my team or I can do about it.

I’ve worked with many engineers who’ve never really tried this. They wait for
managers or leads to identify opportunities, then demonstrate their value by
solving the hardest assigned problems. That can absolutely lead to promotion.
But the projects that have made the biggest impression in my career were the
ones where I found and solved an important problem my leaders did not yet
realize existed.

One caveat: my experience comes mainly from working on infrastructure and
developer tools at large companies, on teams where engineers have a lot of
bottom-up autonomy to influence their roadmaps. In a more top-down environment,
there may simply be less room to work this way.

Absorb problems, not requests#

People love talking about the problems they are facing: in meetings, chat
threads, presentations and email. They explain why their work is hard, complain
about what slows them down and describe what they wish they could do.

When something overlaps with my area, I start pulling on the thread. I might
ask, “If X existed, would it solve your problem?” or point them at an existing
feature in a product I own and ask how much of their use case it covers.

Users often
ask for a particular solution
instead of explaining their root issue. Rather than taking the request at face
value, I keep digging until I understand what they are trying to accomplish and
why existing products do not work for them.

As a natural introvert, this sort of ambient listening works particularly well
for me. I don’t need to fill my calendar with speculative meetings just to find
ideas; there is already an enormous amount of useful information flowing around
me during a normal week.

When a problem seems worth exploring, though, I become more active; I need to
see how it affects the team’s day-to-day work. I’ll sit with them as they walk
me through their workflows and the bugs they’re investigating. When I can, I’ll
try working through some of those bugs myself. Seeing the problem firsthand
makes it easier to separate what the team actually needs from the solution they
asked for.

I also seek out people who see more of the organization than I do: those who own
critical systems, work across several teams or have particularly deep insight
into the work downstream of my team. I’ll arrange a 1:1 or coffee chat and ask
about interesting problems they’ve come across. They may have already seen the
same issue in several places and started connecting the dots, giving me a head
start on patterns I might otherwise have taken much longer to notice.

Let problems accumulate#

Several times, I’ve been burned by moving too fast. I became excited by a
request from a vocal team, built the feature and watched them barely use it.
Their priorities had changed, or the request had come from a one-off
investigation that no longer mattered. How eager a team was in that moment
wasn’t the same as how important the feature was relative to everything else my
product needed to support. By hyperfocusing on their request, I lost sight of
the bigger picture.

That taught me to let potential problems pile up. Listening the way I do leaves
me with far more of them than I could possibly solve, and not all deserve
action. Most don’t need to turn into projects the first time I hear about them;
waiting can be a superpower.

Waiting means the same problem might pop up independently in different teams,
making it a higher priority to solve. Or problems that look different on the
surface might turn out to have the same shape, so I can address several use
cases in one shot. Or, as I’ve learned painfully, the requesting team didn’t
even care that much in the first place.

Instead, I make a mental note and revisit the problem if it comes up again.
Other engineers I know write this sort of thing down more systematically. The
mechanism is a personal choice: everyone has to figure out what works for them.
What matters is keeping unresolved problems around long enough for more evidence
to accumulate.

Find the common shape#

Waiting helps me collect evidence, but that alone doesn’t tell me what to build.
I still need to work out whether the problems I’ve retained are genuinely
related and what, if anything, could address them together.

Perfetto, the performance debugging tool I work on, is a good example. It
displays recordings of system activity on a timeline made up of rows called
“tracks.” Over a couple of years, teams kept asking for small, specific
additions to the UI. One wanted a command to keep their preferred tracks pinned
to the top of the screen; the next team wanted the same, but for a completely
different set of tracks. Others wanted Perfetto to open already zoomed in on a
particular part of a recording, or to show a custom aggregation tuned to what
they cared about. A few had stopped waiting for us and built elaborate
workarounds with bookmarklets.

By the time enough of these had piled up, my head was the usual tangle: the
requests themselves, the constraints on each and a handful of half-formed
solutions. I’ve learned not to force a solution by just sitting at a desk and
thinking. Instead, my best untangling happens on long, aimless walks around
London, where connections come more easily when I’m not trying to force them.

What I eventually realized was that none of these teams really wanted the
specific feature they’d asked for. Each wanted to personalize Perfetto for their
own workflow without imposing their choices on everyone else. The underlying
need wasn’t any one feature but rather the ability to extend the UI. When a
connection like that finally clicks, it’s one of the best feelings in the job:
several awkward requests collapse into a single idea, and possibilities open up
that none of them hinted at on their own.

That feeling, though, is exactly when I have to be careful, because a common
shape is only a hypothesis and elegance is not evidence. When it happened with
extending the UI it turned out to be real, but I’ve been fooled before.

In another recent case I was convinced that building a transparent caching
system for querying Perfetto traces would solve issues with sharing large traces
and repeated queries. It was only as I wrote the RFC and built a prototype that
I realized the elegance was a lie: the two problems wanted genuinely different
solutions. I reluctantly split the design in two, both halves of which have
since shipped.

Pressure-test before building#

You’d think this would be the moment I start building, but it usually isn’t. How
far I go depends on how sure I am that the idea works and that people actually
want it.

If something is useful and low-risk enough, I act straight away: I send the
change and let my manager know. When I’m unsure whether an idea will work or how
much effort it will take, I build a throwaway prototype instead; it exposes the
failure points and gives me something concrete for others to react to. And when
an idea is big but I’m convinced by it, I commit to the full effort: weeks or
months of work and the hard yards of building support across other engineers and
teams.

Through all of it, I’m not only trying to convince other people; I’m also trying
to convince myself. Sometimes the honest answer is to stop: if people don’t see
the value I do, or we hit a major technical wall, I’d rather drop the idea now
than build something no one uses or that becomes a maintenance nightmare. And
sometimes it holds up but the timing is wrong, so I park it, ready to spring
into action the day it becomes an org priority.

When an idea does hold up, I don’t necessarily need to be the person who builds
it. I might implement it, someone else on my team might, or it might change what
the org focuses on. Finding and shaping the right problem can have an impact
even when I don’t own the implementation.

The Perfetto extensions idea was worth that full effort. We were already
building plugins to modularize the UI, but they weren’t enough: teams had to
open source all their plugin code, which wasn’t an option for many internal use
cases. So before building anything new, I took the problem and my proposal to my
manager, teammates and the client teams. I ended up writing two RFCs, having
several 1:1s and giving a couple of talks, refining it as the feedback came in.

In the end, I designed and implemented
macros
as “lightweight extensions”: a way to automate actions in the UI without writing
a plugin.
Extension servers
took the idea further by letting teams share their macros.

Instead of implementing every requested feature ourselves, we gave teams ways to
adapt Perfetto to their own needs. Dozens of teams inside Google now use macros
and extension servers, and several other companies use extension servers
internally too.

Solving useful problems helps me find the next one#

The more often I go through this process, the easier it becomes. When I show
genuine interest in someone’s problem, ask useful questions or help solve it,
they remember. They start coming to me earlier and bring me into conversations
with other people facing related issues.

That gives me a wider view of what is happening across the organization, making
it easier to spot patterns and build things people actually need. Solving one of
those problems brings me into more conversations, and the loop continues.

Those successes build the kind of trust that comes from
long-term stewardship.
Early on, I had to turn many of these ideas into something real myself to prove
that my judgment was sound. Over time, my manager and org gave more weight to my
assessment of what mattered. That allowed me to influence the roadmap without
needing to own every project.

This differs from the idea that becoming a staff engineer means replacing
technical work with meetings and coordination. For me, conversations are inputs
into what I build, not the end result.

Conclusion#

That is what I wanted my mentee to understand: finding problems worth solving
isn’t separate from the rest of the job. It comes from staying engaged with
people’s work long enough to see what no single request can show you.