This is a guest post by Alexis and Adrienne, former PMs (Facebook, Tesla, and Google) who share insights and best practices for working at top companies in Silicon Valley.

Product execution questions are one of the most popular questions  you’ll encounter during PM interviews, including interviews for  Facebook.  Even if you're interviewing for an APM position in your early product management career path, you'll likely still have to answer execution questions!

These questions aim to evaluate how you measure success, and  distill user problems into a metric that a team can rally around. These  questions appear easy at first, but in reality, are challenging because  the question isn’t about naming metrics - it’s about first taking an  educated guess at the north star goal for the team and quantifying it  with a metric that expresses it succinctly.

There are three main execution question types:

1. Success: How would you measure success for X product?
2. Debug: You’re the PM for X. Y metric dropped by Z%. How would you figure out what happened?
3. Trade-off: Y metric went up but Z metric went down. What do you do?

These Success, Debug, and Tradeoff questions closely mirror the critical decisions PMs are accountable for  on a regular basis. For this memo, we’ll focus on defining success  metrics and cover the others in future memos.

Here are three things interviewers are looking for that you can focus on to improve your interview performance:

1.  It’s not a math problem, it’s a social science problem.

It’s  easy to fall into a trap and treat it like a math problem to be solved.  A candidate might believe there is a correct metric the interviewer is  looking for, and try listing every single aspect of the problem, leading  to rabbit holes. Your metrics will differ, however, depending on the  goals you define and the assumptions you make. The interviewer is  looking for your ability to define a reasonable north star goal for a  product, and suggest a metric that is aligned with the goal.

Example: Once, I was defining success metrics for a language learning app in an  interview, and I suggested weekly active users (WAUs) as our goal  metric. The interviewer pushed back on this choice.

• Interviewer’s response: Why weekly, and not daily? If you’re trying to learn a new language, you would practice daily.
• My response: Using a language  learning app daily would be ambitious for your average user, especially  when competing with apps such as Instagram and TikTok, and that daily  wouldn’t capture the fluctuations in people who use it over the course  of a week — for example, people who only have time to learn a new  language on weekends.

My interviewer agreed with my justification. I could have, however, easily gone the other route, and responded with:

Actually,  you’re right. Our goal should be to make language learning a daily  habit, so I’m going to change the measurement cadence from weekly to  daily and measure daily active users (DAUs) instead.

This is why it’s important to be really clear up front about what the  product goals are, and the assumptions you are making. Here’s an  example on how to do that:

Example: Define success for a language learning app.

2. The interviewer is not looking for a long list of metrics, but how you form an opinion on the few metrics that matter most.

When interviewees are asked to say, Define success for Bumble (an online matchmaking app),  candidates will often list 10+ metrics, worried about covering all  aspects of the problem. In this case, however, less is more. It’s more  interesting to have a discussion on 5-6 metrics than 20-30 metrics.  Then, it’s best to end with an opinion on what the one primary metric  should be, and 2-3 secondary metrics that aid the primary metric in  showing the full picture.

Here’s how to do that in action:

Example: Define success for Bumble.

* 1: Use  percentiles, which are not skewed like averages are. If some of your  users have very fast response times and others have very slow response  times, an average response time would not be representative of your  users' experience from the data. Read more on our previous memo: The Allure and Trap of % Goals.

3. Define the downsides of metrics.

While  an OK candidate might list general downsides to their metrics, and talk  about counter metrics such as cannibalization of the company’s other  products or abusive behavior, the best counter metrics are specific to  the north star metric you’ve defined. It’s about asking: If we were to over-optimize for our north star, what are ways our decision-making would be flawed?

Example: Venmo is launching a debit card. Define success.

Questions are often intentionally vague!

Metrics  questions are hard because you can take them in so many directions, and  interviewers are often intentionally vague. They are seeking to  understand: does this candidate dive into coming up with metrics (OK  interviewee) or do they take the time to define the landscape and the  goals (great interviewee)?

It’s less important that you cover  every single aspect, and more important to define all the pieces you  need to solve the problem. As a product manager, most of the time you’ll  have enough resources and people to help you, so it’s less about  correctness, than about being able to frame what is success to your team  in a way that enables them to frame decisions later on -- and this  involves walking through how you think about success.

To read more on how to ace PM interviews and be at the top of your game as a PM, see more of Alexis and Adrienne’s memos here.