metrics product sense - sophiekeke/casestudy GitHub Wiki

https://www.1point3acres.com/bbs/thread-679303-1-1.html

Product Sense/Metrics in Interviews

Product sense and metrics are critical topics in interviews, particularly at top companies. These can be challenging for new graduates or career switchers without practical work experience. However, with proper preparation and research, these questions can be effectively addressed. The key lies in summarizing types and using standard methodologies. Here are some types I've learned and the methodologies associated with them. Contributions to expand these are welcome.

Type 1: Define Metrics

Common Questions:

  • How do you compare the new feature vs. the old feature?
  • How can we test if a feature can increase user engagement?

Methodology:

  • Start by clarifying the purpose of the feature.
  • List common metrics:
    • DAU (Daily Active Users), MAU (Monthly Active Users)
    • Total time users spent with the old feature compared with the new feature
    • Activities users performed (e.g., messages sent)
    • Revenue
    • CTR (Click-Through Rate) and number of impressions

Type 2: Feature Success

Common Questions:

  • How do you measure the success of a new product?
  • How would you evaluate a feature?

Methodology:

  • This is related to defining metrics. Begin by clarifying the purpose of the feature.
  • Define several metrics to measure success.
  • Include a guardrail metric that should not decline or fall below a certain threshold.
  • Conduct experiments and analyze the results to determine whether the feature is worth launching under all scenarios.

Type 3: Improve a Product

Common Questions:

  • What feature would you add to WhatsApp?
  • How can you improve Twitter?

Methodology:

  • Similar to assessing feature success.
  • Clarify potential improvements, scope, and purpose.
  • Define metrics to measure success, possibly including a guardrail metric.
  • Conduct A/B testing to see if the defined metrics improve.
  • Summarize and recommend whether the improvement should be launched.

Type 4: Diagnose Problem

Common Questions:

  • Facebook's user engagement drops by x%, what would you do?
  • Lyft's ETA increased by x%, what would you do?

Methodology:

  • Discuss Sudden drop vs. static drop. A sudden drop might be caused by internal or external factors, whereas a static drop might be due to market or seasonal influences.
  • Consider other products. Changes in user behavior might be caused by similar internal products or competitive products from other companies.
  • Segment the product. Discuss which product segments have changed and which have not.
  • Segment users. Discuss which user groups have changed and which have not, e.g., by region, language, platform.
  • Conclude with a summary and recommendation.

YouTube Resources

Here are some YouTube videos for those who prefer visual content over reading. Note that product sense content is rare on YouTube, and most are mock interviews which may not be very helpful. Here are some useful ones I've found:

中文

Product Sense/Metrics in Interviews

Product sense and metrics are critical topics in interviews, particularly at top companies. 对于new grad或者转行的人来说，没有实际工作经验好像就很难回答。其实好好准备，多看多读就没有问题。回答这类问题的关键就在于总结类型和用套路。 Here are some types I've learned and the methodologies associated with them. 欢迎大家补充。

Type 1: Define Metrics

Common Questions:

  • How do you compare the new feature vs. the old feature?
  • How can we test if a feature can increase user engagement?

Methodology:

  • 需要先沟通明确这个feature的目的是什么。
  • List common metrics:
    • DAU (Daily Active Users), MAU (Monthly Active Users)
    • Total time users spent with the old feature compared with the new feature
    • Activities users performed (e.g., messages sent)
    • Revenue
    • CTR (Click-Through Rate) and number of impressions

Type 2: Feature Success

Common Questions:

  • How do you measure the success of a new product?
  • How would you evaluate a feature?

Methodology:

  • 这个题目和define metrics有关。需要先沟通明确这个feature的目的是什么。
  • Define several metrics to measure success.
  • 定义一个guardrail metric。这个metric 不能下降或者低于某个指标。
  • 做实验然后分析结果。套路所有的情况，以及在各种情况下，这个feature是否值得launch。

Type 3: Improve a Product

Common Questions:

  • What feature would you add to WhatsApp?
  • How can you improve Twitter?

Methodology:

  • 这个套路和feature是否成功很类似。
  • 需要先沟通明确有什么可以提升的。提升的空间在哪，目的是什么。
  • Define metrics to measure success, possibly including a guardrail metric.
  • 做实验A/B testing看定义的metrics是否提升。
  • 总结以及推荐这个improvement是否要launch。

Type 4: Diagnose Problem

Common Questions:

  • Facebook's user engagement drops by x%, what would you do?
  • Lyft's ETA increased by x%, what would you do?

Methodology:

  • Discuss Sudden drop vs. static drop. Sudden drop可能有是由某些内部或外部因素引起。Static drop可能由市场，季节引起的。
  • 其他产品。用户的变化可能由内部类似的产品引起的， 或者是别的公司的竞争产品引起的。
  • 细分产品。可以讨论哪些细分类别变化了，那些没变。
  • 细分用户。可以讨论哪些用户群体变化了，那些没变。例如地区，语言，平台。
  • 最后总结已经做一个推荐。

YouTube Resources

Here are some YouTube videos for those who prefer visual content over reading. Note that product sense content is rare on YouTube, and most are mock interviews which may not be very helpful. Here are some useful ones I've found: