Common Frameworks for Data Science Cases

 

Having a toolkit of frameworks helps you structure your thinking quickly. Here are the most useful ones for data science case interviews.

 

The AARRR Framework (Pirate Metrics)

 

AARRR is a funnel framework that tracks the user journey through five stages. It's essential for product and growth cases.

 

• Acquisition measures how users find your product. Key metrics include app downloads, website visits, sign-ups, and unique visitors by channel

 

• Activation measures whether users experience value on their first visit. Key metrics include completing onboarding, using a core feature, or reaching a defined moment

 

• Retention measures whether users come back. Key metrics include DAU/MAU ratio, cohort retention curves, and churn rate

 

• Referral measures whether users recommend your product. Key metrics include invite rates, referral conversion, and viral coefficient

 

• Revenue measures whether users pay. Key metrics include conversion to paid, average revenue per user, and lifetime value

 

When asked about product metrics, walk through each stage of the funnel and identify what metrics matter most at each step.

 

The Success/Guardrail/Health Framework

 

This framework helps you define a complete set of metrics for any product decision.

 

• Success metrics measure whether you achieved your goal. These are your primary KPIs. Examples include conversion rate, engagement time, or revenue

 

• Guardrail metrics ensure you're not causing unintended harm. These should not degrade even if your success metrics improve. Examples include page load time, unsubscribe rate, or customer complaints

 

• Health metrics track the overall state of the product regardless of any specific change. Examples include DAU, session frequency, and error rates

 

When proposing metrics for any case, always include all three types. This shows you think about second-order effects.

 

The Segmentation Framework

 

This framework helps you investigate metric changes by breaking down data into meaningful groups.

 

• User segments include new vs returning, free vs paid, power users vs casual, and different demographic groups

 

• Product segments include different features, platforms (iOS vs Android vs web), and content types

 

• Geographic segments include country, region, and market maturity

 

• Time segments include day of week, time of day, and before vs after specific events

 

When a metric changes, systematically check if the change is consistent across segments or concentrated in specific groups. This helps isolate the root cause.

 

The Internal/External Framework

 

This framework helps categorize potential causes when investigating problems.

 

• Internal factors are things your company controls. These include product changes, bug introductions, algorithm updates, marketing campaigns, and pricing changes

 

• External factors are things outside your control. These include competitor actions, seasonality, economic conditions, news events, and platform changes

 

Always check both categories when investigating a metric change. Internal factors are usually easier to verify using your release calendar and internal documentation.

 

The CIRCLES Framework

 

Originally from product management, CIRCLES works well for product improvement and feature design cases.

 

• Comprehend the situation by clarifying the question and confirming your understanding

 

• Identify the customer by defining who the product serves

 

• Report customer needs by listing the problems users face

 

• Cut by prioritizing which needs matter most

 

• List solutions that address the prioritized needs

 

• Evaluate tradeoffs between your proposed solutions

 

• Summarize your recommendation and rationale

 

How to Solve Data Science Cases

 

Having a repeatable approach helps you stay organized under pressure. Here's a five-step method that works for most data science cases.

 

Step 1: Clarify

 

Before you do anything else, make sure you understand the problem. Ask questions to fill in the gaps.

 

• What's the business context?
• What decision will this analysis inform?
• Are there any constraints to know about?

 

Repeat the problem back in your own words. This confirms you understood correctly and gives the interviewer a chance to correct any misunderstandings.

 

Identify what success looks like.

 

• How will we know if we've solved the problem?
• What would a good answer look like?

 

Don't skip this step. Candidates who dive straight into analysis often solve the wrong problem.

 

Step 2: Structure

 

Break the problem into manageable pieces.

 

Start by outlining your approach at a high level.

 

• What are the main components of the problem?
• What questions do you need to answer?

 

Organize your approach into a clear framework with buckets and sub-questions. This could be a list of hypotheses to test, a funnel to analyze, or a set of factors to consider. The specific structure depends on the problem.

 

Share your structure with the interviewer before proceeding. This gives them a chance to redirect you if needed and shows that you think before you act.

 

Step 3: Analyze

 

Work through your framework systematically.

 

For each component, explain your reasoning as you go.

 

• What data would you need?
• What method would you use?
• What assumptions are you making?

 

Do calculations when required, but don't get lost in arithmetic. Round numbers aggressively and focus on whether your answer makes directional sense.

 

Connect each piece back to the bigger picture. How does this analysis help answer the original question?

 

Step 4: Conclude

 

Synthesize your findings into a clear recommendation.

 

State your answer directly. Don't bury the lead or hedge excessively. Interviewers want to see you take a stance.

 

Support your recommendation with the key reasons. Two or three strong reasons are better than a laundry list.

 

Acknowledge limitations and uncertainties. What would you want to investigate further? What could change your recommendation?

 

Step 5: Adapt

 

Be ready to pivot when the interviewer pushes back or introduces new information.

 

Follow-up questions are part of the test. The interviewer wants to see how you think on your feet and whether you can incorporate new constraints.

 

If your approach isn't working, don't be afraid to step back and try something different. Flexibility shows maturity.

 

Stay engaged and collaborative. The best case interviews feel like a conversation between colleagues working on a problem together.

 

Data Science Case Interview Examples with Solutions

 

Let's walk through three data science case interview examples to see how they can be solved. 

 

Example 1: Product Analytics Case

 

Case Prompt: You're a data scientist at Instagram. The product team notices that the share rate for Stories has dropped 12% month over month. How would you investigate this?

 

Step 1: Clarify

 

Before investigating, I'd want to understand the context better.

 

• Is this drop happening across all users or specific segments?
• Is it global or concentrated in certain regions?
• Did anything change recently like a product update or a major external event?

 

I'd also confirm the metric definition. Share rate presumably means the percentage of Stories that get shared, but I want to make sure we're measuring this consistently over time.

 

Let's assume the interviewer confirms that share rate equals shares divided by Stories created, and that the drop appears broad-based in initial reports.

 

Step 2: Structure

 

I'll organize my investigation into three main buckets, each with specific sub-questions to answer.

 

Metric Decomposition

 

• Has the numerator (shares) decreased, or has the denominator (Stories created) increased?
• Are fewer users sharing, or are the same users sharing less frequently?
• At which step in the share flow are users dropping off?

 

Segmentation Analysis

 

• User segments: Is this affecting new users, existing users, or both? Power users vs casual users?
• Platform segments: Is the drop consistent across iOS, Android, and web? Any differences by app version?
• Geographic segments: Is this global or concentrated in certain countries or regions?
• Content segments: Are certain types of Stories (video vs photo, filtered vs unfiltered) affected more?

 

Potential Causes

 

• Internal factors: Any recent product changes? Bug introductions? Algorithm updates to Story distribution?
• External factors: Competitor launches? Seasonal effects? Major news events affecting user behavior?
• Technical factors: Performance issues? Increased latency in the share flow?

 

Step 3: Analyze

 

Let me work through each bucket.

 

For metric decomposition, let's say the data shows that Stories created is stable, but total shares dropped. This tells us the problem is in the sharing behavior, not content creation.

 

Looking deeper at the share funnel, imagine we find that share attempts are flat but share completions dropped significantly. Users are trying to share but something is preventing them from finishing.

 

For segmentation analysis, suppose we find the drop is concentrated among Android users and is most severe in emerging markets like India, Brazil, and Indonesia. The drop is minimal on iOS and in North America. This pattern immediately narrows our focus.

 

For potential causes, given the Android and emerging markets pattern, I'd hypothesize this could be related to a recent app update, network connectivity issues, or increased file sizes making shares slower on low-bandwidth connections.

 

I'd cross-reference the timing of the drop with our release calendar. If a new Android version rolled out two weeks ago and the drop started then, that's a strong signal.

 

Step 4: Conclude

 

Based on this analysis, my hypothesis is that a recent Android update introduced a performance regression that's causing share completions to fail, particularly in regions with slower internet connections.

 

“I'd recommend three immediate actions. 

 

First, the engineering team should investigate the share flow in the recent Android release, specifically looking at timeout thresholds and performance on low-bandwidth connections. 

 

Second, we should compare share completion rates between users on the new version versus users who haven't updated yet. 

 

Third, if we confirm the issue, we should consider rolling back or hotfixing the problematic change.”

 

Step 5: Adapt

 

If the interviewer said the timing doesn't match any releases, I'd pivot to investigating whether the content being shared has changed.

 

Perhaps Stories now include larger files or different formats that take longer to upload. I'd also look into whether any third-party dependencies like social sharing APIs changed their behavior recently.

 

If the interviewer said the drop is actually consistent across all platforms, I'd revisit my segmentation analysis and look more carefully at user-level patterns.

 

Maybe a change to the share UI made the button less visible, or perhaps Instagram changed its content recommendation algorithm in a way that surfaces content users are less likely to share.

 

Example 2: Root Cause Analysis Case

 

Case Prompt: You're a data scientist at a food delivery company. The CEO walks in and says "our conversion rate dropped 8% yesterday. What's going on?" How do you investigate?

 

Step 1: Clarify

 

First, I need to understand what we mean by conversion rate. 

 

• Is this the percentage of app visitors who place an order?
• Does it include only new users or all users?
• Are we comparing yesterday to the same day last week, or to a rolling average?

 

I'd also want to know if this is outside normal daily variance. An 8% drop sounds large, but if our baseline has high day-to-day variability, it might not be unusual.

 

Let's assume the interviewer confirms this is visitor-to-order conversion for all users, and that 8% is about three standard deviations below our typical daily average.

 

Step 2: Structure

 

I'll organize my investigation into four buckets.

 

Funnel Breakdown

 

• Where in the conversion funnel are users dropping off?
• App open to restaurant browse: Are users making it past the home screen?
• Restaurant browse to restaurant select: Are users finding restaurants they want?
• Restaurant select to cart add: Are users adding items?
• Cart add to checkout start: Are users proceeding to checkout?
• Checkout start to payment: Are users completing payment?

 

Segmentation

 

• User type: New users vs returning users? First-time orderers vs repeat customers?
• Platform: iOS vs Android vs web? Any differences by app version?
• Geography: Specific cities or regions affected?
• Restaurant type: Certain cuisines or price points affected more?
• Time pattern: Was the drop sudden or gradual throughout the day?

 

Internal Factors

 

• Product changes: Any releases, feature flags, or A/B tests launched?
• Pricing changes: Any changes to delivery fees, service fees, or promotions?
• Supply issues: Any problems with restaurant availability or driver supply?
• Technical issues: Any outages, high latency, or error rates?

 

External Factors

 

• Competitor activity: Did a competitor launch a major promotion?
• Weather: Any unusual weather affecting ordering patterns?
• Events: Any holidays, local events, or news that might affect behavior?
• Payment systems: Any issues with credit card processors or payment providers?

 

Step 3: Analyze

 

Let me work through each bucket.

 

For the funnel breakdown, suppose we find that the drop is concentrated between checkout start and payment completion. Earlier stages look normal. Users are browsing, selecting restaurants, and adding items at typical rates. They're starting checkout but not completing orders.

 

For segmentation, let's say the drop is consistent across user types and geographies but significantly worse on iOS. Android and web show only a small decline.

 

For internal factors, I'd check our release calendar. Suppose we pushed an iOS update two days ago. I'd also verify that no pricing or fee changes went live.

 

For external factors, there were no major weather events, holidays, or known competitor actions. But I'd check if Apple made any changes to Apple Pay or if our payment provider reported any issues.

 

Given the iOS concentration and the payment step, the evidence points to a payment processing issue specific to iOS.

 

Step 4: Conclude

 

My leading hypothesis is that there's a payment processing bug in our recent iOS release. The evidence supporting this includes the funnel showing problems at payment completion, the iOS concentration, and the timing aligning with our recent iOS update.

 

“There are four things that I’d recommend: 

 

First, pull payment error rates by platform over the last 72 hours and look for a spike on iOS starting when the update rolled out.

 

Second, if error rates confirm the hypothesis, escalate to engineering immediately to investigate the payment integration in the new release.

 

Third, consider rolling back the iOS update or pushing a hotfix.

 

Fourth, estimate the revenue impact by calculating orders lost times average order value.”

 

Step 5: Adapt

 

If the interviewer revealed that payment error rates are normal, I'd pivot to investigating the checkout experience itself.

 

Maybe a UI change in the iOS update made the payment button less prominent or introduced a confusing flow. I'd pull up screenshots of the checkout flow from before and after the update to compare.

 

If the interviewer said the issue is actually consistent across all platforms, I'd reconsider external factors.

 

Perhaps our primary payment processor had a partial outage, or maybe a large credit card issuer flagged our transactions for some reason. I'd also look at whether we ran any promotions that expired, which might cause users to abandon checkout when they expected a discount that's no longer available.

 

Example 3: Machine Learning Case

 

Case Prompt: You're a data scientist at a bank. The risk team wants to build a model to predict which credit card applicants will default within their first year. How would you approach this?

 

Step 1: Clarify

 

I'd want to understand the business context first.

 

• What's the current process for evaluating applicants?
• Are we trying to replace human underwriters or augment them?
• What's the cost of approving someone who defaults versus rejecting someone who would have been a good customer?

 

I'd also ask about constraints.

 

• Are there regulatory requirements about what features we can use?
• Do we need to explain individual decisions to applicants who are denied?

 

Let's assume the interviewer says we're augmenting human underwriters, that we must be able to explain decisions, and that the cost of a default is about 10x the profit from a good customer.

 

Step 2: Structure

 

I'll organize my approach into five buckets covering the full ML lifecycle.

 

Problem Definition

 

• What exactly are we predicting? How is "default" defined (missed payments for how long)?
• What's the prediction timeframe? Default within 12 months of account opening?
• What's the decision point? At application time before any account history exists?

 

Data Assessment

 

• What historical data do we have on past applicants?
• What outcome data shows whether approved applicants defaulted?
• What's the selection bias risk? We only have outcomes for people we approved.
• How far back does our data go? Is it representative of current conditions?

 

Feature Engineering

 

• Applicant demographics: Age, income, employment status, time at job
• Credit history: Credit score, existing debt, payment history, credit utilization
• Application details: Requested credit limit, card type, acquisition channel
• External data: Economic indicators, regional unemployment rates

 

Model Selection and Training

 

• What algorithms fit our explainability requirement?
• How do we handle class imbalance if defaults are rare?
• What validation strategy accounts for temporal patterns?
• How do we avoid overfitting to historical approval decisions?

 

Evaluation and Deployment

 

• What metrics capture the business tradeoffs?
• How do we ensure fairness across protected groups?
• What's the deployment strategy? Shadow mode first?
• How do we monitor performance over time?

 

Step 3: Analyze

 

Let me work through each bucket.

 

For problem definition, I'd confirm with the risk team that we're predicting 90+ days past due within the first 12 months, evaluated at the point of application before any account behavior exists.

 

For data assessment, we'd use historical application data including income, employment, credit score, and existing debt. We'd join this with outcome data showing which approved applicants defaulted. A key challenge is selection bias. We only have outcomes for applicants we approved. 

 

Rejected applicants might have been good customers, but we can't know. I'd ask how consistent our approval criteria have been historically and whether we've done any randomized approval experiments.

 

For feature engineering, I'd start with standard credit features like credit score, debt-to-income ratio, and payment history. I'd also engineer features like credit utilization trend and number of recent credit inquiries. For applicants with thin credit files, I'd consider alternative data sources if permitted.

 

For model selection, given the explainability requirement, I'd use either logistic regression or a gradient boosted tree model like XGBoost. Both can achieve strong predictive performance while allowing us to explain which factors drove each decision. I'd use time-based cross-validation to prevent leakage from future information.

 

For evaluation, I'd focus on metrics that capture the business tradeoff. Given that defaults cost 10x what good customers earn, I'd optimize for a threshold where the cost of false negatives (approving defaults) is balanced against false positives (rejecting good customers). I'd also audit performance across demographic groups to ensure the model doesn't discriminate.

 

Step 4: Conclude

 

“My recommendation has four parts:

 

First, build a gradient boosted tree model using application features and credit bureau data, with time-based validation to simulate real deployment conditions.

 

Second, optimize the decision threshold based on the 10x cost asymmetry. This means we should be relatively conservative and accept some false positives to avoid costly defaults.

 

Third, build an explanation system that shows underwriters which features drove each prediction. This satisfies the explainability requirement and helps underwriters override the model when they have additional context.

 

Fourth, deploy in shadow mode for 3-6 months, running the model alongside current processes to evaluate real-world performance before making live decisions.”

 

Step 5: Adapt

 

If the interviewer asked how I'd handle the selection bias problem, I'd discuss several approaches. 

 

We could use reject inference techniques to estimate outcomes for rejected applicants based on similar approved applicants. We could also weight our training data to account for the selection process. Ideally, we'd advocate for a small randomized experiment where we approve a sample of applicants who would normally be rejected, but I'd acknowledge this has ethical implications.

 

If the interviewer asked what I'd do if the model performed worse for certain demographic groups, I'd discuss fairness interventions.

 

We could adjust thresholds by group to equalize false positive or false negative rates. We could remove features that proxy for protected characteristics. We could also use fairness-constrained training algorithms. The right approach depends on which fairness definition the business and regulators prioritize.