Data Scientist Interview Questions: Test the Conclusion, Not Just the Model

Data scientist interview questions often leave a crucial detail unstated: what decision will the analysis support? Before calculating a metric or choosing an algorithm, clarify the population, time frame, outcome, and action the result is supposed to inform.

The following practice cases are original. They are intended to reveal reasoning, not to predict a particular employer's interview format.

A classifier is 99% accurate. Is that good?

Not enough information. In a fictional dataset of 1,000 cases, suppose only 10 are positive. Predicting negative for everyone gives 990 correct answers, or 99% accuracy, while finding none of the positives.

Now consider another classifier:

Outcome	Count
Correctly flagged positive cases	8
Negative cases incorrectly flagged	12
Positive cases missed	2
Correctly unflagged negative cases	978

Its precision is 8 out of 20 flagged cases, or 40%. Its recall is 8 out of 10 actual positives, or 80%. Its accuracy is 986 out of 1,000, or 98.6%.

The lower accuracy does not by itself make it worse. Ask about the cost of missed positives, false alarms, and reviewing flagged cases. These counts alone do not establish that the model should be deployed.

Why might a model perform suspiciously well?

Check whether training used information unavailable at the intended prediction time. Also examine duplicates and how preprocessing was fitted relative to the train/test split.

Scikit-learn's common-pitfalls guide explains how leakage can inflate evaluation results and why fitting transformations on test data is a problem. For example, a future cancellation reason is not a legitimate input to a prediction made before the customer cancels.

Describe the unit that must remain separate across evaluation sets. If the same customer's repeated records appear on both sides, ask whether the test still represents the intended new-customer use case. The appropriate split follows the question, not a reflexive random split.

How would you handle missing values?

Investigate what missing means. A blank survey response, a failed sensor reading, and a field that does not apply are not necessarily the same condition.

Explain the proposed treatment and its possible distortion. If you fill values using a statistic learned from data, keep that learning inside the training process. Check whether the missingness itself changed between the data used for development and the intended use.

A treatment group improved. Did the intervention cause it?

Ask how the groups were formed, what else changed, and whether the outcome was specified before inspecting results. A before-and-after comparison alone may not separate the intervention from seasonality or a different population.

An illustrative answer:

I would report the observed difference, then identify what the design lets us infer. If assignment was not randomized, I would not describe the difference as a proven treatment effect. I would check plausible alternative explanations and propose a design that better addresses the decision if the cost of being wrong warrants it.

Avoid pretending that adding more decimal places resolves a design limitation.

How do you communicate an inconclusive result?

State what was measured, what remains uncertain, and what decision is still possible. "We did not establish a difference" is not automatically "there is no difference." Explain what additional evidence would be worth collecting, rather than recommending endless analysis by default.

Use data analyst interview questions for reporting fundamentals and technical interview preparation to practice making your assumptions audible. A strong answer can be conditional without becoming evasive.