50 Startups: Multiple Linear Regression & Feature Selection Explorer

Feature Selection & Parsimony Analysis

Compare 5 Machine Learning Feature Selection Algorithms (Lasso, Forward SFS, RFE, ElasticNet, Exhaustive Search) against Baseline OLS. Explore the bias-variance tradeoff across feature counts $k=1 \dots 5$, pinpointing the 2-feature optimal sweet spot and the 5-feature overfitting cliff.

50 Startups Dataset 5 Selection Methods Global Sweet Spot: k=2 ($8,198.80 RMSE) Overfitting Warning: k=5 ($9,137.99 RMSE)

Test RMSE Optimal

$8,198.80

Lower is better (Best across methods)

R² Score 94.7% Fit

0.9474

Explains 94.74% of profit variance

Adjusted R²

0.9324

Penalizes redundant parameters

Mean Absolute Error

$6,488.75

Average prediction error ($)

F-Statistic p < 0.0001

347.8

Model is highly statistically significant

Active Predictors (2):

R&D Spend Marketing Spend

Occam's Razor: 2-feature model minimizes out-of-sample error.

RMSE by Number of Features

Lower is better. Sharp valley at k=2, jump at k=5.

Lasso (L1)

R-squared by Number of Features

Higher is better. Peaks at k=2 (0.9474), drops at k=5.

Peak at k=2

Model Coefficients & Statistical Tests

Detailed parameter estimates, standard errors, t-statistics, and VIF multicollinearity.

Predictor	Estimate (β)	Std. Error	t-Stat	p-Value	95% Conf. Int	VIF

Live Profit Predictor

Test predictions in real-time with the active model.

Real-time

R&D Spend ($) $100,000

Marketing Spend ($) $150,000

Administration ($) $120,000

State Location

Predicted Startup Profit

$130,154.20

Calculated using: β₀ + β₁(R&D) + β₂(Marketing)

Test Set Residuals & Outliers Diagnostic

Top absolute residuals on the out-of-sample test set (10 startups).

Sorted by |Residual|

Rank	Sample ID	Actual Profit	Predicted Profit	Residual ($)	Abs Error	% Error	Diagnostic Status