Have you ever spent hours cleaning a dataset, launched a model, and got a disappointing score? The problem is almost never the algorithm. It's how you presented the data. We, at Meteora Web, see this every day in projects we receive: features built at random, inconsistent scales, missing values handled with a simple drop. Machine learning is math, not magic. Feature engineering is the bridge between raw data and signals a model can exploit. Here we don't give you abstract theory: we show you how to turn columns into performance levers, with real code, concrete examples, and metrics in hand.
This guide is part of our pillar on Machine Learning with Python — Business-Ready Models, where we combine technical expertise and economic vision. Because good feature engineering costs time upfront, but returns real margins.
Why is feature engineering critical in machine learning?
A model is an engine: feed it dirty fuel, it stalls. Features are the fuel. A well-built variable can shift R² by 0.1 or more. Time spent here is not a cost — it's the highest investment you can make. In one of our projects for an e-commerce client, a feature that crossed day of week and purchase hour increased sales forecast accuracy by 15%. The algorithm was the same. The data was the same. The difference? How we packaged it.
Sponsored Protocol
Feature engineering helps:
- Transform non-linear variables into linear signals (e.g., log-transformation)
- Reduce noise (e.g., binning, smoothing)
- Inject domain knowledge (e.g., ratio between two variables)
- Allow simple models to compete with complex ones
How to handle missing values effectively?
Dropping rows with NaN is a luxury few can afford, especially with small or expensive datasets. But filling with the mean is not always right either. The choice depends on the nature of the data and why it's missing. Here are three targeted strategies.
Distribution-based imputation
If data is missing completely at random (MCAR), use mean or median. For skewed distributions, median is more robust. In Python:
import pandas as pd
import numpy as np
df['age'].fillna(df['age'].median(), inplace=True)Imputation by category
If you have a categorical variable and a numeric one with NaN, compute median per group:
Sponsored Protocol
df['income'] = df.groupby('region')['income'].transform(lambda x: x.fillna(x.median()))Missing indicator feature
Add a binary column marking where data was missing. The model can learn that absence itself is a signal:
df['age_missing'] = df['age'].isna().astype(int)
df['age'].fillna(df['age'].median(), inplace=True)Why it pays off: you recover rows that still carry information on other variables, and you give the model a chance to learn the relationship between absence and target.
How to encode categorical variables for best performance?
One-hot encoding works for few categories. With 50+ values, dimensionality explodes. Alternatives exist and work.
Target encoding
Replace each category with the mean of the target for that group. Warning: risk of overfitting. Use smoothing or cross-validation. Here's a simple version:
from sklearn.model_selection import KFold
import numpy as np
def target_encoding(df, feature, target, k=5):
kf = KFold(n_splits=k, shuffle=True, random_state=42)
encoded = np.empty(df.shape[0])
for train_idx, val_idx in kf.split(df):
X_train, X_val = df.iloc[train_idx], df.iloc[val_idx]
means = X_train.groupby(feature)[target].mean()
encoded[val_idx] = X_val[feature].map(means).values
return encodedOrdinal Encoding with real order
If categories have an ordering (e.g., 'low', 'medium', 'high'), map to integers:
Sponsored Protocol
order_map = {'low': 1, 'medium': 2, 'high': 3}
df['priority'] = df['priority'].map(order_map)Frequency encoding
Replace each category with its frequency. Useful for high-cardinality variables:
freq = df['product_code'].value_counts()
df['product_code_freq'] = df['product_code'].map(freq)How to create interaction and polynomial features?
Linear models don't capture interactions between variables. If you suspect the combined effect of age and income is more than the sum, you must create it explicitly.
Sponsored Protocol
Binary interactions
df['age_income'] = df['age'] * df['income']Polynomials with scikit-learn
from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2, interaction_only=False, include_bias=False)
X_poly = poly.fit_transform(df[['age', 'income']])
# generates also age², income², age*incomeCaution: with many features, polynomials explode. Use only where you have evidence of non-linearity (EDA or domain knowledge).
How to scale and normalize features properly?
Distance-based algorithms (SVM, k-NN, PCA) are sensitive to scale. Tree-based models are not. But even when not required by the algorithm, scaling can help convergence (neural networks) or regularization.
Standardization (Z-score)
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
df_scaled = scaler.fit_transform(df[['age', 'income']])Min-Max scaling
from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
df_scaled = scaler.fit_transform(df[['age', 'income']])Robust scaling
Resistant to outliers: uses median and IQR:
Sponsored Protocol
from sklearn.preprocessing import RobustScaler
scaler = RobustScaler()
df_scaled = scaler.fit_transform(df[['age', 'income']])When to use what: StandardScaler if data is already normal. MinMax if you have physical bounds (e.g., percentages). Robust if there are outliers you don't want to amplify.
How to leverage domain knowledge to create high-impact features?
The difference between a data scientist and an engineer who knows business is this. We, at Meteora Web, come from accounting. When working on a sales forecast for a clothing store, we don't just look at past sales. We build features like: ratio of average inventory to sales, seasonality based on real sale dates, stock-out indicators. Here's a practical example:
# Feature: days since end of sale (rebound effect)
df['days_since_sale_end'] = (pd.to_datetime('2026-01-31') - df['date']).dt.days
df['post_sale'] = (df['days_since_sale_end'] >= 0) & (df['days_since_sale_end'] <= 30)Why it pays off: a feature that encapsulates a business pattern is worth a thousand automatic variables. The model learns from you, not just numbers.
How to automate feature engineering with libraries like Feature-engine?
After mastering manual techniques, automation is an accelerator. Feature-engine is a Python library that integrates many techniques with an sklearn interface.
from feature_engine.encoding import TargetEncoder
from feature_engine.imputation import MeanMedianImputer
from feature_engine.creation import MathFeatures
# Pipeline
imp = MeanMedianImputer(imputation_method='median', variables=['age'])
enc = TargetEncoder(variables=['region'])
math = MathFeatures(variables=['height', 'weight'], func=['ratio', 'product'])
from sklearn.pipeline import Pipeline
pipe = Pipeline([('imputer', imp), ('encoder', enc), ('math', math)])
df_transformed = pipe.fit_transform(df, df['target'])Other useful libraries: category_encoders for advanced encoding (CatBoost, LeaveOneOut), tsfresh for time series, featuretools for deep feature synthesis.
Remember: automation doesn't replace reasoning. We use it to speed up, but every generated feature must be validated with business.
What to do now
You have the tools. Now put them into practice:
- Analyze your dataset: check missing data, outliers, distributions with pandas profiling or ydata-profiling.
- Choose a non-trivial imputation strategy: for each column evaluate whether to use median, mean, or a predictive model.
- Encode categoricals: if you have high cardinality, try target encoding with cross-validation.
- Create at least 3 domain features: ask yourself what your business knows that the dataset doesn't show explicitly.
- Test the impact: compare model performance with and without the new features. Document improvements.
And if you want concrete support on your project, contact us. We, at Meteora Web, turn raw data into competitive advantage, since 2017.
