Little Bits of Artificial Intelligence

A. Basic Pointers for Learning Algorithm Selection

1. Type of Problem

• Classification: Predicting categories (e.g., spam vs. not spam) → Logistic Regression, Decision Trees, Random Forest, SVM, etc.
• Regression: Predicting continuous values (e.g., house prices) → Linear Regression, Ridge, Lasso, etc.
• Clustering: Grouping similar data (unsupervised) → K-Means, DBSCAN.
• Sequence/Time Series: Predicting over time → ARIMA, LSTM, RNN.
• Recommendation: Personalized ranking → Collaborative Filtering, Matrix Factorization.

2. Size and Nature of Data

• Small Dataset:
• Use simpler models (e.g., Logistic/Linear Regression, Naive Bayes).
• Avoid deep learning unless augmented data is available.
• Large Dataset:
• We can use complex models like Random Forests, Gradient Boosting, Neural Networks.
• High Dimensionality (many features):
• Use algorithms with regularization (e.g., Lasso, Ridge).
• Consider feature selection or dimensionality reduction (PCA).
• Sparse Data (many missing or zero values):
• Use Naive Bayes, Tree-based models, or apply imputation techniques.

3. Interpretability vs Accuracy

• Need for Interpretability (e.g., healthcare, finance):
• Use Logistic Regression, Decision Trees, Rule-based models.
• Need for Accuracy / Less Interpretability:
• Go for Ensemble models (Random Forest, XGBoost) or Neural Networks.

4. Training Time & Resources

• Limited resources:
• Prefer lightweight models like Logistic Regression, SVM (with small data), Naive Bayes.
• Access to GPU / Cloud:
• Use Deep Learning, Transformer-based models, etc.

5. Data Characteristics

• Linearly Separable: Use Linear Models (Logistic/Linear Regression, Linear SVM).
• Non-linear relationships: Use Kernel SVM, Tree-based models, Neural Networks.
• Noise-sensitive: Avoid overfitting models (e.g., prefer Random Forest over Decision Tree).

6. Overfitting vs Underfitting Behavior

• Start with simple model → analyze training/validation accuracy.
• If underfitting → go for complex model.
• If overfitting → use regularization, cross-validation, or reduce model complexity.

7. Business Requirements

• Speed vs Precision
• Real-time prediction needs fast models.
• Offline batch predictions can use heavier models.
• Cost of wrong prediction: Impacts model choice & evaluation metric (e.g., F1 vs Accuracy).

B. Example: Practical Mapping