why estimate the estimator function ??

• when the input data is available, but the output target variable ain’t available as easily
  • imputing tasks, like filling missing values
• or in inference, that is understanding how the estimators are related to the estimatee
  • what part of the estimators contribute most to the estimation of a target variable
  • in this case we aren’t concerned with the internals of the estimator function, instead we are only concerned with how it works
    • we are also interested in how many features are really playing a substantial part in predicting the target
    • is the relationship complex or linear ??
    • how is target related to each predictor feature ?

Irreducible and reducible error

• the estimator function always carries in it some irreducible error
  • which can be attributed to the unmeasured variables that are useful in predicting the target variable
  • or some particular variation in the specimen, which is practically impossible to model because of it’s dynamic unpredictable nature

how to estimate the estimator, given we appreciate the importance of the estimator function

Parametric methods

• assuming the basic structure of the function being predicted
• the data may not follow the regime, we began our analysis with

Non Parametric methods

• don’t make an assumption about the underlying relationship between the features and target
• since this takes no assumptions, the number of estimations sky rocket

Interpretability of a particular approach of modeling

• we still choose more restrictive models, because they offer good interpretability which is required in inference mode of the modelling agenda
• flexible methods like boosting or svms, maybe useful when we are’t interested in the underlying relationships but we want to predict with highest accuracy and make a model for a very niche category of problem

How to choose from a mirage of methods ?

• fundamentally, we should choose the model which fits bests to the test data
• but then a problem arises that we don’t always have the test results to compare our predictions to