Definition
(Estimator). Suppose a fixed parameter \(\theta\) needs to be estimated. An estimator is a function that maps a sample space to a set of sample estimates. An estimator of \(\theta\) is typically denoted \(\hat \theta\). If \(X\) is a random variable corresponding to the observed data, the estimator (itself treated as a random variable) is \(\hat\theta(X)\). More concretely, if you have a relationship
\begin{align*} y = f(x) + \varepsilon \end{align*}where \(f\) is a deterministic function and \(\varepsilon\) is some random error term, and \(D = \{(\hat x_i, \hat y_i)\}\) is some data, then an estimator \(\hat f\) for \(f\) would be a function determined by some learning process. The learning process itself can be viewed as a function
\begin{align*} \{D = \{(x_i, y_i)\} ~ \mid ~ |D| < \infty\} \to \Hom(x, y). \end{align*}Common things used to evaluate an estimator or a class of estimators are the bias and variance of an estimator.