Data points as outcomes of a random experiment

Question

It is well known that

Random variable is a function from sample space of a random experiment to $\mathbb{R}$.

Consider the following sentences from deep learning book

Let $\{x^{(1)}, \cdots , x^{(m)}\}$ be a set of $m$ independent and identically distributed data points. A point estimator or statistic is any function of the data: $$\hat{θ}_m = g(x(1), \cdots , x(m))$$. ........................................................................................................................................................... For now, we take the frequentist perspective on statistics. That is, we assume that the true parameter value $θ$ is fixed but unknown, while the point estimate $\hat{θ}$ is a function of the data. Since the data is drawn from a random process, any function of the data is random. Therefore $\hat{θ}$ is a random variable.

Now my doubts are

1) Input to a random variable is an outcome of a random experiment. Then is it true that each data point is an outcome of a random experiment? If yes, then what may be that random experiment and who are running it?

2) If dataset is a subset of or itself a sample space, then what is the reason for many statements stating every dataset has an underlying probability distribution? instead of saying that each dataset is a subset of a sample space of a random experiment?

3) Output of a random variable is a real number. But in general parameter can be a vector of real number. How to understand it?

Answer 1

It is well known that

Random variable is a function from sample space of a random experiment to ℝ.

Well-known, but false. Check out random variable on Wikipedia. We will not need the formal definition. Rather, a random variable is an object whose value is determined randomly. For example, consider a dice whose faces are labeled $a,b,c,d,e,f$. A dice throw is a random variable over $\{a,b,c,d,e,f\}$.

1) Input to a random variable is an outcome of a random experiment. Then is it true that each data point is an outcome of a random experiment? If yes, then what may be that random experiment and who are running it?

I'm not sure what "input to a random variable is an outcome of a random experiment" means. In your case, here is what happens. You are interested in solving some learning problem, say digit recognition for concreteness. The process providing your samples goes like this:

• choose a random person.
• choose a random digit.
• let the person draw that digit.
• return the drawing as well as the digit.

As to who is running this experiment, it is the people who created the dataset.

2) If dataset is a subset of or itself a sample space, then what is the reason for many statements stating every dataset has an underlying probability distribution? instead of saying that each dataset is a subset of a sample space of a random experiment?

The dataset is a bunch of samples drawn from the random experiment described above. It consists of $N$ pairs of drawings and digits.

3) Output of a random variable is a real number. But in general parameter can be a vector of real number. How to understand it?

In this case the random variable ranges over pairs (drawing, digit).

---

There is a slightly different way of looking at the creation of a dataset. We are given a large dataset, which we randomly divide into training and test. The random experiment in this case corresponds to this random partition. But I think that it's more useful to think of both training and test to be coming from the same underlying random experiment, which corresponds to the process we are trying to learn.