1
$\begingroup$

A lot of the work in data science -- machine learning in particular --involves "reshaping" arrays, tables, and other data structures. These data transformations are not really lossy transformations (except for subsets/selective views), but rather simply getting existing data into a different form acceptable to existing algorithms.

This seems to be a big step backward in computer science, away from data abstraction. In particular, it seems polymorphism has somehow been forgotten in all the work geared toward morphing the data into the proper form.

Am I missing something about this problem that renders it particularly immune to the application of mature technologies to hide data representation from the abstract functions?

$\endgroup$
2
  • $\begingroup$ Are you able to articulate in the question what alternative approach you think would be workable and would count as mature and a step forward? $\endgroup$
    – D.W.
    Nov 18 '20 at 6:25
  • $\begingroup$ Do you have concrete examples? SQL is quite successful in abstracting data in such a way that it can be pulled out easily in complicated forms. $\endgroup$
    – Joppy
    Nov 18 '20 at 10:37
1
$\begingroup$

Am I missing something about this problem that renders it particularly immune to the application of mature technologies to hide data representation from the abstract functions?

I would say so. And that is empirical experience of what works.

In theory data is data. A general artificial intelligence agent should be able to classify a picture as a cat or a dog regardless of whether it's a photo, a drawing, in color or black/white, altered through an Instagram filter to have a human face, a PNG, JPEG or MP4, compressed with zip, or infinite other possibilities.

But that's purely theory. In practice data representation matters, a lot. The very same task your machine learning method gets state of the art on might not learn at all if you change the data representation. In practice feature engineering is a huge part of any machine learning project. And the existence and importance of this task should indicate that with current techniques it's not feasible to even begin to think of hiding the data representation.


As an example, take the recent success of GPT-3 and transformers. The entire novelty of this technique comes from the way data is represented and learned from inside the neural network.

Another example, AlexNet. It popularized the now standard deep convolutional neural network. A convolutional neural network only really 'works' on (multiple slices of) 2D data. Any other representation and the convolution stops making any sense at all.

In state of the art machine learning, data representation is everything.

$\endgroup$
1
  • $\begingroup$ Actually data representation matters a lot no matter what the application since it drives resource consumption. But I get your point: Normal programming involves manually constructed models but machine learning involves manually constructed meta models (most generally, program synthesis as in algorithmic information approaches to model selection). So really we need to come up with better terminology addressing this meta level. Then I can reword my question. There must be literature on this involving model theory. $\endgroup$ Nov 27 '20 at 15:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.