Am I missing something about this problem that renders it particularly immune to the application of mature technologies to hide data representation from the abstract functions?
I would say so. And that is empirical experience of what works.
In theory data is data. A general artificial intelligence agent should be able to classify a picture as a cat or a dog regardless of whether it's a photo, a drawing, in color or black/white, altered through an Instagram filter to have a human face, a PNG, JPEG or MP4, compressed with zip, or infinite other possibilities.
But that's purely theory. In practice data representation matters, a lot. The very same task your machine learning method gets state of the art on might not learn at all if you change the data representation. In practice feature engineering is a huge part of any machine learning project. And the existence and importance of this task should indicate that with current techniques it's not feasible to even begin to think of hiding the data representation.
As an example, take the recent success of GPT-3 and transformers. The entire novelty of this technique comes from the way data is represented and learned from inside the neural network.
Another example, AlexNet. It popularized the now standard deep convolutional neural network. A convolutional neural network only really 'works' on (multiple slices of) 2D data. Any other representation and the convolution stops making any sense at all.
In state of the art machine learning, data representation is everything.