1
$\begingroup$

I am looking to create a neural network architecture that takes two kinds of inputs: image and text, and outputs a predicted continuous variable. More specifically, I will have multiple images of a house as an input alongside numerical/text data like square meters, number of beds, baths, carspaces, latitude, longitude, features etc. I want to use this to output the predicted price value of the given house.

I understand you can combine image and text inputs as seen here:

enter image description here

Could I use a similar model for the situation I described? If so, what kind of neural network architecture should I use for the image and numeric/text input streams, and how would I combine them?

$\endgroup$

1 Answer 1

1
$\begingroup$

Yes, that is a reasonable architecture. You could use any architecture for processing images, and have it output a representation of the image (take any classification architecture, and remove the last layer). You could also use any architecture for text, and have it output a representation of the text. Finally, for other inputs, you could use a fully connected architecture. Then, concatenate those representations and run them through another fully connected network. That would be a reasonable architecture to try.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.