Disclaimer: First of all I've had a lot of trouble finding the right stack exchange site for this question and I don't know if it's correct here either.
The Problem: I'm working on a search algorithm for a django powered website which allows people to write articles on various subjects and search for articles on the webpage.
I've got a database in which I store articles for people to read on a website. I'm storing each article with a bit of information like:
Articles:
- A title.
- A predetermined category where the writer can choose from when adding an article (sports, economy, politics).
Tags which the writer of the article can input themselves.
Length of the article in characters
- A short description
- Dated added
- Date updated
I've a few easy ways a user can search an article based on the title ( for example shortest first or newest first)
I'm now working on adding a way to search based on popularity but I have a hard time to determine what I define as popular.
I'm saving article views in a different database table which contains:
Views:
- User ID
- article ID
- Timestamp
Each row inside this table is considered 1 view so I can see exactly when certain articles are doing well in terms of amount of views it gets. If I would search for an article with the most views in the database it would look something like this in django:
Article.objects.order_by('-views')[:30]
In this case views is stored in the table, However I store it in a different table so I can have the functionality described above.
The problem however is that if I have 1 article which is a few years old with a high number of views, for example lets say there has been a sports event (like the Olympic games of 2014) and a lot of people have read the article but the article is no longer very relevant as this same event has been done again (that would be the Olympic games of 2018 happening this year).
Now lets say someone searches for popular articles about the Olympic games. Even though the articles about the Olympic games of 2014 have gathered more views than those of 2018 (as they have been written roughly 4 years ago and therefor have had more time to gather views)
In this case the more relevant articles should be presented to the user before the more popular articles.
My Potential Solution:
I'm thinking of adding a column to the article table for memorizing the 'importance' of each article, then I can have Importance be determined by other factors
for example if I want to have the amount of views be a factor for each view importance gets a bit higher, but if the view has been within the last week it might count as 2 views. therefor favoring more actual articles slightly more over articles with more views but written a few years ago. this also allows me to for example measure the time each reader spends on an article and have that influence importance as well.
However I'm not sure if this would be the best way of doing things as there are a few problems like:
If importance is stored in the database it has to be updated regularly and periodically which may cause performance problems whenever the user accesses an article while the database is updating it's articles importance.
Calculating the importance in real time every time a user doe a search will also take a huge amount of processing power if the amount of users is very high, it will also probably cause a lot of redundancy whenever 2 users search for the same articles and the exact same importance factor is calculated twice.
How would I go about implementing this way of sorting the search results without needing lots of processing power? Is this the best way to do it? How do other search engines handle this problem?
