You don't really need any complex machine learning algorithms for this. I mean, it would be cool to use it and try to guess what questions would stump them with AI, but I think you could get away with something simple.
Consider the following strategy:
- Questions are given tags that identify the categories they fall
under. You can use multiple tags to identify broader categories and more specific concepts.
- Your system tracks the users percentage in each tag.
- Optionally: The user selects some broad categories they want to study right now
- Your system selects random questions from categories the user has a low success rate with
- Occasionally the system breaks from that strategy to ask questions in categories it has a low confidence with, for example categories that the user has never tried. For example, it may follow a random selection that weights your worst categories highly, your low confidence categories moderately, and categories it know you are good at are weighted the least.
This AI doesn't really use any complex machine learning algorithms, but it gets the job done, probing the users abilities, and focusing on their weaknesses.