I have incoming stream of data which has following values. Country, City, State
Along with maximum number of unique values to store (e.g. 50,22,12). I need to design a system which will keep count of occurrences of above values and its sub-values. If there are more than maximum number of values then instead of storing actual value store ETC.
Country and count of occurrences States and count of occurrences Cities and count of occurrences
At defined time interval requirement is to calculate all counts along with subtype counts. e.g.
Country A 435, States B 264, City C 300 Country B 123, States B 825, City C 321 Country A 153, States B 122, City C 534
My system enforces following restrictions. 1. Can't use any database system to store and retrieve values. 2. For speed and use of simplicity this needs to be in memory (hence limit on maximum values stored) 3. Values are cleared at particular interval. 4. These values are nested, I can't use separate tables to keep track of 3 fields separately.
I have thought about using custom data structure which stores nested values and its count. What other ways I can achieve following design. Which data structures can be used? Are there any existing examples which does similar things?
Note : This is not a homework assignment, I am developing my own application and I am stuck at designing elegant solution for this requirement. If you think this question is too general or inappropriate for StackOverflow, before closing or voting it down leave remark which forum should be used to ask this question. Any positive feedback appreciated.