Hopefully this is the right board for my question, We are facing an issue creating a unique ID from the following inputs :

  • datetime (can be present more than once in the same set, cannot not be present in 2 different sets)
  • offset ( an integer representing each line position in the set, the offset will be reset when switching to next/previous set)

As output we would like to get a unique ID, which will be used for sorting purpose later.

The concrete example for this issue is the rotating logs files. They will be collected into same database/table and later-on they will be shown and sorted/filtered based on this unique id.

Example :

  • log1.txt (set 1)

    • 2017-08-06 10:36:07,988 something (let say offset=54)
    • 2017-08-06 10:36:07,988 something (let say offset=108)
    • 2017-08-06 10:36:07,988 something (let say offset=162)
    • ...
    • 2017-08-06 10:40:06,488 something (let say offset=1200) => A
  • log2.txt (set 2)

    • 2017-08-06 10:40:06,788 xyz (let say offset=20)
    • ...
    • 2017-08-06 10:40:07,488 xyz (let say offset=200) => B
    • ...

Obviously a naïve uid function by adding offset as milliseconds to the date will lead to collusion, the frequency of the collusion cannot be foreseen, it depends on the logged text and the rotation size. (A & B for collusion example).


  • 2
    $\begingroup$ How about concatenating the two fields? $\endgroup$ – Yuval Filmus Aug 6 '17 at 20:55
  • $\begingroup$ you are right :) ! ... but I made a small mistake below by assuming the same datetime cannot be present into more than one file. (anyhow, by adding the file name to the sorting criteria it works) $\endgroup$ – noob 86466 Aug 22 '17 at 17:03

You can create a unique identifier by concatenating the two fields in question. If there are files where both fields are identical, concatenate an extra field disambiguating them.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.