0
$\begingroup$

I have a list of 256-bit binary data to store. Any algorithm for doing lossless compression on it in a way I can retrieve an entry by its index without decompressing the whole data (if possible). The decompressed data will also be stored as binary. I will also need to code this algorithm ground-up in a domain-specific language.

I'm trying to store 256-bit addresses in a blockchain smart-contract. I will need to later lookup addresses by index so, it'd be helpful if I can do that without full decompression.

Sample data with each entry pasted end to end here. I tried this site to compress using LSW and it seems I get a good reduction. Also here is another sample with each entry separated with line-break. The real data will contain from 2 to 65,000 entries.

Improve this question
$\endgroup$
11
  • 3
    $\begingroup$ I doubt that this will be answerable with the information provided. I think you need to provide more context, e.g., about the nature of the data. A trivial compression algorithm is "do nothing" and that will meet all of your stated requirements, so do you have some additional requirements, such as ensuring a certain level of compression or something? $\endgroup$
    – D.W. ♦
    Commented May 19, 2022 at 9:36
  • $\begingroup$ A hash code could do, but collisions are possible. What can you say about the sparseness of the data ? $\endgroup$
    – user16034
    Commented May 19, 2022 at 9:52
  • $\begingroup$ A binary trie can do the job, but will cost one pointer per entry. $\endgroup$
    – user16034
    Commented May 19, 2022 at 9:55
  • 1
    $\begingroup$ How significant was the reduction? And did it compress well because each entry itself compresses well or because the entries are somehow related? Can you share the entries? Seems hard to solve without being able to analyze them. $\endgroup$
    – Kelly Bundy
    Commented May 19, 2022 at 13:38
  • 1
    $\begingroup$ "seperated by a linebreak" btw sounds suspicious, as the entries themselves could already contain linebreaks. Unless you wrote them in hex or so, in which case that might explain the significant reduction. $\endgroup$
    – Kelly Bundy
    Commented May 19, 2022 at 13:42

2 Answers 2

Reset to default
1
$\begingroup$

You can't compress blockchain addresses (in any blockchain I am familiar with), so this is a dead end.

Improve this answer
$\endgroup$
2
  • $\begingroup$ @KellyBundy, I've revised my answer. I've tried to get the questioner to provide context but they have have unfortunately not been terribly successful. This is one of the problems with unclear/vague questions -- it is easy for answers to end up being not useful to the asker, and it can be hard to judge which answers will meet the criteria. $\endgroup$
    – D.W. ♦
    Commented May 24, 2022 at 19:35
  • $\begingroup$ Right, they could've provided better information, didn't really address my own questions/suggestions, either. If the question gave details for "seems I get a good reduction", we could at least show that/why that's mistaken... $\endgroup$
    – Kelly Bundy
    Commented May 24, 2022 at 19:42
0
$\begingroup$

Due to nature of data, just storing the data is better of than compressing it.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.