I am new in this field but as I understand Kolmogorov complexity is non-computable, but the Shannon entropy is computable.
The definition of Shannon entropy is:
$${\displaystyle \mathrm {H} (X):=-\sum _{x\in {\mathcal {X}}}p(x)\log p(x)}$$
Here are a few examples to help me explain my question:
$1.)\text{ Random numbers}$
So now for a string of random numbers between 0 and 9, $s=3752893864299238764826$ the Shannon entropy is
$$n\in[0,9]$$
$$p(n)=1/10$$
$$H = - \sum_{i=0}^{9} \left( \frac{1}{10} \log_2 \frac{1}{10} \right) = \log_2 10 = 3.32 \text{ bits per digit}$$
Its Kolmogorov complexity $K(s)$ is $$s=3752893864299238764826$$ $$K(s)\approx 22$$
approximately, because the string is random and it has $22$ characters.
$2.)\text{ String of "ab", i.e. a system that has some degree of order}$
For a string "abababababab"
$$p(a)=0.5$$
$$p(b)=0.5$$
So the Shannon entropy is:
$$H = -\sum_{i=1}^{2} \left( \frac{1}{2} \log_2 \frac{1}{2} \right)=1 \text{ bit}$$
but its Kolmogorov complexity is
$$s=abababababab$$ $$K(s) \approx 4$$
because, say, a program can write "abx6" which is a string that has lenght $4$.
$3.) \text{ String of "a", i.e. totally ordered system}$
Now for a totally ordered system "aaaaaaaa", its Shannon entropy is:
$$H = - \sum_{i=1}^{n} p_i \log_2 p_i=- (1 \cdot \log_2 1) = 0$$
and its Kolmogorov complexity is pretty low also:
$$K(s)\approx 3$$
because we can write a program : "ax8"
So we can conclude that as the Shannon entropy of a system decreases (a system is more ordered), its Kolmogorov complexity decreases and on the other hand, as the Shannon entropy of a system increases, the Kolmogorov complexity approaches asymptotically the length of the string of the system itself.
$$K(s) = \begin{cases} \lim\limits_{H(s) \to 0} k & \text{if } H(s) \to 0,\\ \lim\limits_{H(s) \to \infty} |s| & \text{if } H(s) \to \infty. \end{cases}$$
where $k$ is some constant that is smallest quantity that is needed to describe the system (because Kolmogorov complexity can not be exactly $0$) and $|s|$ is the length of the string (system).
My question is can we use this kind of thinking to further refine the bounds of Kolmogorov complexity and any insight on the appropriate literature would be really appreciated.
