If I am correct, why do we never specify that binary search is linear in the number of bits, but we always specify that knapsack is exponential in the number of bits?
It depends on the perspective from which one tackle the problem. Often, when dealing with numbers, people tend to give more importance to the number of bits. Why is that ? Simply because numbers problem comes from their size (number of digits or bits). Such algorithm for finding out whether a number N is prime or not. It is quite easy to compute it for a 10 digit number, but harder as we go further.